kata-containers/regorus
1
0
Fork 0
mirror of https://github.com/microsoft/regorus.git synced 2026-08-05 02:16:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat: Complete target system with C# bindings and resource inference (#458)

Browse Source 
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* feat: Complete target system with C# bindings and resource inference

- Add comprehensive target system with TargetRegistry and target-aware compilation
- Implement resource type inference from policy equality expressions
- Create modular C# bindings with separate wrapper classes for each concept
- Add thread-safe CompiledPolicy with reference counting for safe disposal
- Enhance FFI with detailed error propagation and target functionality
- Create TargetExampleApp demonstrating Azure Policy integration
- Add CI/CD pipeline testing for all C# applications
- Support target definitions with schema validation and resource selectors
- Implement PolicyModule struct and target-aware compilation methods
- Add comprehensive test coverage for target functionality

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This commit is contained in:
Anand Krishnamoorthi
2025-08-19 20:23:43 -05:00
committed by GitHub
parent 3c33d31d08
commit cc917ea75d
71 changed files with 10278 additions and 1000 deletions
Download Patch File Download Diff File Expand all files Collapse all files

421
bindings/csharp/API.md Normal file
View File

@@ -0,0 +1,421 @@
# Regorus C# API Documentation
This document describes the C# API for Regorus, focusing on the compiled policy approach for high-performance policy evaluation.
## Overview
The Regorus C# bindings provide a modern, thread-safe API for compiling and evaluating Open Policy Agent (OPA) Rego policies. The API is designed around pre-compiled policies that can be evaluated efficiently multiple times with different inputs.
## Architecture
```
┌─────────────────────────────────────────────────────────────────┐
│ CompiledPolicy Workflow │
└─────────────────────────────────────────────────────────────────┘
┌─────────────────┐ ┌──────────────────┐ ┌─────────────────┐
│ Policy Modules │ │ Target/Schema │ │ Static Data │
│ (.rego files) │ │ Registries │ │ (JSON) │
└─────────┬───────┘ └────────┬─────────┘ └─────────┬───────┘
│ │ │
└─────────────────────┼────────────────────────┘
│
▼
┌─────────────────────────┐
│ Compile │
│ ┌─────────────────────┐│
│ │ Parse & Analyze ││
│ │ Infer Resource Types││
│ │ Build AST & Rules ││
│ │ Target Integration ││
│ └─────────────────────┘│
└─────────────┬───────────┘
│
▼
┌─────────────────────────┐
│ CompiledPolicy │
│ ┌─────────────────────┐ │
│ │ AST & Rules │ │
│ │ Target Info │ │
│ │ Resource Types │ │
│ │ Function Table │ │
│ │ Compiled Modules │ │
│ └─────────────────────┘ │
└─────────────┬───────────┘
│
▼
┌─────────────────────┐
│ Service Cache │
│ (Policy Framework, │
│ MS Graph, etc.) │
│ ┌─────────────────┐ │
│ │ CompiledPolicy │ │ ◄─── Same LOCK-FREE policy
│ │ (cached) │ │ instance shared across
│ └─────────────────┘ │ all threads
└─────────┬───────────┘
│
┌───────┼───────┬───────┐
│ │ │ │
▼ ▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ Thread 1 │ │ Thread 2 │ │ Thread N │
│ │ │ │ │ │
│ input1 ────▶│ │ input2 ────▶│ │ inputN ────▶│
│ ◄─── result │ │ ◄─── result │ │ ◄─── result │
└─────────────┘ └─────────────┘ └─────────────┘
┌─────────────────────────────────────────────────────────────────┐
│ Key Benefits │
├─────────────────────────────────────────────────────────────────┤
│ ✓ Compile Once, Evaluate Many ✓ Lock-Free Concurrent Eval │
│ ✓ No Re-parsing Overhead ✓ Reference Counting Safety │
│ ✓ Reduced GC Pressure ✓ Proper Resource Management │
│ ✓ Cache-Friendly Design ✓ Target System Integration │
└─────────────────────────────────────────────────────────────────┘
```
## Key Features
- **Pre-compiled Policies**: Compile once, evaluate many times for optimal performance
- **Target System Support**: Built-in support for Azure Policy targets with resource type inference
- **Thread Safety**: All operations are thread-safe without external synchronization
- **Registry Management**: Centralized management of targets and schemas
- **Policy Introspection**: Rich metadata about compiled policies
## Core Classes
### CompiledPolicy
The `CompiledPolicy` class represents a pre-compiled Rego policy that can be evaluated efficiently.
```csharp
public sealed class CompiledPolicy : IDisposable
{
// Evaluate the policy with input data
public string? EvalWithInput(string inputJson);
// Get comprehensive policy metadata
public PolicyInfo GetPolicyInfo();
// Dispose of unmanaged resources
public void Dispose();
}
```
**Thread Safety**: All methods are thread-safe. Multiple threads can call `EvalWithInput()` concurrently, and `Dispose()` will safely wait for active evaluations to complete.
### Compiler
The `Compiler` class provides static methods for compiling policies.
```csharp
public static class Compiler
{
// Compile a policy with a specific entrypoint rule
public static CompiledPolicy CompilePolicyWithEntrypoint(
string dataJson,
IEnumerable<PolicyModule> modules,
string entryPointRule);
// Compile a target-aware policy (requires azure_policy feature)
public static CompiledPolicy CompilePolicyForTarget(
string dataJson,
IEnumerable<PolicyModule> modules);
}
```
### PolicyModule
Represents a single policy module to be compiled. Each PolicyModule corresponds to a Rego file (.rego), and each Rego file defines a Rego package using the `package` declaration at the top of the file.
```csharp
public struct PolicyModule
{
public string Id { get; set; }
public string Content { get; set; }
public PolicyModule(string id, string content);
}
```
**Properties:**
- `Id`: A unique identifier for the module, typically the filename (e.g., "policy.rego", "rules/storage.rego")
- `Content`: The complete Rego policy content, including the `package` declaration and all rules
**Example:**
```csharp
var module = new PolicyModule("storage-policy.rego", @"
package azure.storage
import rego.v1
default allow := false
allow if input.type == ""Microsoft.Storage/storageAccounts""
");
```
### PolicyInfo
Provides comprehensive metadata about a compiled policy.
```csharp
public class PolicyInfo
{
// List of module identifiers
public List<string> ModuleIds { get; set; }
// Target name (for target-aware policies)
public string? TargetName { get; set; }
// Resource types this policy can evaluate
public List<string> ApplicableResourceTypes { get; set; }
// Primary rule/entrypoint
public string EntrypointRule { get; set; }
// Effect rule (for target-aware policies)
public string? EffectRule { get; set; }
// Policy parameters
public List<PolicyParameters> Parameters { get; set; }
}
```
## Registry Classes
### TargetRegistry
Manages target definitions for Azure Policy-style evaluations.
```csharp
public static class TargetRegistry
{
// Register a target from JSON
public static void RegisterFromJson(string targetJson);
// Check if a target exists
public static bool Contains(string name);
// List all registered targets
public static string ListNames();
// Remove a target
public static bool Remove(string name);
// Clear all targets
public static void Clear();
// Get count of registered targets
public static int Count { get; }
// Check if registry is empty
public static bool IsEmpty { get; }
}
```
### SchemaRegistry
Manages schema definitions for validation.
```csharp
public static class SchemaRegistry
{
// Register resource schemas
public static void RegisterResourceSchema(string name, string schemaJson);
public static bool ContainsResourceSchema(string name);
public static string ListResourceSchemas();
// Register effect schemas
public static void RegisterEffectSchema(string name, string schemaJson);
public static bool ContainsEffectSchema(string name);
public static string ListEffectSchemas();
// Clear methods
public static void ClearResourceSchemas();
public static void ClearEffectSchemas();
}
```
## Usage Examples
### Basic Policy Compilation and Evaluation
```csharp
// Define policy modules
var modules = new List<PolicyModule>
{
new PolicyModule("policy.rego", @"
package example
import rego.v1
default allow := false
allow if input.user == ""admin""
")
};
// Compile the policy
using var policy = Compiler.CompilePolicyWithEntrypoint("{}", modules, "data.example.allow");
// Evaluate with different inputs
var result1 = policy.EvalWithInput(@"{""user"": ""admin""}"); // true
var result2 = policy.EvalWithInput(@"{""user"": ""guest""}"); // false
```
### Target-Aware Policy (Azure Policy Style)
```csharp
// Register target definition
TargetRegistry.RegisterFromJson(@"{
""name"": ""azure.storage"",
""resource_schema_selector"": ""type"",
""resource_types"": {
""Microsoft.Storage/storageAccounts"": {
""schema"": { /* JSON Schema */ }
}
}
}");
// Define policy with target
var modules = new List<PolicyModule>
{
new PolicyModule("policy.rego", @"
package policy
import rego.v1
__target__ := ""azure.storage""
default effect := ""deny""
effect := ""allow"" if {
input.type == ""Microsoft.Storage/storageAccounts""
input.properties.supportsHttpsTrafficOnly == true
}
")
};
// Compile for target
using var policy = Compiler.CompilePolicyForTarget("{}", modules);
// Evaluate Azure resource
var resource = @"{
""type"": ""Microsoft.Storage/storageAccounts"",
""properties"": {
""supportsHttpsTrafficOnly"": true
}
}";
var result = policy.EvalWithInput(resource); // "allow"
```
### Policy Introspection
```csharp
// Get policy metadata
var info = policy.GetPolicyInfo();
Console.WriteLine($"Target: {info.TargetName}");
Console.WriteLine($"Effect Rule: {info.EffectRule}");
Console.WriteLine($"Modules: {string.Join(", ", info.ModuleIds)}");
Console.WriteLine($"Resource Types: {string.Join(", ", info.ApplicableResourceTypes)}");
// Access parameters
if (info.Parameters != null && info.Parameters.Count > 0)
{
foreach (var parameterSet in info.Parameters)
{
Console.WriteLine($"Module: {parameterSet.SourceFile}");
foreach (var param in parameterSet.Parameters)
{
Console.WriteLine($"Parameter: {param.Name} ({param.Type})");
if (param.Default != null)
Console.WriteLine($" Default: {param.Default}");
}
}
}
```
### Concurrent Evaluation
```csharp
// CompiledPolicy is thread-safe
var tasks = Enumerable.Range(0, 100).Select(i =>
Task.Run(() => policy.EvalWithInput($@"{{""id"": {i}}}"))
).ToArray();
var results = await Task.WhenAll(tasks);
```
## Performance Considerations
### Compilation Overhead
- Policy compilation has significant overhead due to parsing and analysis
- **Best Practice**: Compile once, reuse many times
- Consider caching compiled policies for repeated use
### Memory Management
- `CompiledPolicy` manages unmanaged resources
- **Always** dispose of compiled policies using `using` statements or explicit `Dispose()`
- Disposal is thread-safe and waits for active evaluations
### Thread Safety
- All classes are thread-safe for concurrent reads/evaluations
- Registry modifications should be done during initialization
- No external synchronization required
## Error Handling
All methods throw `Exception` on errors with descriptive messages:
```csharp
try
{
var policy = Compiler.CompilePolicyWithEntrypoint(data, modules, rule);
var result = policy.EvalWithInput(input);
}
catch (Exception ex)
{
Console.WriteLine($"Error: {ex.Message}");
}
```
## Feature Flags
Some functionality requires specific Rust feature flags:
- **azure_policy**: Required for target-aware compilation and policy parameters
- Without this feature, target-related methods will not be available
## Version Compatibility
- Requires .NET Standard 2.0 or later
- Compatible with .NET Framework 4.6.1+, .NET Core 2.0+, .NET 5+
- Uses System.Text.Json for JSON serialization (added as dependency)
## Best Practices
1. **Compile Once, Evaluate Many**: Pre-compile policies for repeated evaluation
2. **Use Disposable Pattern**: Always dispose of CompiledPolicy instances
3. **Thread-Safe Design**: Take advantage of built-in thread safety
4. **Registry Setup**: Configure targets and schemas during application startup
5. **Error Handling**: Wrap operations in try-catch blocks for robust error handling
6. **Performance Monitoring**: Monitor evaluation times for performance optimization
## Migration from Engine-Based API
If migrating from an engine-based approach:
```csharp
// Old approach (if it existed)
// var engine = new Engine();
// engine.AddPolicy("policy.rego", policyContent);
// engine.SetInputJson(inputJson);
// var result = engine.EvalRule("data.policy.allow");
// New compiled approach
var modules = new[] { new PolicyModule("policy.rego", policyContent) };
using var policy = Compiler.CompilePolicyWithEntrypoint("{}", modules, "data.policy.allow");
var result = policy.EvalWithInput(inputJson);
```
The compiled approach provides better performance for repeated evaluations and clearer resource management.