Files
regorus/bindings/csharp/API.md
Anand Krishnamoorthi cc917ea75d feat: Complete target system with C# bindings and resource inference (#458)
* feat: Add Schema Registry and Validation Framework

This commit introduces a comprehensive schema registry and validation framework, providing schema-based validation of resources and policy effects.

- Thread-safe, in-memory registry for schema storage and management
- Global registry patterns for effects and resources
- Concurrent access with proper error handling
- Unicode schema names support

- JSON Schema-compliant validation for all primitive types
- Advanced constraint validation (patterns, ranges, length limits)
- Discriminated union support with anyOf schemas
- Detailed error reporting with nested validation paths
- Discriminated subobject validation for polymorphic schemas

- **Registry Tests**: All registry operations
- **Effect Tests**: Policy effect validation
- **Resource Tests**: Resource validation
- **Validation Tests**: Core validation engine
- Thread-safety, error handling, integration scenarios, edge cases

- **Dependencies**: dashmap, once_cell, regex
- **Thread Safety**: Minimal locking with Rc<Schema> sharing
- **Error Types**: TypeMismatch, OutOfRange, PatternMismatch, etc.

- Complete schema registry and validation subsystem
- Comprehensive test coverage
- Foundation for policy validation in Regorus

Benchmarks:

- Criterion benchmarks for basic types, effects and Azure resources
- Performance range: 3.22ns (string) to 34.74µs (Azure VM resource schema validation)
- String withs patterns validation: 30.2µs. Need to explore whether regex caching helps
  bring this down.
- Azure policy effects: 188ns-1.4µs

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

* feat: Complete target system with C# bindings and resource inference

- Add comprehensive target system with TargetRegistry and target-aware compilation
- Implement resource type inference from policy equality expressions
- Create modular C# bindings with separate wrapper classes for each concept
- Add thread-safe CompiledPolicy with reference counting for safe disposal
- Enhance FFI with detailed error propagation and target functionality
- Create TargetExampleApp demonstrating Azure Policy integration
- Add CI/CD pipeline testing for all C# applications
- Support target definitions with schema validation and resource selectors
- Implement PolicyModule struct and target-aware compilation methods
- Add comprehensive test coverage for target functionality

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>

---------

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
2025-08-19 20:23:43 -05:00

15 KiB

Regorus C# API Documentation

This document describes the C# API for Regorus, focusing on the compiled policy approach for high-performance policy evaluation.

Overview

The Regorus C# bindings provide a modern, thread-safe API for compiling and evaluating Open Policy Agent (OPA) Rego policies. The API is designed around pre-compiled policies that can be evaluated efficiently multiple times with different inputs.

Architecture

┌─────────────────────────────────────────────────────────────────┐
│                    CompiledPolicy Workflow                      │
└─────────────────────────────────────────────────────────────────┘

┌─────────────────┐    ┌──────────────────┐    ┌─────────────────┐
│  Policy Modules │    │   Target/Schema  │    │   Static Data   │
│  (.rego files)  │    │    Registries    │    │     (JSON)      │
└─────────┬───────┘    └────────┬─────────┘    └─────────┬───────┘
          │                     │                        │
          └─────────────────────┼────────────────────────┘
                                │
                                ▼
                    ┌─────────────────────────┐
                    │        Compile          │
                    │  ┌─────────────────────┐│
                    │  │ Parse & Analyze     ││
                    │  │ Infer Resource Types││
                    │  │ Build AST & Rules   ││
                    │  │ Target Integration  ││
                    │  └─────────────────────┘│
                    └─────────────┬───────────┘
                                  │
                                  ▼
                      ┌─────────────────────────┐
                      │    CompiledPolicy       │
                      │ ┌─────────────────────┐ │
                      │ │ AST & Rules         │ │
                      │ │ Target Info         │ │
                      │ │ Resource Types      │ │
                      │ │ Function Table      │ │
                      │ │ Compiled Modules    │ │
                      │ └─────────────────────┘ │
                      └─────────────┬───────────┘
                                    │
                                    ▼
                          ┌─────────────────────┐
                          │    Service Cache    │
                          │ (Policy Framework,  │
                          │  MS Graph, etc.)    │
                          │ ┌─────────────────┐ │
                          │ │ CompiledPolicy  │ │ ◄─── Same LOCK-FREE policy
                          │ │    (cached)     │ │      instance shared across
                          │ └─────────────────┘ │      all threads
                          └─────────┬───────────┘
                                    │
                            ┌───────┼───────┬───────┐
                            │       │       │       │
                            ▼       ▼       ▼       ▼
                    ┌─────────────┐ ┌─────────────┐ ┌─────────────┐
                    │  Thread 1   │ │  Thread 2   │ │  Thread N   │
                    │             │ │             │ │             │
                    │ input1 ────▶│ │ input2 ────▶│ │ inputN ────▶│
                    │ ◄─── result │ │ ◄─── result │ │ ◄─── result │
                    └─────────────┘ └─────────────┘ └─────────────┘

┌─────────────────────────────────────────────────────────────────┐
│                        Key Benefits                             │
├─────────────────────────────────────────────────────────────────┤
│ ✓ Compile Once, Evaluate Many     ✓ Lock-Free Concurrent Eval   │
│ ✓ No Re-parsing Overhead          ✓ Reference Counting Safety   │
│ ✓ Reduced GC Pressure             ✓ Proper Resource Management  │
│ ✓ Cache-Friendly Design           ✓ Target System Integration   │
└─────────────────────────────────────────────────────────────────┘

Key Features

  • Pre-compiled Policies: Compile once, evaluate many times for optimal performance
  • Target System Support: Built-in support for Azure Policy targets with resource type inference
  • Thread Safety: All operations are thread-safe without external synchronization
  • Registry Management: Centralized management of targets and schemas
  • Policy Introspection: Rich metadata about compiled policies

Core Classes

CompiledPolicy

The CompiledPolicy class represents a pre-compiled Rego policy that can be evaluated efficiently.

public sealed class CompiledPolicy : IDisposable
{
    // Evaluate the policy with input data
    public string? EvalWithInput(string inputJson);
    
    // Get comprehensive policy metadata
    public PolicyInfo GetPolicyInfo();
    
    // Dispose of unmanaged resources
    public void Dispose();
}

Thread Safety: All methods are thread-safe. Multiple threads can call EvalWithInput() concurrently, and Dispose() will safely wait for active evaluations to complete.

Compiler

The Compiler class provides static methods for compiling policies.

public static class Compiler
{
    // Compile a policy with a specific entrypoint rule
    public static CompiledPolicy CompilePolicyWithEntrypoint(
        string dataJson, 
        IEnumerable<PolicyModule> modules, 
        string entryPointRule);
    
    // Compile a target-aware policy (requires azure_policy feature)
    public static CompiledPolicy CompilePolicyForTarget(
        string dataJson, 
        IEnumerable<PolicyModule> modules);
}

PolicyModule

Represents a single policy module to be compiled. Each PolicyModule corresponds to a Rego file (.rego), and each Rego file defines a Rego package using the package declaration at the top of the file.

public struct PolicyModule
{
    public string Id { get; set; }
    public string Content { get; set; }
    
    public PolicyModule(string id, string content);
}

Properties:

  • Id: A unique identifier for the module, typically the filename (e.g., "policy.rego", "rules/storage.rego")
  • Content: The complete Rego policy content, including the package declaration and all rules

Example:

var module = new PolicyModule("storage-policy.rego", @"
    package azure.storage
    import rego.v1
    
    default allow := false
    allow if input.type == ""Microsoft.Storage/storageAccounts""
");

PolicyInfo

Provides comprehensive metadata about a compiled policy.

public class PolicyInfo
{
    // List of module identifiers
    public List<string> ModuleIds { get; set; }
    
    // Target name (for target-aware policies)
    public string? TargetName { get; set; }
    
    // Resource types this policy can evaluate
    public List<string> ApplicableResourceTypes { get; set; }
    
    // Primary rule/entrypoint
    public string EntrypointRule { get; set; }
    
    // Effect rule (for target-aware policies)
    public string? EffectRule { get; set; }
    
    // Policy parameters
    public List<PolicyParameters> Parameters { get; set; }
}

Registry Classes

TargetRegistry

Manages target definitions for Azure Policy-style evaluations.

public static class TargetRegistry
{
    // Register a target from JSON
    public static void RegisterFromJson(string targetJson);
    
    // Check if a target exists
    public static bool Contains(string name);
    
    // List all registered targets
    public static string ListNames();
    
    // Remove a target
    public static bool Remove(string name);
    
    // Clear all targets
    public static void Clear();
    
    // Get count of registered targets
    public static int Count { get; }
    
    // Check if registry is empty
    public static bool IsEmpty { get; }
}

SchemaRegistry

Manages schema definitions for validation.

public static class SchemaRegistry
{
    // Register resource schemas
    public static void RegisterResourceSchema(string name, string schemaJson);
    public static bool ContainsResourceSchema(string name);
    public static string ListResourceSchemas();
    
    // Register effect schemas
    public static void RegisterEffectSchema(string name, string schemaJson);
    public static bool ContainsEffectSchema(string name);
    public static string ListEffectSchemas();
    
    // Clear methods
    public static void ClearResourceSchemas();
    public static void ClearEffectSchemas();
}

Usage Examples

Basic Policy Compilation and Evaluation

// Define policy modules
var modules = new List<PolicyModule>
{
    new PolicyModule("policy.rego", @"
        package example
        import rego.v1
        
        default allow := false
        allow if input.user == ""admin""
    ")
};

// Compile the policy
using var policy = Compiler.CompilePolicyWithEntrypoint("{}", modules, "data.example.allow");

// Evaluate with different inputs
var result1 = policy.EvalWithInput(@"{""user"": ""admin""}");  // true
var result2 = policy.EvalWithInput(@"{""user"": ""guest""}");  // false

Target-Aware Policy (Azure Policy Style)

// Register target definition
TargetRegistry.RegisterFromJson(@"{
    ""name"": ""azure.storage"",
    ""resource_schema_selector"": ""type"",
    ""resource_types"": {
        ""Microsoft.Storage/storageAccounts"": {
            ""schema"": { /* JSON Schema */ }
        }
    }
}");

// Define policy with target
var modules = new List<PolicyModule>
{
    new PolicyModule("policy.rego", @"
        package policy
        import rego.v1
        
        __target__ := ""azure.storage""
        
        default effect := ""deny""
        effect := ""allow"" if {
            input.type == ""Microsoft.Storage/storageAccounts""
            input.properties.supportsHttpsTrafficOnly == true
        }
    ")
};

// Compile for target
using var policy = Compiler.CompilePolicyForTarget("{}", modules);

// Evaluate Azure resource
var resource = @"{
    ""type"": ""Microsoft.Storage/storageAccounts"",
    ""properties"": {
        ""supportsHttpsTrafficOnly"": true
    }
}";

var result = policy.EvalWithInput(resource);  // "allow"

Policy Introspection

// Get policy metadata
var info = policy.GetPolicyInfo();

Console.WriteLine($"Target: {info.TargetName}");
Console.WriteLine($"Effect Rule: {info.EffectRule}");
Console.WriteLine($"Modules: {string.Join(", ", info.ModuleIds)}");
Console.WriteLine($"Resource Types: {string.Join(", ", info.ApplicableResourceTypes)}");

// Access parameters
if (info.Parameters != null && info.Parameters.Count > 0)
{
    foreach (var parameterSet in info.Parameters)
    {
        Console.WriteLine($"Module: {parameterSet.SourceFile}");
        foreach (var param in parameterSet.Parameters)
        {
            Console.WriteLine($"Parameter: {param.Name} ({param.Type})");
            if (param.Default != null)
                Console.WriteLine($"  Default: {param.Default}");
        }
    }
}

Concurrent Evaluation

// CompiledPolicy is thread-safe
var tasks = Enumerable.Range(0, 100).Select(i => 
    Task.Run(() => policy.EvalWithInput($@"{{""id"": {i}}}"))
).ToArray();

var results = await Task.WhenAll(tasks);

Performance Considerations

Compilation Overhead

  • Policy compilation has significant overhead due to parsing and analysis
  • Best Practice: Compile once, reuse many times
  • Consider caching compiled policies for repeated use

Memory Management

  • CompiledPolicy manages unmanaged resources
  • Always dispose of compiled policies using using statements or explicit Dispose()
  • Disposal is thread-safe and waits for active evaluations

Thread Safety

  • All classes are thread-safe for concurrent reads/evaluations
  • Registry modifications should be done during initialization
  • No external synchronization required

Error Handling

All methods throw Exception on errors with descriptive messages:

try
{
    var policy = Compiler.CompilePolicyWithEntrypoint(data, modules, rule);
    var result = policy.EvalWithInput(input);
}
catch (Exception ex)
{
    Console.WriteLine($"Error: {ex.Message}");
}

Feature Flags

Some functionality requires specific Rust feature flags:

  • azure_policy: Required for target-aware compilation and policy parameters
  • Without this feature, target-related methods will not be available

Version Compatibility

  • Requires .NET Standard 2.0 or later
  • Compatible with .NET Framework 4.6.1+, .NET Core 2.0+, .NET 5+
  • Uses System.Text.Json for JSON serialization (added as dependency)

Best Practices

  1. Compile Once, Evaluate Many: Pre-compile policies for repeated evaluation
  2. Use Disposable Pattern: Always dispose of CompiledPolicy instances
  3. Thread-Safe Design: Take advantage of built-in thread safety
  4. Registry Setup: Configure targets and schemas during application startup
  5. Error Handling: Wrap operations in try-catch blocks for robust error handling
  6. Performance Monitoring: Monitor evaluation times for performance optimization

Migration from Engine-Based API

If migrating from an engine-based approach:

// Old approach (if it existed)
// var engine = new Engine();
// engine.AddPolicy("policy.rego", policyContent);
// engine.SetInputJson(inputJson);
// var result = engine.EvalRule("data.policy.allow");

// New compiled approach
var modules = new[] { new PolicyModule("policy.rego", policyContent) };
using var policy = Compiler.CompilePolicyWithEntrypoint("{}", modules, "data.policy.allow");
var result = policy.EvalWithInput(inputJson);

The compiled approach provides better performance for repeated evaluations and clearer resource management.