kata-containers/regorus
1
0
Fork 0
mirror of https://github.com/microsoft/regorus.git synced 2026-08-05 02:16:11 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat! Mimalloc as the default allocator (#434)

Browse Source 
This change integrates mimalloc as the default memory allocator for Regorus,
delivering significant performance improvements across all evaluation modes
and language bindings.

Technical Implementation:
- Build mimalloc in vendored mode from C sources (following QSharp approach)
- Implement GlobalAlloc trait for seamless Rust integration
- Add optional 'mimalloc' feature flag for conditional compilation
- Add comprehensive ACI benchmarks to measure evaluation performance

Performance Impact:

Rust Engine Evaluation:
- Single-threaded: ~29% improvement (423 vs 328 Kelem/s)
- Multi-threaded: Better scaling with reduced thread contention
- Fresh engines: ~24% improvement (56 vs 45 Kelem/s)

Rust Compiled Policy Evaluation:
- Single-threaded: ~41% improvement (426 vs 303 Kelem/s)
- Multi-threaded: Improved allocation efficiency under contention
- Fresh compilation: ~26% improvement (53 vs 42 Kelem/s)

C# FFI Bindings:
- Engine evaluation: ~27% improvement (279 vs 219 Kelem/s)
- Compiled policies: ~29% improvement (273 vs 211 Kelem/s)
- Better threading characteristics through improved underlying allocation

Key Benefits:
- Reduced allocation-related contention in multi-threaded scenarios
- More consistent performance across different thread counts
- Improved memory allocation efficiency for both native Rust and FFI workloads
- Better scaling characteristics for production deployments

The mimalloc integration provides substantial performance gains while
maintaining full compatibility with existing code through feature flags.

Reference: QSharp allocator implementation
(https://github.com/microsoft/qsharp/tree/main/source/allocator)

Fixes #297

Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This commit is contained in:
Anand Krishnamoorthi
2025-08-25 15:01:38 -05:00
committed by GitHub
parent 6c5338228b
commit 2a0b4ae6b5
59 changed files with 17039 additions and 328 deletions
Download Patch File Download Diff File Expand all files Collapse all files

135
bindings/csharp/Benchmarks/compiled_policy_evaluation_benchmark.md
View File

@@ -5,6 +5,7 @@
- **CPU**: 16 cores
- **Architecture**: ARM64 (aarch64-apple-darwin)
- **.NET Version**: 8.0
- **Allocator**: mimalloc (default allocator for Rust FFI)
- **Benchmark Framework**: Custom time-based benchmarking
- **Test Data**: 20,000 inputs per evaluation (distributed across threads)
- **Policy**: Complex authorization policy with nested rules
@@ -27,77 +28,113 @@ The C# compiled policy evaluation benchmark tests Regorus compiled policy perfor
### Compiled Shared Policies (Best Performance)
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|--------:|---------------------------:|---------------------:|
| 1 | 2928.81 | 211 |
| 2 | 5892.53 | 146 |
| 4 | 11750.71 | 155 |
| 6 | 17686.92 | 134 |
| 8 | 23543.53 | 90 |
| 10 | 29503.80 | 72 |
| 12 | 35494.81 | 58 |
| 14 | 41408.36 | 50 |
| 16 | 47333.65 | 44 |
| 18 | 53050.24 | 38 |
| 20 | 58807.20 | 34 |
| 22 | 406022.45 | 32 |
| 24 | 65480.69 | 32 |
| 26 | 70952.34 | 30 |
| 28 | 72064.03 | 30 |
| 30 | 492405.74 | 27 |
| 32 | 81210.83 | 27 |
| 1 | 2905.41 | 273 |
| 2 | 5808.07 | 240 |
| 4 | 11631.23 | 227 |
| 6 | 17431.95 | 216 |
| 8 | 23183.42 | 126 |
| 10 | 28886.11 | 118 |
| 12 | 34659.87 | 108 |
| 14 | 40564.07 | 84 |
| 16 | 46446.38 | 72 |
| 18 | 52047.06 | 63 |
| 20 | 56983.45 | 58 |
| 22 | 404931.47 | 55 |
| 24 | 61673.71 | 55 |
| 26 | 64370.41 | 51 |
| 28 | 56897.04 | 59 |
| 30 | 406850.06 | 52 |
| 32 | 56786.24 | 58 |
### Compiled Per Iteration
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|--------:|---------------------------:|---------------------:|
| 1 | 2984.00 | 39 |
| 2 | 5969.45 | 38 |
| 4 | 11948.28 | 32 |
| 6 | 17927.24 | 30 |
| 8 | 23889.01 | 24 |
| 10 | 29882.38 | 20 |
| 12 | 35865.06 | 18 |
| 14 | 41838.70 | 15 |
| 16 | 47800.92 | 14 |
| 18 | 53257.22 | 10 |
| 20 | 59596.93 | 11 |
| 22 | 435853.41 | 10 |
| 24 | 70870.86 | 9 |
| 26 | 76120.59 | 9 |
| 28 | 80717.51 | 8 |
| 30 | 544207.96 | 8 |
| 32 | 91540.91 | 7 |
| 1 | 2978.06 | 49 |
| 2 | 5965.09 | 47 |
| 4 | 11928.23 | 46 |
| 6 | 17892.58 | 45 |
| 8 | 23773.82 | 43 |
| 10 | 29705.61 | 42 |
| 12 | 35631.97 | 40 |
| 14 | 41563.35 | 34 |
| 16 | 47452.93 | 31 |
| 18 | 53505.42 | 27 |
| 20 | 59393.86 | 25 |
| 22 | 436115.28 | 23 |
| 24 | 71088.08 | 21 |
| 26 | 76928.70 | 19 |
| 28 | 82759.27 | 18 |
| 30 | 560658.97 | 17 |
| 32 | 93949.39 | 16 |
## Analysis
The C# compiled policy benchmark demonstrates important performance characteristics:
The C# compiled policy benchmark demonstrates important performance characteristics with mimalloc as the default allocator:
1. **Compilation Strategy Impact**: Shared compiled policies significantly outperform per-iteration compilation (~5.4x at 1 thread)
2. **Scaling Patterns**:
1. **Compilation Strategy Impact**: Shared compiled policies significantly outperform per-iteration compilation (~5.6x at 1 thread)
2. **Scaling Patterns with mimalloc**:
- Best throughput achieved at 1 thread for shared policies
- Performance generally degrades with increased thread count
- Performance generally degrades with increased thread count, but mimalloc provides better allocation efficiency
3. **Performance Hierarchy**:
- Shared compiled policies: Best performance (optimal configuration)
- Per-iteration compilation: ~82% reduction from optimal
4. **Compilation Overhead**: Per-iteration compilation creates substantial overhead, similar to fresh engine creation
5. **Thread Contention**: Significant performance degradation beyond 8 threads for both configurations
5. **Thread Contention**: Significant performance degradation beyond 8 threads for both configurations, though mimalloc helps mitigate some allocation-related issues
## Comparison with Rust Compiled Policy Evaluation
| Configuration | C# Performance (1 thread) | Rust Performance (1 thread) | Relative Performance |
|:-----------------|:----------------------------|:-----------------------------|---------------------:|
| Shared Policies | Best performance | Higher throughput | 0.40x-0.70x |
| Per-iteration | ~82% reduction from optimal | ~85% reduction from optimal | 0.47x-0.89x |
### Multi-Thread Performance Comparison
| Configuration | 1 Thread (Kelem/s) | 4 Threads (Kelem/s) | 8 Threads (Kelem/s) |
|:-----------------|:-------------------|:--------------------|:--------------------|
| | C# / Rust | C# / Rust | C# / Rust |
| Shared Policies | 273 / 426 | 227 / 342 | 126 / 185 |
| Per-iteration | 49 / 55 | 46 / 50 | 43 / 50 |
### Threading Efficiency Analysis
| Configuration | Low Contention (1-4t) | Medium Contention (6-12t) | High Contention (16+t) |
|:-----------------|:----------------------|:--------------------------|:-----------------------|
| | Avg C# / Rust | Avg C# / Rust | Avg C# / Rust |
| Shared Policies | 249 / 384 | 150 / 203 | 58 / 123 |
| Per-iteration | 47 / 54 | 40 / 50 | 22 / 42 |
**Key Observations:**
- **Single-threaded performance**: C# achieves 64% of Rust performance for shared policies, 89% for per-iteration
- **Threading scaling**: Both platforms show similar degradation patterns, but Rust maintains better absolute performance
- **Contention resistance**: Per-iteration compilation shows more consistent relative performance across thread counts
- **Platform differences**: C# shows more pronounced performance drops at higher thread counts, particularly for shared policies
*Note: Rust benchmarks include additional input data variations (cloned vs fresh inputs) that are not present in the C# implementation.*
## Comparison with C# Engine Evaluation
| Configuration | Compiled Policy (1 thread) | Engine Evaluation (1 thread) | Performance Ratio |
|:---------------|:----------------------------|:------------------------------|------------------:|
| Optimal Config | Best performance | Slightly higher throughput | 0.96x |
### Multi-Thread Performance Comparison
| Configuration | 1 Thread (Kelem/s) | 4 Threads (Kelem/s) | 8 Threads (Kelem/s) |
|:----------------|:-------------------|:--------------------|:--------------------|
| | CP / EE | CP / EE | CP / EE |
| Shared Policies | 273 / 279 | 227 / 217 | 126 / 114 |
| Per-iteration | 49 / 50 | 46 / 47 | 43 / 45 |
### Threading Efficiency Analysis
| Configuration | Low Contention (1-4t) | Medium Contention (6-12t) | High Contention (16+t) |
|:----------------|:----------------------|:--------------------------|:-----------------------|
| | Avg CP / EE | Avg CP / EE | Avg CP / EE |
| Shared Policies | 249 / 248 | 150 / 128 | 58 / 54 |
| Per-iteration | 47 / 48 | 40 / 39 | 22 / 27 |
**Key Observations:**
- **Single-threaded parity**: Both systems perform nearly identically at 1 thread
- **Threading behavior**: Compiled policies slightly outperform engine evaluation at higher thread counts for shared policies
- **Contention resistance**: Per-iteration configurations show very similar performance characteristics across all thread counts
- **Platform consistency**: Both C# implementations show similar scaling patterns and contention behavior
## Performance Insights
1. **Compilation Efficiency**: Pre-compiled policies provide massive performance benefits over per-iteration compilation
2. **C# Performance Gap**: C# compiled policies achieve 40%-70% of Rust performance for shared policies
3. **Engine vs Compiled**: In C#, engine evaluation slightly outperforms compiled policies (96%-104% range)
1. **C# vs Rust Performance**: C# compiled policies achieve 65% average performance of Rust for shared policies, 87% average for per-iteration across low contention scenarios
2. **Engine vs Compiled**: In C#, engine and compiled policy evaluation show very similar average performance (compiled policies achieve 100% of engine performance for shared policies, 98% for per-iteration)
3. **mimalloc Impact**: The use of mimalloc as the default allocator in the underlying Rust FFI provides better memory allocation efficiency and improved threading characteristics
4. **Threading Scaling**: Both C# configurations demonstrate similar contention patterns, with shared policies showing more pronounced degradation under high thread contention compared to per-iteration compilation