mirror of
https://github.com/microsoft/regorus.git
synced 2026-08-05 02:16:11 +00:00
feat! Mimalloc as the default allocator (#434)
This change integrates mimalloc as the default memory allocator for Regorus, delivering significant performance improvements across all evaluation modes and language bindings. Technical Implementation: - Build mimalloc in vendored mode from C sources (following QSharp approach) - Implement GlobalAlloc trait for seamless Rust integration - Add optional 'mimalloc' feature flag for conditional compilation - Add comprehensive ACI benchmarks to measure evaluation performance Performance Impact: Rust Engine Evaluation: - Single-threaded: ~29% improvement (423 vs 328 Kelem/s) - Multi-threaded: Better scaling with reduced thread contention - Fresh engines: ~24% improvement (56 vs 45 Kelem/s) Rust Compiled Policy Evaluation: - Single-threaded: ~41% improvement (426 vs 303 Kelem/s) - Multi-threaded: Improved allocation efficiency under contention - Fresh compilation: ~26% improvement (53 vs 42 Kelem/s) C# FFI Bindings: - Engine evaluation: ~27% improvement (279 vs 219 Kelem/s) - Compiled policies: ~29% improvement (273 vs 211 Kelem/s) - Better threading characteristics through improved underlying allocation Key Benefits: - Reduced allocation-related contention in multi-threaded scenarios - More consistent performance across different thread counts - Improved memory allocation efficiency for both native Rust and FFI workloads - Better scaling characteristics for production deployments The mimalloc integration provides substantial performance gains while maintaining full compatibility with existing code through feature flags. Reference: QSharp allocator implementation (https://github.com/microsoft/qsharp/tree/main/source/allocator) Fixes #297 Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This commit is contained in:
Anand Krishnamoorthi
committed by
GitHub
GitHub
parent
6c5338228b
commit
2a0b4ae6b5
@@ -5,6 +5,7 @@
|
||||
- **CPU**: 16 cores
|
||||
- **Architecture**: ARM64 (aarch64-apple-darwin)
|
||||
- **.NET Version**: 8.0
|
||||
- **Allocator**: mimalloc (default allocator for Rust FFI)
|
||||
- **Benchmark Framework**: Custom time-based benchmarking
|
||||
- **Test Data**: 20,000 inputs per evaluation (distributed across threads)
|
||||
- **Policy**: Complex authorization policy with nested rules
|
||||
@@ -27,77 +28,113 @@ The C# compiled policy evaluation benchmark tests Regorus compiled policy perfor
|
||||
### Compiled Shared Policies (Best Performance)
|
||||
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|
||||
|--------:|---------------------------:|---------------------:|
|
||||
| 1 | 2928.81 | 211 |
|
||||
| 2 | 5892.53 | 146 |
|
||||
| 4 | 11750.71 | 155 |
|
||||
| 6 | 17686.92 | 134 |
|
||||
| 8 | 23543.53 | 90 |
|
||||
| 10 | 29503.80 | 72 |
|
||||
| 12 | 35494.81 | 58 |
|
||||
| 14 | 41408.36 | 50 |
|
||||
| 16 | 47333.65 | 44 |
|
||||
| 18 | 53050.24 | 38 |
|
||||
| 20 | 58807.20 | 34 |
|
||||
| 22 | 406022.45 | 32 |
|
||||
| 24 | 65480.69 | 32 |
|
||||
| 26 | 70952.34 | 30 |
|
||||
| 28 | 72064.03 | 30 |
|
||||
| 30 | 492405.74 | 27 |
|
||||
| 32 | 81210.83 | 27 |
|
||||
| 1 | 2905.41 | 273 |
|
||||
| 2 | 5808.07 | 240 |
|
||||
| 4 | 11631.23 | 227 |
|
||||
| 6 | 17431.95 | 216 |
|
||||
| 8 | 23183.42 | 126 |
|
||||
| 10 | 28886.11 | 118 |
|
||||
| 12 | 34659.87 | 108 |
|
||||
| 14 | 40564.07 | 84 |
|
||||
| 16 | 46446.38 | 72 |
|
||||
| 18 | 52047.06 | 63 |
|
||||
| 20 | 56983.45 | 58 |
|
||||
| 22 | 404931.47 | 55 |
|
||||
| 24 | 61673.71 | 55 |
|
||||
| 26 | 64370.41 | 51 |
|
||||
| 28 | 56897.04 | 59 |
|
||||
| 30 | 406850.06 | 52 |
|
||||
| 32 | 56786.24 | 58 |
|
||||
|
||||
### Compiled Per Iteration
|
||||
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|
||||
|--------:|---------------------------:|---------------------:|
|
||||
| 1 | 2984.00 | 39 |
|
||||
| 2 | 5969.45 | 38 |
|
||||
| 4 | 11948.28 | 32 |
|
||||
| 6 | 17927.24 | 30 |
|
||||
| 8 | 23889.01 | 24 |
|
||||
| 10 | 29882.38 | 20 |
|
||||
| 12 | 35865.06 | 18 |
|
||||
| 14 | 41838.70 | 15 |
|
||||
| 16 | 47800.92 | 14 |
|
||||
| 18 | 53257.22 | 10 |
|
||||
| 20 | 59596.93 | 11 |
|
||||
| 22 | 435853.41 | 10 |
|
||||
| 24 | 70870.86 | 9 |
|
||||
| 26 | 76120.59 | 9 |
|
||||
| 28 | 80717.51 | 8 |
|
||||
| 30 | 544207.96 | 8 |
|
||||
| 32 | 91540.91 | 7 |
|
||||
| 1 | 2978.06 | 49 |
|
||||
| 2 | 5965.09 | 47 |
|
||||
| 4 | 11928.23 | 46 |
|
||||
| 6 | 17892.58 | 45 |
|
||||
| 8 | 23773.82 | 43 |
|
||||
| 10 | 29705.61 | 42 |
|
||||
| 12 | 35631.97 | 40 |
|
||||
| 14 | 41563.35 | 34 |
|
||||
| 16 | 47452.93 | 31 |
|
||||
| 18 | 53505.42 | 27 |
|
||||
| 20 | 59393.86 | 25 |
|
||||
| 22 | 436115.28 | 23 |
|
||||
| 24 | 71088.08 | 21 |
|
||||
| 26 | 76928.70 | 19 |
|
||||
| 28 | 82759.27 | 18 |
|
||||
| 30 | 560658.97 | 17 |
|
||||
| 32 | 93949.39 | 16 |
|
||||
|
||||
## Analysis
|
||||
|
||||
The C# compiled policy benchmark demonstrates important performance characteristics:
|
||||
The C# compiled policy benchmark demonstrates important performance characteristics with mimalloc as the default allocator:
|
||||
|
||||
1. **Compilation Strategy Impact**: Shared compiled policies significantly outperform per-iteration compilation (~5.4x at 1 thread)
|
||||
2. **Scaling Patterns**:
|
||||
1. **Compilation Strategy Impact**: Shared compiled policies significantly outperform per-iteration compilation (~5.6x at 1 thread)
|
||||
2. **Scaling Patterns with mimalloc**:
|
||||
- Best throughput achieved at 1 thread for shared policies
|
||||
- Performance generally degrades with increased thread count
|
||||
- Performance generally degrades with increased thread count, but mimalloc provides better allocation efficiency
|
||||
3. **Performance Hierarchy**:
|
||||
- Shared compiled policies: Best performance (optimal configuration)
|
||||
- Per-iteration compilation: ~82% reduction from optimal
|
||||
4. **Compilation Overhead**: Per-iteration compilation creates substantial overhead, similar to fresh engine creation
|
||||
5. **Thread Contention**: Significant performance degradation beyond 8 threads for both configurations
|
||||
5. **Thread Contention**: Significant performance degradation beyond 8 threads for both configurations, though mimalloc helps mitigate some allocation-related issues
|
||||
|
||||
## Comparison with Rust Compiled Policy Evaluation
|
||||
|
||||
| Configuration | C# Performance (1 thread) | Rust Performance (1 thread) | Relative Performance |
|
||||
|:-----------------|:----------------------------|:-----------------------------|---------------------:|
|
||||
| Shared Policies | Best performance | Higher throughput | 0.40x-0.70x |
|
||||
| Per-iteration | ~82% reduction from optimal | ~85% reduction from optimal | 0.47x-0.89x |
|
||||
### Multi-Thread Performance Comparison
|
||||
|
||||
| Configuration | 1 Thread (Kelem/s) | 4 Threads (Kelem/s) | 8 Threads (Kelem/s) |
|
||||
|:-----------------|:-------------------|:--------------------|:--------------------|
|
||||
| | C# / Rust | C# / Rust | C# / Rust |
|
||||
| Shared Policies | 273 / 426 | 227 / 342 | 126 / 185 |
|
||||
| Per-iteration | 49 / 55 | 46 / 50 | 43 / 50 |
|
||||
|
||||
### Threading Efficiency Analysis
|
||||
|
||||
| Configuration | Low Contention (1-4t) | Medium Contention (6-12t) | High Contention (16+t) |
|
||||
|:-----------------|:----------------------|:--------------------------|:-----------------------|
|
||||
| | Avg C# / Rust | Avg C# / Rust | Avg C# / Rust |
|
||||
| Shared Policies | 249 / 384 | 150 / 203 | 58 / 123 |
|
||||
| Per-iteration | 47 / 54 | 40 / 50 | 22 / 42 |
|
||||
|
||||
**Key Observations:**
|
||||
- **Single-threaded performance**: C# achieves 64% of Rust performance for shared policies, 89% for per-iteration
|
||||
- **Threading scaling**: Both platforms show similar degradation patterns, but Rust maintains better absolute performance
|
||||
- **Contention resistance**: Per-iteration compilation shows more consistent relative performance across thread counts
|
||||
- **Platform differences**: C# shows more pronounced performance drops at higher thread counts, particularly for shared policies
|
||||
|
||||
*Note: Rust benchmarks include additional input data variations (cloned vs fresh inputs) that are not present in the C# implementation.*
|
||||
|
||||
## Comparison with C# Engine Evaluation
|
||||
|
||||
| Configuration | Compiled Policy (1 thread) | Engine Evaluation (1 thread) | Performance Ratio |
|
||||
|:---------------|:----------------------------|:------------------------------|------------------:|
|
||||
| Optimal Config | Best performance | Slightly higher throughput | 0.96x |
|
||||
### Multi-Thread Performance Comparison
|
||||
|
||||
| Configuration | 1 Thread (Kelem/s) | 4 Threads (Kelem/s) | 8 Threads (Kelem/s) |
|
||||
|:----------------|:-------------------|:--------------------|:--------------------|
|
||||
| | CP / EE | CP / EE | CP / EE |
|
||||
| Shared Policies | 273 / 279 | 227 / 217 | 126 / 114 |
|
||||
| Per-iteration | 49 / 50 | 46 / 47 | 43 / 45 |
|
||||
|
||||
### Threading Efficiency Analysis
|
||||
|
||||
| Configuration | Low Contention (1-4t) | Medium Contention (6-12t) | High Contention (16+t) |
|
||||
|:----------------|:----------------------|:--------------------------|:-----------------------|
|
||||
| | Avg CP / EE | Avg CP / EE | Avg CP / EE |
|
||||
| Shared Policies | 249 / 248 | 150 / 128 | 58 / 54 |
|
||||
| Per-iteration | 47 / 48 | 40 / 39 | 22 / 27 |
|
||||
|
||||
**Key Observations:**
|
||||
- **Single-threaded parity**: Both systems perform nearly identically at 1 thread
|
||||
- **Threading behavior**: Compiled policies slightly outperform engine evaluation at higher thread counts for shared policies
|
||||
- **Contention resistance**: Per-iteration configurations show very similar performance characteristics across all thread counts
|
||||
- **Platform consistency**: Both C# implementations show similar scaling patterns and contention behavior
|
||||
|
||||
## Performance Insights
|
||||
|
||||
1. **Compilation Efficiency**: Pre-compiled policies provide massive performance benefits over per-iteration compilation
|
||||
2. **C# Performance Gap**: C# compiled policies achieve 40%-70% of Rust performance for shared policies
|
||||
3. **Engine vs Compiled**: In C#, engine evaluation slightly outperforms compiled policies (96%-104% range)
|
||||
1. **C# vs Rust Performance**: C# compiled policies achieve 65% average performance of Rust for shared policies, 87% average for per-iteration across low contention scenarios
|
||||
2. **Engine vs Compiled**: In C#, engine and compiled policy evaluation show very similar average performance (compiled policies achieve 100% of engine performance for shared policies, 98% for per-iteration)
|
||||
3. **mimalloc Impact**: The use of mimalloc as the default allocator in the underlying Rust FFI provides better memory allocation efficiency and improved threading characteristics
|
||||
4. **Threading Scaling**: Both C# configurations demonstrate similar contention patterns, with shared policies showing more pronounced degradation under high thread contention compared to per-iteration compilation
|
||||
|
||||
|
||||
@@ -5,6 +5,7 @@
|
||||
- **CPU**: 16 cores
|
||||
- **Architecture**: ARM64 (aarch64-apple-darwin)
|
||||
- **.NET Version**: 8.0
|
||||
- **Allocator**: mimalloc (default allocator for Rust FFI)
|
||||
- **Benchmark Framework**: Custom time-based benchmarking
|
||||
- **Test Data**: 20,000 inputs per evaluation (distributed across threads)
|
||||
- **Policy**: Complex authorization policy with nested rules
|
||||
@@ -27,73 +28,91 @@ The C# engine evaluation benchmark tests Regorus policy evaluation performance a
|
||||
### Cloned Engines (Best Performance)
|
||||
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|
||||
|--------:|---------------------------:|---------------------:|
|
||||
| 1 | 2930.56 | 219 |
|
||||
| 2 | 5868.46 | 177 |
|
||||
| 4 | 11771.01 | 146 |
|
||||
| 6 | 17682.52 | 129 |
|
||||
| 8 | 23633.65 | 78 |
|
||||
| 10 | 29489.12 | 67 |
|
||||
| 12 | 35455.23 | 57 |
|
||||
| 14 | 41353.65 | 47 |
|
||||
| 16 | 47378.91 | 42 |
|
||||
| 18 | 52750.68 | 36 |
|
||||
| 20 | 58131.31 | 35 |
|
||||
| 22 | 62964.88 | 31 |
|
||||
| 24 | 64337.75 | 34 |
|
||||
| 26 | 70044.96 | 29 |
|
||||
| 28 | 72553.98 | 28 |
|
||||
| 30 | 79323.25 | 26 |
|
||||
| 32 | 78624.33 | 26 |
|
||||
| 1 | 2903.43 | 279 |
|
||||
| 2 | 5808.35 | 227 |
|
||||
| 4 | 11645.08 | 217 |
|
||||
| 6 | 17469.69 | 207 |
|
||||
| 8 | 23268.07 | 114 |
|
||||
| 10 | 28996.14 | 104 |
|
||||
| 12 | 34808.60 | 98 |
|
||||
| 14 | 40703.21 | 72 |
|
||||
| 16 | 46488.23 | 63 |
|
||||
| 18 | 52078.52 | 56 |
|
||||
| 20 | 57014.31 | 51 |
|
||||
| 22 | 60482.22 | 47 |
|
||||
| 24 | 62445.67 | 46 |
|
||||
| 26 | 65128.74 | 45 |
|
||||
| 28 | 58001.92 | 50 |
|
||||
| 30 | 66154.78 | 42 |
|
||||
| 32 | 64999.03 | 45 |
|
||||
|
||||
### Fresh Engines
|
||||
| Threads | Total Evaluation Time (ms) | Throughput (Kelem/s) |
|
||||
|--------:|---------------------------:|---------------------:|
|
||||
| 1 | 2985.49 | 41 |
|
||||
| 2 | 5968.13 | 38 |
|
||||
| 4 | 11942.10 | 34 |
|
||||
| 6 | 17918.75 | 32 |
|
||||
| 8 | 23873.57 | 25 |
|
||||
| 10 | 29863.85 | 20 |
|
||||
| 12 | 35823.98 | 19 |
|
||||
| 14 | 41811.53 | 16 |
|
||||
| 16 | 47819.89 | 14 |
|
||||
| 18 | 53478.32 | 13 |
|
||||
| 20 | 59191.93 | 12 |
|
||||
| 22 | 64630.71 | 11 |
|
||||
| 24 | 70215.54 | 10 |
|
||||
| 26 | 75732.06 | 9 |
|
||||
| 28 | 80897.59 | 9 |
|
||||
| 30 | 949904.84 | 8 |
|
||||
| 32 | 92592.64 | 8 |
|
||||
| 1 | 2982.28 | 50 |
|
||||
| 2 | 5962.62 | 48 |
|
||||
| 4 | 11917.94 | 47 |
|
||||
| 6 | 17874.77 | 46 |
|
||||
| 8 | 23729.94 | 45 |
|
||||
| 10 | 29635.17 | 42 |
|
||||
| 12 | 35574.71 | 38 |
|
||||
| 14 | 41482.61 | 34 |
|
||||
| 16 | 47425.16 | 32 |
|
||||
| 18 | 53248.87 | 29 |
|
||||
| 20 | 58424.34 | 27 |
|
||||
| 22 | 61302.24 | 26 |
|
||||
| 24 | 67430.08 | 23 |
|
||||
| 26 | 65226.79 | 24 |
|
||||
| 28 | 73118.48 | 22 |
|
||||
| 30 | 326472.94 | 23 |
|
||||
| 32 | 63805.03 | 24 |
|
||||
|
||||
## Analysis
|
||||
|
||||
The C# benchmark results demonstrate important performance characteristics:
|
||||
The C# benchmark results demonstrate important performance characteristics with mimalloc as the default allocator:
|
||||
|
||||
1. **Engine Reuse Impact**: Cloned engines significantly outperform fresh engines (~5.3x at 1 thread)
|
||||
2. **Scaling Patterns**:
|
||||
1. **Engine Reuse Impact**: Cloned engines significantly outperform fresh engines (~5.6x at 1 thread)
|
||||
2. **Scaling Patterns with mimalloc**:
|
||||
- Best throughput achieved at 1 thread for both configurations
|
||||
- Performance degrades with increased thread count due to contention
|
||||
- Performance degrades with increased thread count due to contention, but mimalloc provides better allocation efficiency
|
||||
- Cloned engines show better relative scaling characteristics
|
||||
3. **Performance Hierarchy**:
|
||||
- Cloned engines: Best performance (optimal configuration)
|
||||
- Fresh engines: ~81% reduction from optimal
|
||||
4. **Thread Contention**: Significant performance drop beyond 8 threads, especially for fresh engines
|
||||
5. **C# vs Rust Performance**: C# shows ~67% of Rust performance for equivalent cloned engine configuration
|
||||
- Fresh engines: ~82% reduction from optimal
|
||||
4. **Thread Contention**: Significant performance drop beyond 8 threads, especially for fresh engines, though mimalloc helps mitigate some allocation-related issues
|
||||
5. **C# vs Rust Performance**: C# shows ~66% of Rust performance for equivalent cloned engine configuration
|
||||
|
||||
## Comparison with Rust Engine Evaluation
|
||||
|
||||
| Configuration | C# Performance (1 thread) | Rust Performance (1 thread) | Relative Performance |
|
||||
|:---------------|:---------------------------|:-----------------------------|---------------------:|
|
||||
| Cloned Engines | Best performance | Higher throughput | 0.67x-0.92x |
|
||||
| Fresh Engines | ~81% reduction from optimal| ~87% reduction from optimal | 0.75x-0.95x |
|
||||
### Multi-Thread Performance Comparison
|
||||
|
||||
| Configuration | 1 Thread (Kelem/s) | 4 Threads (Kelem/s) | 8 Threads (Kelem/s) |
|
||||
|:---------------|:-------------------|:--------------------|:--------------------|
|
||||
| | C# / Rust | C# / Rust | C# / Rust |
|
||||
| Cloned Engines | 279 / 423 | 217 / 406 | 114 / 341 |
|
||||
| Fresh Engines | 50 / 56 | 47 / 54 | 45 / 53 |
|
||||
|
||||
### Threading Efficiency Analysis
|
||||
|
||||
| Configuration | Low Contention (1-4t) | Medium Contention (6-12t) | High Contention (16+t) |
|
||||
|:---------------|:----------------------|:--------------------------|:-----------------------|
|
||||
| | Avg C# / Rust | Avg C# / Rust | Avg C# / Rust |
|
||||
| Cloned Engines | 253 / 414 | 128 / 329 | 54 / 250 |
|
||||
| Fresh Engines | 48 / 55 | 39 / 52 | 27 / 42 |
|
||||
|
||||
**Key Observations:**
|
||||
- **Single-threaded performance**: C# achieves 66% of Rust performance for cloned engines, 89% for fresh engines
|
||||
- **Threading scaling**: Both platforms show similar degradation patterns, but Rust maintains better absolute performance
|
||||
- **Contention resistance**: Fresh engines show more consistent relative performance across thread counts
|
||||
- **Platform differences**: C# shows more pronounced performance drops at higher thread counts, particularly for cloned engines
|
||||
|
||||
*Note: Rust benchmarks include additional input data variations (cloned vs fresh inputs) that are not present in the C# implementation.*
|
||||
|
||||
## Performance Insights
|
||||
|
||||
1. **Engine Creation Overhead**: Fresh engine creation has massive performance impact in C# (~5.3x slower)
|
||||
2. **Thread Scaling**: C# shows more significant thread contention than Rust implementation
|
||||
3. **Memory Management**: .NET garbage collection may contribute to performance variations
|
||||
4. **Interop Overhead**: C# bindings add measurable overhead compared to native Rust
|
||||
1. **Engine Creation Overhead**: Fresh engine creation has significant performance impact in C# (~5.6x slower than cloned engines)
|
||||
2. **Thread Scaling**: C# shows moderate thread contention with better characteristics when using mimalloc
|
||||
3. **Memory Management**: .NET garbage collection patterns combined with mimalloc allocation efficiency
|
||||
4. **Interop Performance**: C# bindings achieve 66% of Rust performance for cloned engines, demonstrating effective FFI implementation
|
||||
5. **mimalloc Benefits**: The use of mimalloc as the default allocator in the underlying Rust FFI provides improved memory allocation efficiency and better threading characteristics
|
||||
|
||||
|
||||
Reference in New Issue
Block a user