Policy evaluation at scale needs to be able to set memory limits
so that a bad policy does not hog memory or to ensure that
policy evaluation itself does not use too much memory which could
cause other components to suffer.
This PR introduces capability to set and enforce global memory limits.
It also lays the groundwork for enabling per evaluation limits in future.
Once a global memory limit is set, Regorus maintains per thread counters
to track memory activity (allocation, deallocation) of a thread.
These counters are periodically flushed to global memory counters.
Per thread counters avoid the contention that updating global counters
on each alloc/free would cause.
Policy evaluation periodically checks these counters and raises errors
if allocated memory has exceeded the configured limit.
Currently memory limit capability is exposed only to FFI and C#.
Also update mimalloc to v2.2.6
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Lints are added (deny) at crate level.
In each offending file, the failing lints are explicitly allowed.
Each file will be fixed in subsequent PRs.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Supply chain: Use the popular num-bigint crate for handling large integers
- Optimization: Handle f64, i64, u64 directly. These will be the most common instances of a number.
OPA number semantics isn't clear.
https://github.com/open-policy-agent/opa/issues/6281
As part of this change, we update the following failing tests:
- A local test that relies on what 15.3/3 evaluates to.
With our current change, we round in a different direction than what OPA does, but consistent
with Rust. We produce 5.1000000000000005 where as the OPA test expects 5.1.
There is no clear definition in Rego of what the right answer is. Moreover, policies should not
rely on exact floating point value comparison. Therefore this deviations is justified.
The test is patched to pass.
- Another local vm test that exercised 1.1 + 2.2
- Another local vm test that exercises 5.5 - 2.2
- An OPA test that expects that a large integer number say 10e308 is printed in exponent notation.
num-bigint does not print using scientific notation and instead prints all the digits.
The benefit of preserving this compatibility is not clear. We skip this test.
- Doc tests that exercised handling floating point numbers with more than 15 (what f64 supports)
digits of precision. There is no usecase for this scenario. The tests are updated to reflect
the behavior.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- add a dedicated `compiler/destructuring_planner` feature that precomputes binding plans for assignments, parameters, and `some in` expressions
- enrich `ScopeContext` with same-scope tracking, local scheduling hints, and module globals so the planner enforces := shadowing rules without blocking parent scopes
- wire the planner through compiler, hoist, interpreter, and engine paths while updating binding plan variants and adding query traversal helpers for dependency analysis
- document the new planner architecture and ship interpreter regressions that exercise nested destructuring, shadowing, and error reporting
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- `std` feature is enabled by default
- By default enable #![no_std] compilation
- Import std create if `std` feature is enabled or if testing
- Use core, alloc types
- Make it clear where std types are being used
- In no std, use BTreeMap in place of HashMap.
HashMap is not available in no std due to lack of a
secure random number generator
Note: The project does not yet compile without std feature being specified.
But it's really close to being able to do so.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Replace std with alloc, core in most places in src
Tests, bindings aren't changed.
- Introduce BuiltinsMap type alias inplace of HashMap.
In no_std case, this could be aliases to BTreeMap
- Fix clippy warnings
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* `arc` feature to make engine usable from multiple threads.
`arc` is turned on by default. When enabled, std::sync::Arc
will be used instead of std::rc::Rc. The former makes regorus
types like Engine, Value, ast nodes etc Send, allowing for
usability from multiple threads.
Arc would add a performance overhead though since the reference
counting will now become atomic.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* Make engine and related types Debug
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
* Input, Data as json. Evaluate bool queries.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
---------
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Separate out public, unstable and internal APIs.
- Cleanup README.md and include it as the crate documentation.
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
- Switch to scientific crate. Large values are printed in scientific notations.
Regular values are printed as u64, i64 or f64.
- Skip copying commit hooks in git worktrees
- urlquery.decode, urlquery.encode, urlquery.encode_object
- substring, indexof_n string builtins
- Make sprintf more OPA conformant
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
Number is implemented using rust_decimal::Decimal which uses a 96 bit mantissa.
TODO:
a) Support u64, i64 variants
b) Determine desired semantics for floating-point
c) Determine desired big integer length
d) Explore other big int/big float crates
Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
