mirror of
https://github.com/microsoft/regorus.git
synced 2026-08-05 02:16:11 +00:00
feat!: add Rego Virtual Machine (RVM) implementation (#495)
* feat!: add Rego Virtual Machine (RVM) implementation This commit introduces a register-based virtual machine for executing Rego policies with bytecode-style instructions. Unlike the existing tree-walking interpreter, the RVM compiles policies into instruction sequences that operate on virtual registers, offering better performance and optimization potential. Core Components: Instruction Set Architecture: - Define instruction types for data operations, control flow, and builtins - Implement instruction parameter encoding and display formatting - Add instruction parser with comprehensive test coverage Virtual Machine Engine: - Register-based execution model with program counter management - Loop execution supporting iterators, comprehensions, and quantifiers - Function call handling with argument evaluation and context management - Rule evaluation with default value resolution and virtual data support - Arithmetic and comparison operation implementations Program Representation: - Program listing builder with instruction sequencing - Rule tree construction for organizing policy rules - Binary and JSON serialization for compiled programs - Recompilation support for program modification Testing Infrastructure: - Extensive YAML test suites covering all VM features - Rust unit tests for VM execution and instruction parsing - Test suites for loops, comprehensions, builtins, and control flow BREAKING CHANGE: Introduces new VM execution path alongside interpreter Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> * docs: add detailed RVM architecture references Introduce architecture.md explaining program artifacts, serialization, and runtime subsystems. Document the full opcode catalog in instruction-set.md, including operands, parameter tables, and outcomes. Walk through execution flow, stacks, and operational guidance in vm-runtime.md, tying the runtime to the new architecture docs. Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com> --------- Signed-off-by: Anand Krishnamoorthi <anakrish@microsoft.com>
This commit is contained in:
Anand Krishnamoorthi
committed by
GitHub
GitHub
parent
6dc505c88b
commit
49bd3c22f3
238
docs/rvm/instruction-set.md
Normal file
238
docs/rvm/instruction-set.md
Normal file
@@ -0,0 +1,238 @@
|
||||
# RVM Instruction Set Reference
|
||||
|
||||
This reference captures every opcode emitted by the compiler and executed by
|
||||
`RegoVM`. Each instruction is defined in `src/rvm/instructions/mod.rs` and
|
||||
implemented by the dispatcher tree in `src/rvm/vm/dispatch.rs` plus specialised
|
||||
submodules (`arithmetic.rs`, `loops.rs`, `functions.rs`, `rules.rs`,
|
||||
`comprehension.rs`, `virtual_data.rs`).
|
||||
|
||||
Use this guide to understand operand semantics, parameter tables, and runtime
|
||||
side effects.
|
||||
|
||||
---
|
||||
|
||||
## Reading the tables
|
||||
|
||||
- **Operands**: registers (`rX`), literals (`litY`), parameter indices (`pZ`) and
|
||||
immediate values.
|
||||
- **Parameters**: links into `InstructionData` (`src/rvm/instructions/params.rs`).
|
||||
The compiler stores complex metadata here; instructions reference it by index.
|
||||
- **Outcome**: mentioned in prose where relevant (`Continue`, `Return`, `Break`,
|
||||
`Suspend`).
|
||||
|
||||
---
|
||||
|
||||
## Load and Move instructions
|
||||
|
||||
| Mnemonic | Operands | Behaviour |
|
||||
| :--------- | :-------------------------- | :--------------------------------------------------- |
|
||||
| `Load` | `dest=rD, literal_idx=litN` | Copies literal `N` into register `D`. |
|
||||
| `LoadTrue` | `dest=rD` | Stores boolean `true`. |
|
||||
| `LoadFalse`| `dest=rD` | Stores boolean `false`. |
|
||||
| `LoadNull` | `dest=rD` | Stores `Value::Null`. |
|
||||
| `LoadBool` | `dest=rD, value` | Stores inline boolean literal. |
|
||||
| `LoadData` | `dest=rD` | Stores the VM's `data` value. |
|
||||
| `LoadInput`| `dest=rD` | Stores the VM's `input` value. |
|
||||
| `Move` | `dest=rD, src=rS` | Copies register `S` into register `D`. |
|
||||
|
||||
Out-of-range literal indices raise `VmError::LiteralIndexOutOfBounds`. Registers
|
||||
must have been allocated by the current frame.
|
||||
|
||||
---
|
||||
|
||||
## Arithmetic and comparison instructions
|
||||
|
||||
| Mnemonic | Operands | Behaviour |
|
||||
| :------- | :---------------------- | :------------------------------------------------------------ |
|
||||
| `Add` | `dest, left, right` | Numeric addition; undefined operands trigger loop condition checks. |
|
||||
| `Sub` | `dest, left, right` | Numeric subtraction. |
|
||||
| `Mul` | `dest, left, right` | Numeric multiplication. |
|
||||
| `Div` | `dest, left, right` | Numeric division with runtime checks (division by zero errors). |
|
||||
| `Mod` | `dest, left, right` | Modulo. |
|
||||
| `Eq` | `dest, left, right` | Equality comparison resulting in `Value::Bool`. |
|
||||
| `Ne` | `dest, left, right` | Inequality. |
|
||||
| `Lt`/`Le`/`Gt`/`Ge` | `dest, left, right` | Ordering comparisons. |
|
||||
| `And` | `dest, left, right` | Logical conjunction (truthiness semantics). |
|
||||
| `Or` | `dest, left, right` | Logical disjunction. |
|
||||
| `Not` | `dest, operand` | Logical negation. |
|
||||
| `AssertCondition` | `condition` | Fails current loop/rule when the condition is falsey. |
|
||||
| `AssertNotUndefined` | `register` | Fails when register holds `Value::Undefined`. |
|
||||
|
||||
`handle_condition` routes through `loops.rs` to propagate failures to loop and
|
||||
comprehension contexts. Outside loops it aborts the current rule.
|
||||
|
||||
---
|
||||
|
||||
## Collection and indexing instructions
|
||||
|
||||
| Mnemonic | Operands / Params | Behaviour |
|
||||
| :------------------------- | :---------------------------- | :---------------------------------------------------------- |
|
||||
| `ObjectSet` | `obj, key, value` | Mutates object in `obj` with key/value from registers. |
|
||||
| `ObjectCreate` | `params_index=pN` | Builds object from literal template and register entries. |
|
||||
| `ArrayNew` | `dest` | Creates empty array. |
|
||||
| `ArrayPush` | `arr, value` | Appends to array. |
|
||||
| `ArrayCreate` | `params_index=pN` | Builds array from register list; undefined element ⇒ result undefined. |
|
||||
| `SetNew` | `dest` | Creates empty set. |
|
||||
| `SetAdd` | `set, value` | Adds element to set. |
|
||||
| `SetCreate` | `params_index=pN` | Builds set from register list; undefined element ⇒ result undefined. |
|
||||
| `Index` | `dest, container, key` | Indexes container with runtime key. |
|
||||
| `IndexLiteral` | `dest, container, literal_idx`| Indexes container using literal stored in program. |
|
||||
| `ChainedIndex` | `params_index=pN` | Resolves multi-hop path from root register. |
|
||||
| `Contains` | `dest, collection, value` | Checks membership; returns `Value::Bool`. |
|
||||
| `Count` | `dest, collection` | Returns length or `Value::Undefined` for unsupported types. |
|
||||
| `VirtualDataDocumentLookup`| `params_index=pN` | Evaluates `data` path, invoking rules lazily. |
|
||||
|
||||
Parameter structures:
|
||||
|
||||
- `ObjectCreateParams` reuses arrays of literal key/value pairs and register
|
||||
pairs. Literal keys must be sorted to match template order.
|
||||
- `ArrayCreateParams` and `SetCreateParams` store register lists. The VM checks
|
||||
all referenced registers for `Value::Undefined` before constructing the
|
||||
collection.
|
||||
- `VirtualDataDocumentLookupParams` and `ChainedIndexParams` encode `Vec<LiteralOrRegister>`
|
||||
path components. `LiteralOrRegister` is defined in `src/rvm/instructions/types.rs`.
|
||||
|
||||
---
|
||||
|
||||
## Loop instructions
|
||||
|
||||
Loops use dedicated parameter tables (`LoopStartParams`) and the `LoopMode`
|
||||
enum.
|
||||
|
||||
| Mnemonic | Operands / Params | Behaviour |
|
||||
| :---------- | :----------------------- | :------------------------------------------------------------- |
|
||||
| `LoopStart` | `params_index=pN` | Initialises loop context and decides first body iteration. |
|
||||
| `LoopNext` | `body_start`, `loop_end` | Finalises iteration, updates accumulators, advances to next element. |
|
||||
|
||||
`LoopMode` values:
|
||||
|
||||
- `Any`: succeed on first passing iteration, short-circuit on success.
|
||||
- `Every`: fail on first failing iteration.
|
||||
- `ForEach`: evaluate all iterations, typically for comprehensions or complete
|
||||
rules.
|
||||
|
||||
`LoopStartParams` fields:
|
||||
|
||||
- `collection`: source register.
|
||||
- `key_reg` / `value_reg`: iteration registers (for arrays, key is index).
|
||||
- `result_reg`: accumulator storing loop outcome (`bool` for quantifiers).
|
||||
- `body_start` / `loop_end`: PCs identifying loop boundaries.
|
||||
|
||||
The dispatcher converts `LoopStartParams` into a VM-specific `LoopParams` used by
|
||||
both execution modes. In suspendable mode, loops own their own `ExecutionFrame`.
|
||||
|
||||
---
|
||||
|
||||
## Comprehension instructions
|
||||
|
||||
| Mnemonic | Operands / Params | Behaviour |
|
||||
| :------------------- | :---------------------- | :------------------------------------------------- |
|
||||
| `ComprehensionBegin` | `params_index=pN` | Allocates collection builder and iteration context. |
|
||||
| `ComprehensionYield` | `value_reg`, `key_reg?` | Emits value (and optional key) into builder. |
|
||||
| `ComprehensionEnd` | — | Finalises collection and stores result. |
|
||||
|
||||
`ComprehensionBeginParams` captures:
|
||||
|
||||
- `mode: ComprehensionMode` (Set, Array, Object)
|
||||
- `collection_reg`: source register for iteration
|
||||
- `result_reg`: register that will hold the final collection
|
||||
- `key_reg` / `value_reg`: iteration registers
|
||||
- `body_start` / `comprehension_end`: branch targets
|
||||
|
||||
Comprehensions manage their own stack (`ComprehensionContext`) to maintain
|
||||
ordering guarantees (arrays), uniqueness (sets) or key/value pairing (objects).
|
||||
|
||||
---
|
||||
|
||||
## Call and return instructions
|
||||
|
||||
| Mnemonic | Operands / Params | Behaviour |
|
||||
| :-------------------- | :------------------------ | :---------------------------------------------- |
|
||||
| `BuiltinCall` | `params_index=pN` | Invokes builtin via resolved function pointer. |
|
||||
| `FunctionCall` | `params_index=pN` | Invokes function rule. |
|
||||
| `CallRule` | `dest, rule_index` | Requests rule evaluation with caching. |
|
||||
| `RuleInit` | `result_reg, rule_index` | Prepares rule accumulator and cache state. |
|
||||
| `Return` | `value_reg` | Returns value from current function body. |
|
||||
| `RuleReturn` | — | Finalises rule evaluation frame. |
|
||||
| `DestructuringSuccess`| — | Signals successful destructuring, breaks rule block. |
|
||||
|
||||
Parameter tables:
|
||||
|
||||
- `BuiltinCallParams` / `FunctionCallParams` store destination register, index
|
||||
into builtin table / rule index, argument count and up to eight argument
|
||||
register numbers.
|
||||
- The VM dynamically resizes registers when a callee requires a larger window
|
||||
using program metadata (`max_rule_window_size`).
|
||||
|
||||
---
|
||||
|
||||
## Host interaction
|
||||
|
||||
| Mnemonic | Operands | Behaviour |
|
||||
| :--------- | :---------------- | :--------------------------------------- |
|
||||
| `HostAwait`| `dest, arg, id` | Yields control to host with payload value. |
|
||||
|
||||
- Run-to-completion: consumes a response from `host_await_responses` keyed by
|
||||
the identifier register. Missing responses raise `VmError::HostAwaitResponseMissing`.
|
||||
- Suspendable: emits `InstructionOutcome::Suspend` with `SuspendReason::HostAwait`.
|
||||
The host must resume with a value that will be written into `dest`.
|
||||
|
||||
---
|
||||
|
||||
## Halt instruction
|
||||
|
||||
| Mnemonic | Behaviour | Notes |
|
||||
| :------- | :-------------------------------- | :---- |
|
||||
| `Halt` | Terminates execution immediately. | Used during debugging or emitted for guard rails. |
|
||||
|
||||
When encountered during run-to-completion execution, `Halt` returns the current
|
||||
value in register `0`.
|
||||
|
||||
---
|
||||
|
||||
## Parameter data overview
|
||||
|
||||
`InstructionData` (`src/rvm/instructions/params.rs`) collects all complex
|
||||
parameter types. Each `add_*` method returns a `u16` index suitable for storing
|
||||
inside instructions. The VM retrieves tables via `get_*` accessors.
|
||||
|
||||
| Struct | Field | Purpose |
|
||||
| :----------------------- | :---------------------------------------- | :------------------------------------------------------------------------- |
|
||||
| `LoopStartParams` | `mode` | Loop semantics (`Any`, `Every`, `ForEach`). |
|
||||
| | `collection` | Register holding the iterable collection. |
|
||||
| | `key_reg` / `value_reg` | Registers populated with the current key/value each iteration. |
|
||||
| | `result_reg` | Accumulator for loop outcome (`bool` for quantifiers). |
|
||||
| | `body_start` / `loop_end` | Instruction pointers delimiting the loop body and exit. |
|
||||
| `BuiltinCallParams` | `dest` | Register that receives the builtin result. |
|
||||
| | `builtin_index` | Slot into `builtin_info_table` for dispatch. |
|
||||
| | `num_args` | Count of argument registers actually populated. |
|
||||
| | `args[8]` | Up to eight registers supplying builtin arguments. |
|
||||
| `FunctionCallParams` | `dest` | Register that receives the function rule result. |
|
||||
| | `func_rule_index` | Rule index for the target function definition. |
|
||||
| | `num_args` | Number of argument registers provided. |
|
||||
| | `args[8]` | Argument register numbers (unused slots ignored). |
|
||||
| `ObjectCreateParams` | `dest` | Destination register for the constructed object. |
|
||||
| | `template_literal_idx` | Literal template containing all expected keys. |
|
||||
| | `literal_key_fields: Vec<(u16, u8)>` | Mapping of literal-key indices to value registers. |
|
||||
| | `fields: Vec<(u8, u8)>` | Dynamic key/value register pairs for non-literal keys. |
|
||||
| `ArrayCreateParams` | `dest` | Destination register for the array literal. |
|
||||
| | `elements: Vec<u8>` | Registers providing array elements (order preserved). |
|
||||
| `SetCreateParams` | `dest` | Destination register for the set literal. |
|
||||
| | `elements: Vec<u8>` | Registers providing set members (duplicates dropped at runtime). |
|
||||
| `VirtualDataDocumentLookupParams` | `dest` | Destination register for lookup result. |
|
||||
| | `path_components: Vec<LiteralOrRegister>` | Ordered path traversal steps; mix of literals and register-based keys. |
|
||||
| `ChainedIndexParams` | `dest` | Destination register for resolved value. |
|
||||
| | `root` | Register containing the root object/collection. |
|
||||
| | `path_components: Vec<LiteralOrRegister>` | Path components applied relative to the root register. |
|
||||
| `ComprehensionBeginParams` | `mode` | Comprehension output type (array, set, object). |
|
||||
| | `collection_reg` | Source collection register for iteration. |
|
||||
| | `result_reg` | Register receiving the final collection. |
|
||||
| | `key_reg` / `value_reg` | Iteration registers (keys optional for arrays/sets). |
|
||||
| | `body_start` / `comprehension_end` | Instruction pointers framing comprehension body and exit. |
|
||||
|
||||
All parameter structs derive `Serialize`/`Deserialize` and can be stored inside
|
||||
artifacts. Some contain `Vec` fields; the compiler is responsible for ensuring
|
||||
indices remain valid and stable across serialization boundaries.
|
||||
|
||||
---
|
||||
|
||||
Reference in New Issue
Block a user