mirror of
https://github.com/microsoft/regorus.git
synced 2026-08-05 02:16:11 +00:00
* feat(compiler): support registered host-await builtins Allow hosts to register function names at compile time so that calls to those names emit HostAwait instructions directly, enabling natural syntax like fetch(x) instead of __builtin_host_await(x, "fetch"). - Add host_await_builtins map and register_host_await_builtin() to Compiler - Validate arg_count == 1 and reject reserved __builtin_host_await name - Extend determine_call_target() resolution: explicit > registered > user > builtin - Both explicit and registered paths emit identical HostAwait bytecode - Add compile_from_policy_with_host_await() entry point in rules.rs - Extended test harness with HostAwaitBuiltinSpec and args assertion - 9 YAML test cases: suspend/resume, run-to-completion, multiple names, queue, shadowing, object packing, arg_count rejection, reserved name rejection, standard builtin override - Documentation: instruction-set.md, architecture.md * Update src/languages/rego/compiler/function_calls.rs Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Mark Birger <birgerm@yandex.ru> * fix(compiler): address PR #667 review feedback on host-await registration - Compiler::register_host_await_builtin now rejects duplicate, empty, and whitespace-only names. Previously a duplicate registration would silently overwrite the existing entry, which could mask the host's own registration mistakes. - YAML test cases added: empty registration list as no-op, duplicate name rejection, empty/whitespace name rejection, out-param (a, out) calling syntax with a single-arg registered builtin, and mixed __builtin_host_await + registered builtins in the same policy consuming from their respective identifier queues. - Test harness: replace assert_eq! on HostAwait argument mismatch with anyhow::Error so mismatches propagate through the case reporter instead of panicking and skipping the harness's normal error path. - YAML comment fix: "Registration panics" -> "Registration fails with an error" (registration returns Err, never panics). Addresses anakrish + Copilot inline review comments on PR #667. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * compiler: split CallTarget::HostAwait into explicit and registered variants Addresses PR #667 review item #8: at the emit site in `compile_function_call`, the discrimination between explicit `__builtin_host_await(arg, id)` and a registered host-awaitable builtin was being recovered by string-comparing `original_fcn_path` against `"__builtin_host_await"`. The information was already known in `determine_call_target` and was being thrown away. Replace the single `CallTarget::HostAwait` variant with two: * `ExplicitHostAwait` (unit) — the two-argument call form. The identifier register comes from the user's second argument. * `RegisteredHostAwait { identifier: String }` — the one-argument call form for registered builtins. The identifier is the registered name and is captured in the variant at recognition time, so the emit site never re-derives it from the function path. This removes the magic-string comparison at the emit site (the source of truth is now `determine_call_target`) and makes both match sites in `compile_function_call` exhaustive over the two forms — adding a third host-await form in the future would force a compile error at every match site instead of silently falling through. Arities are now hardcoded in the `expected_args` extraction (`Some(2)` for explicit, `Some(1)` for registered) rather than carried in the variant; registered builtins are constrained to `arg_count == 1` at registration time, so there is no per-call variability to carry. Bytecode output is unchanged; the full RVM test suite (97 cases) and the registered_host_await suite (15 cases) pass without modification. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(compiler): clarify registered host-await intercepts unqualified calls only PR #667 review (Medium): the docs implied registered host-await names shadow user functions and builtins unconditionally, but determine_call_target matches only the bare original_fcn_path. A package-qualified call such as data.demo.resolve(x) is therefore not intercepted -- it resolves through the normal path like any other call. Rather than expand registration to qualified paths (which would let a registered name leak into every package exposing a same-named rule), document the unqualified-only behavior and pin it with tests. - register_host_await_builtin: doc now states only the unqualified call form is intercepted; qualified calls resolve normally. - determine_call_target: inline comment explaining the deliberate original_fcn_path-only match. - docs/rvm/instruction-set.md: describe qualified-call resolution, including that builtins have no qualified form. - tests: cross-package and same-package qualified calls resolve to the rule; bare-name shadowing of a standard builtin; Unknown-function outcome when no rule exists at the qualified path. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): compare host-await argument without re-running process_value PR #667 review (Low): the suspendable test harness compared the host-await argument via process_value(argument), but argument is already a runtime Value. process_value is a YAML-fixture decoder -- it rewrites "#undefined" to Undefined, {set!: [...]} to a set, and errors on a runtime Value::Set. Re-running it on the runtime argument could coerce a legitimate payload into a fixture sentinel (passing for the wrong reason) or error outright on sets. Compare the runtime argument directly against the expected value, which is already decoded once at YAML load time. Add a regression case (registered_builtin_suspendable_set_argument) that passes a set payload: it fails under the old double-processing ("unexpected set in value read from json/yaml") and passes with the fix. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(tests): reject `args:` payload expectations in run-to-completion mode PR #667 review (Low): a run-to-completion host-await response could carry an `args:` payload expectation, but RTC execution pre-loads responses and never surfaces the call argument to the harness, so the expectation was parsed and silently dropped. A case with `args: "WRONG"` passed as long as the result matched -- asserting a payload that was never checked. Reject `args:` for run-to-completion fixtures at load time, directing the author to suspendable mode where arguments are validated. Also only build the run-to-completion response vector when the case actually runs in RTC mode, so a suspendable case using the shared host_await_responses field with `args:` is not wrongly rejected. Route the fixture-load error through the same want_error handling used for compilation errors, and add registered_builtin_run_to_completion_rejects_args which now fails loudly instead of passing silently. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(compiler): reject host-await builtin names with surrounding whitespace PR #667 review (Low): register_host_await_builtin rejected all-whitespace names via name.trim().is_empty(), but accepted padded names like " lookup" or "lookup ". Those were inserted into host_await_builtins, but Rego function-call paths produce the trimmed identifier, so a padded registration could never match -- a silent dead registration. Reject any name that is not already trimmed (name != name.trim()) in addition to empty names, and update the error message accordingly. Add test cases for leading and trailing whitespace. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Signed-off-by: Mark Birger <birgerm@yandex.ru> Co-authored-by: Mark Birger <markbirger@microsoft.com> Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
262 lines
8.9 KiB
Rust
262 lines
8.9 KiB
Rust
#![allow(
|
|
missing_debug_implementations,
|
|
clippy::missing_const_for_fn,
|
|
clippy::option_if_let_else,
|
|
clippy::if_then_some_else_none,
|
|
clippy::unused_self
|
|
)] // compiler internals do not require Debug
|
|
|
|
mod comprehensions;
|
|
mod core;
|
|
mod destructuring;
|
|
mod error;
|
|
mod expressions;
|
|
mod function_calls;
|
|
mod loops;
|
|
mod program;
|
|
mod queries;
|
|
mod references;
|
|
mod rules;
|
|
|
|
pub use error::{CompilerError, Result, SpannedCompilerError};
|
|
|
|
use crate::ast::ExprRef;
|
|
use crate::lexer::Span;
|
|
use crate::rvm::program::{Program, RuleType, SpanInfo};
|
|
use crate::CompiledPolicy;
|
|
use crate::Value;
|
|
use alloc::collections::{BTreeMap, BTreeSet};
|
|
use alloc::format;
|
|
use alloc::string::String;
|
|
use alloc::string::ToString as _;
|
|
use alloc::vec;
|
|
use alloc::vec::Vec;
|
|
use indexmap::IndexMap;
|
|
|
|
pub type Register = u8;
|
|
|
|
#[derive(Debug, Clone, Default)]
|
|
struct Scope {
|
|
bound_vars: BTreeMap<String, Register>,
|
|
unbound_vars: BTreeSet<String>,
|
|
}
|
|
|
|
#[derive(Debug, Clone)]
|
|
pub enum ComprehensionType {
|
|
Array,
|
|
Object,
|
|
Set,
|
|
}
|
|
|
|
#[derive(Debug, Clone)]
|
|
pub enum ContextType {
|
|
Comprehension(ComprehensionType),
|
|
Rule(RuleType),
|
|
Every,
|
|
}
|
|
|
|
/// Compilation context for handling different types of rule bodies and comprehensions
|
|
#[derive(Debug, Clone)]
|
|
pub struct CompilationContext {
|
|
pub(super) context_type: ContextType,
|
|
pub(super) dest_register: Register,
|
|
pub(super) key_expr: Option<ExprRef>,
|
|
pub(super) value_expr: Option<ExprRef>,
|
|
pub(super) span: Span,
|
|
pub(super) key_value_loops_hoisted: bool,
|
|
}
|
|
|
|
/// Entry in the rule compilation worklist that tracks both rule path and full call stack for recursion detection
|
|
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
|
|
pub struct WorklistEntry {
|
|
/// Rule path to be compiled (e.g., "data.package.rule")
|
|
pub rule_path: String,
|
|
/// Call stack of rule indices leading to this rule (empty for entry point)
|
|
pub call_stack: Vec<u16>,
|
|
}
|
|
|
|
impl WorklistEntry {
|
|
pub fn new(rule_path: String, call_stack: Vec<u16>) -> Self {
|
|
Self {
|
|
rule_path,
|
|
call_stack,
|
|
}
|
|
}
|
|
|
|
pub fn entry_point(rule_path: String) -> Self {
|
|
Self {
|
|
rule_path,
|
|
call_stack: Vec::new(),
|
|
}
|
|
}
|
|
|
|
/// Create a new entry by extending the call stack with the caller's rule index
|
|
pub fn with_caller(
|
|
rule_path: String,
|
|
current_call_stack: &[u16],
|
|
caller_rule_index: u16,
|
|
) -> Self {
|
|
let mut new_call_stack = current_call_stack.to_vec();
|
|
new_call_stack.push(caller_rule_index);
|
|
Self {
|
|
rule_path,
|
|
call_stack: new_call_stack,
|
|
}
|
|
}
|
|
|
|
/// Check if this entry would create a recursive call
|
|
pub fn would_create_recursion(&self, target_rule_index: u16) -> bool {
|
|
self.call_stack.contains(&target_rule_index)
|
|
}
|
|
}
|
|
|
|
pub struct Compiler<'a> {
|
|
program: Program,
|
|
spans: Vec<SpanInfo>,
|
|
register_counter: Register,
|
|
scopes: Vec<Scope>,
|
|
policy: &'a CompiledPolicy,
|
|
current_package: String,
|
|
current_module_index: u32,
|
|
rule_index_map: BTreeMap<String, u16>,
|
|
rule_worklist: Vec<WorklistEntry>,
|
|
rule_definitions: Vec<Vec<Vec<u32>>>,
|
|
rule_definition_function_params: Vec<Vec<Option<Vec<String>>>>,
|
|
rule_definition_destructuring_patterns: Vec<Vec<Option<u32>>>,
|
|
/// Per-rule, per-definition: the static value produced by this definition,
|
|
/// or `None` if the value is dynamic or differs across else-branches.
|
|
/// Used to compute `RuleInfo::early_exit_on_first_success`.
|
|
rule_definition_static_values: Vec<Vec<Option<Value>>>,
|
|
rule_types: Vec<RuleType>,
|
|
rule_function_param_count: Vec<Option<usize>>,
|
|
rule_result_registers: Vec<u8>,
|
|
rule_num_registers: Vec<u8>,
|
|
context_stack: Vec<CompilationContext>,
|
|
loop_expr_register_map: BTreeMap<ExprRef, Register>,
|
|
source_to_index: BTreeMap<String, usize>,
|
|
builtin_index_map: BTreeMap<String, u16>,
|
|
current_input_register: Option<Register>,
|
|
current_data_register: Option<Register>,
|
|
current_rule_path: String,
|
|
current_call_stack: Vec<u16>,
|
|
entry_points: IndexMap<String, usize>,
|
|
soft_assert_mode: bool,
|
|
/// Registered host-awaitable builtins: name → expected arg count.
|
|
/// When the compiler encounters a call to one of these names, it emits a
|
|
/// `HostAwait` instruction instead of a regular function or builtin call.
|
|
host_await_builtins: BTreeMap<String, usize>,
|
|
}
|
|
|
|
impl<'a> Compiler<'a> {
|
|
pub fn with_policy(policy: &'a CompiledPolicy) -> Self {
|
|
let mut program = Program::new();
|
|
program.rego_v0 = policy.is_rego_v0();
|
|
Self {
|
|
program,
|
|
spans: Vec::new(),
|
|
register_counter: 1,
|
|
scopes: vec![Scope::default()],
|
|
policy,
|
|
current_package: String::new(),
|
|
current_module_index: 0,
|
|
rule_index_map: BTreeMap::new(),
|
|
rule_worklist: Vec::new(),
|
|
rule_definitions: Vec::new(),
|
|
rule_definition_function_params: Vec::new(),
|
|
rule_definition_destructuring_patterns: Vec::new(),
|
|
rule_definition_static_values: Vec::new(),
|
|
rule_types: Vec::new(),
|
|
rule_function_param_count: Vec::new(),
|
|
rule_result_registers: Vec::new(),
|
|
rule_num_registers: Vec::new(),
|
|
context_stack: vec![],
|
|
loop_expr_register_map: BTreeMap::new(),
|
|
source_to_index: BTreeMap::new(),
|
|
builtin_index_map: BTreeMap::new(),
|
|
current_input_register: None,
|
|
current_data_register: None,
|
|
current_rule_path: String::new(),
|
|
current_call_stack: Vec::new(),
|
|
entry_points: IndexMap::new(),
|
|
soft_assert_mode: false,
|
|
host_await_builtins: BTreeMap::new(),
|
|
}
|
|
}
|
|
|
|
/// Register a function name as a host-awaitable builtin.
|
|
///
|
|
/// When the compiler encounters an **unqualified** call to `name(arg)`
|
|
/// (i.e. `name(arg)` from inside the policy's own package, not
|
|
/// `data.pkg.name(arg)` or any other package-qualified form), it will
|
|
/// emit a `HostAwait` instruction with the argument and `name` as the
|
|
/// identifier, instead of treating it as a user-defined or standard
|
|
/// builtin function.
|
|
///
|
|
/// Package-qualified calls (e.g. `data.other.name(arg)`) are **not**
|
|
/// intercepted by registration. Those resolve through the normal
|
|
/// user-defined / builtin lookup against their fully-qualified path
|
|
/// (`data.other.name`).
|
|
///
|
|
/// `arg_count` must be exactly 1. The `HostAwait` instruction carries a
|
|
/// single argument register; use object packing to pass multiple values
|
|
/// (e.g. `name({"key1": v1, "key2": v2})`).
|
|
///
|
|
/// Returns `Err` when:
|
|
/// - `name` is the reserved identifier `__builtin_host_await`,
|
|
/// - `name` is empty, only whitespace, or has leading/trailing
|
|
/// whitespace (whitespace-padded names would never match the
|
|
/// trimmed identifier produced by the Rego parser, creating dead
|
|
/// registrations),
|
|
/// - `name` is already registered (duplicate registration is rejected
|
|
/// rather than silently overwritten),
|
|
/// - `arg_count` is not exactly 1.
|
|
pub fn register_host_await_builtin(&mut self, name: &str, arg_count: usize) -> Result<()> {
|
|
if name == "__builtin_host_await" {
|
|
return Err(CompilerError::General {
|
|
message: "__builtin_host_await is a reserved name and cannot be registered as a host-await builtin"
|
|
.to_string(),
|
|
}
|
|
.into());
|
|
}
|
|
if name.is_empty() || name != name.trim() {
|
|
return Err(CompilerError::General {
|
|
message: format!(
|
|
"host-await builtin name {name:?} must not be empty or contain leading/trailing whitespace"
|
|
),
|
|
}
|
|
.into());
|
|
}
|
|
if self.host_await_builtins.contains_key(name) {
|
|
return Err(CompilerError::General {
|
|
message: format!(
|
|
"host-await builtin '{name}' is already registered; \
|
|
duplicate registration is not allowed"
|
|
),
|
|
}
|
|
.into());
|
|
}
|
|
if arg_count != 1 {
|
|
return Err(CompilerError::General {
|
|
message: format!(
|
|
"registered host-await builtin '{name}' must have arg_count == 1, got {arg_count}. \
|
|
Use object packing to pass multiple values."
|
|
),
|
|
}
|
|
.into());
|
|
}
|
|
self.host_await_builtins.insert(name.to_string(), arg_count);
|
|
Ok(())
|
|
}
|
|
|
|
pub(super) fn with_soft_assert_mode<F, R>(&mut self, enabled: bool, f: F) -> R
|
|
where
|
|
F: FnOnce(&mut Self) -> R,
|
|
{
|
|
let previous = self.soft_assert_mode;
|
|
self.soft_assert_mode = enabled;
|
|
let result = f(self);
|
|
self.soft_assert_mode = previous;
|
|
result
|
|
}
|
|
}
|