diff --git a/docs/destructuring.md b/docs/destructuring.md new file mode 100644 index 0000000..d7885bf --- /dev/null +++ b/docs/destructuring.md @@ -0,0 +1,260 @@ +# Destructuring Planner + +The destructuring planner pre-computes how Rego assignments, function parameters, loop indices, and `some ... in` expressions bind variables. By materializing explicit plans during compilation, the interpreter can execute complex binding patterns without re-inspecting the abstract syntax tree (AST) each time an expression runs. + +``` ++--------------+ +----------------------------+ +-------------------+ +| AST walker | ---> | Destructuring planner core | ---> | BindingPlans table | ++--------------+ +----------------------------+ +-------------------+ + | | ^ | + | v | v + | +------------------+ +-------------------+ + | | ScopeContext | <--------> | Planner utilities | + | +------------------+ +-------------------+ + v ++------------------+ +| Scheduler output | ++------------------+ + +Downstream compiler passes reuse the same plans: + +``` +BindingPlans table + | + +--> Rego VM compiler (RVM) for bytecode emission + +--> Type propagation pass + +--> Constant folding and other analyzers +``` +``` + +## Planner building blocks + +### Scope awareness + +The planner relies on `ScopeContext` implementations to answer two questions for every variable candidate: + +| Question | Method | Why it matters | +| :----------------------------------------- | :----------------------------- | :--------------------------------------------------------------------- | +| "Is this name currently unbound?" | `is_var_unbound(var, scoping)` | Determines whether a symbol becomes a new binding or should be treated as an equality check. | +| "Has this scope already introduced the name?" | `has_same_scope_binding(var)` | Blocks same-scope rebinding for `:=` while still permitting shadowing in child scopes. | + +The planner uses two scoping modes: + +| Scoping mode | Description | Used by | +| :-------------- | :-------------------------------------------------------------------------- | :---------------------------------------------------------- | +| `RespectParent` | Honors existing bindings. Only treats names that are not yet visible as new bindings. | `=` comparisons, loop indices, `some ... in` value/key plans. | +| `AllowShadowing` | Allows new bindings even if the name is defined in an ancestor scope. | Function parameters, `:=` LHS, `some ... in` overlay contexts. | + +### Plan families + +Three layers of plan types describe the complete binding strategy. + +#### `DestructuringPlan` + +| Variant | Purpose | Notes on bindings | +| :------------------------------------- | :------------------------------------------------- | :-------------------------------------------------------------- | +| `Var(span)` | Bind the complete value to the variable at `span`. | Adds the variable to the current scope. | +| `Ignore` | Consume a wildcard (`_`). | No bindings emitted. | +| `EqualityExpr(expr)` | Require runtime equality with a dynamic expression. | Used when a candidate variable is already bound. | +| `EqualityValue(value)` | Require equality with a literal known at compile time. | Enables static structural checks. | +| `Array { element_plans }` | Destructure arrays element-by-element. | Recursively nests `DestructuringPlan` values. | +| `Object { field_plans, dynamic_fields }` | Destructure objects. Literal keys use `field_plans`; dynamic keys appear in `dynamic_fields`. | Ensures literal shape compatibility during planning. | + +#### `AssignmentPlan` + +| Variant | Triggers | Binding behavior | +| :--------------- | :-------------------------- | :-------------------------------------------------------------------------------- | +| `ColonEquals` | `:=` | Only LHS may introduce bindings; RHS must match structure/literals. Same-scope rebinding raises an error. | +| `EqualsBindLeft` | `=` where LHS has free vars | Binds the LHS pattern after structural + literal checks. | +| `EqualsBindRight` | `=` where RHS has free vars | Symmetric to `EqualsBindLeft`. | +| `EqualsBothSides` | `=` where both sides have free vars | Flattens matching sub-expressions into `(value_expr, plan)` pairs and orders them using dependency analysis. | +| `EqualityCheck` | `=` with no free vars | Pure equality comparison. | +| `WildcardMatch` | `=` when either side is `_` | Short-circuits to avoid materializing a plan. | + +#### `BindingPlan` + +| Variant | Created by | Typical consumers | +| :----------- | :---------------------------------- | :-------------------------------------------------- | +| `Assignment` | `create_assignment_binding_plan` | Rule bodies for `:=` and `=`. | +| `LoopIndex` | `create_loop_index_binding_plan` | Hoisted loops and comprehensions. | +| `Parameter` | `create_parameter_binding_plan` | Functions and rule heads. | +| `SomeIn` | `create_some_in_binding_plan` | `some key, value in collection` statements. | + +## Planner workflow + +1. **Entry point selection** — The compiler pass decides which helper to call based on the AST node (assignment, comprehension, function parameter, etc.). +2. **Pattern inspection** — `create_destructuring_plan` walks the candidate pattern and records which names would become new bindings under the selected scoping rules. +3. **Conflict detection** — The planner asks the context for same-scope bindings and raises `VariableAlreadyDefined` when a duplicate `:=` appears in the same block. +4. **Structural validation** — Helpers such as `ensure_structural_compatibility` and `ensure_literal_match` verify that literal shapes are consistent. +5. **Plan assembly** — The resulting `DestructuringPlan`, `AssignmentPlan`, or higher-level `BindingPlan` is stored in the binding lookup table for quick interpreter access. + +### Example flow + +``` +[Rule body] -- := --> [create_assignment_binding_plan] + | + v + [create_destructuring_plan] + | + +------v--------------+ + | ScopeContext checks | + +------+--------------+ + | + +-----------v-----------+ + | AssignmentPlan::ColonEquals | + +-----------+-----------+ + | + stores in BindingPlans table +``` + +## Worked examples + +Each example shows the original Rego snippet, the resulting binding plan, and highlights of the emitted bindings. + +### 1. Nested `:=` patterns + +```rego +package test + +result := { + "outer": outer, + "inner": inner, + "tag": tag, +} if { + [outer, {"meta": {"inner": inner, "tag": tag}}] := [ + "alpha", + {"meta": {"inner": "omega", "tag": "v1"}}, + ] +} +``` + +Plan overview: + +``` +BindingPlan::Assignment +└── AssignmentPlan::ColonEquals + ├── lhs_expr: array pattern + └── lhs_plan: DestructuringPlan::Array + ├── [0] -> Var("outer") + └── [1] -> DestructuringPlan::Object + └── key "meta": DestructuringPlan::Object + ├── key "inner": Var("inner") + └── key "tag": Var("tag") +``` + +| New binding | Source span | Notes | +| --- | --- | --- | +| `outer` | LHS array index 0 | New symbol in scope. | +| `inner` | Object field `meta.inner` | Shares scope with `outer`. | +| `tag` | Object field `meta.tag` | Must not reappear in same `:=` block. | + +### 2. Symmetric `=` binding + +```rego +package test + +values := [[left_id, right_id, val] | + some left, right, left_id, right_id, val + data.transitions[_] = [left, right] + [{"id": left_id, "next": {"target": right_id}}, {"id": right_id, "payload": {"value": val}}] = [left, right] +] +``` + +Plan fragments: + +``` +BindingPlan::Assignment +└── AssignmentPlan::EqualsBothSides + └── element_pairs (ordered) + 1. value_expr -> rhs[0] + plan -> DestructuringPlan::Object + key "id" -> Var("left_id") + key "next" -> DestructuringPlan::Object { key "target" -> Var("right_id") } + 2. value_expr -> rhs[1] + plan -> DestructuringPlan::Object + key "id" -> Var("right_id") + key "payload" -> DestructuringPlan::Object { key "value" -> Var("val") } +``` + +Dependency ordering ensures `left_id` is available before `right_id`/`val` comparisons run. + +### 3. Function parameter destructuring + +```rego +package test + +# f([id, payload]) := payload +f([id, payload]) := result { + result := payload +} +``` + +``` +BindingPlan::Parameter +└── param_expr: array pattern + destructuring_plan: + Array + ├── [0] -> Var("id") + └── [1] -> Var("payload") +``` + +Both bindings use `ScopingMode::AllowShadowing`, allowing `id` or `payload` to shadow outer names when the function executes. + +### 4. `some ... in` loop + +```rego +package test + +some user, record in data.users +record.role == "admin" +``` + +Plan summary: + +``` +BindingPlan::SomeIn +├── collection_expr: data.users +├── key_plan: DestructuringPlan::Var("user") +└── value_plan: DestructuringPlan::Var("record") +``` + +Tables for bindings: + +| Element | Plan | New bindings | +| --- | --- | --- | +| `key_plan` | `Var("user")` | Introduces `user` if unbound. | +| `value_plan` | `Var("record")` | Introduces `record`. | + +Literal arrays used in `collection_expr` are checked so the planner can report mismatched element shapes upfront. + +### 5. Rebinding error detection + +```rego +package test + +flag := true if { + value := "initial" + value := "shadowed" +} +``` + +``` +BindingPlan::Assignment +└── AssignmentPlan::ColonEquals (lhs := value) +``` + +During planning, the second `:=` consults `has_same_scope_binding("value")` which returns `true`. The planner emits `BindingPlannerError::VariableAlreadyDefined` and compilation reports: + +``` +error: var `value` used before definition below +``` + +## Interpreter handoff + +Planned bindings are stored in the same lookup tables as hoisted loops. At runtime the interpreter: + +1. Fetches the `BindingPlan` using `(module_id, expr_idx)`. +2. Executes the plan, binding or validating values without re-walking the AST. +3. Falls back to legacy evaluation if a plan is missing (useful for incremental compilation or mixed modules). + +This division keeps the hot execution path small while letting the compiler perform aggressive validation and error reporting ahead of time. diff --git a/src/builtins/bitwise.rs b/src/builtins/bitwise.rs index fbf2567..2a94146 100644 --- a/src/builtins/bitwise.rs +++ b/src/builtins/bitwise.rs @@ -3,7 +3,7 @@ use crate::ast::{Expr, Ref}; use crate::builtins; -use crate::builtins::utils::{ensure_args_count, ensure_numeric}; +use crate::builtins::utils::{ensure_args_count, ensure_numeric, validate_integer_arg}; use crate::lexer::Span; use crate::value::Value; @@ -19,77 +19,111 @@ pub fn register(m: &mut builtins::BuiltinsMap<&'static str, builtins::BuiltinFcn m.insert("bits.xor", (xor, 2)); } -fn and(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn and(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.and"; ensure_args_count(span, name, params, args, 2)?; let v1 = ensure_numeric(name, ¶ms[0], &args[0])?; let v2 = ensure_numeric(name, ¶ms[1], &args[1])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v1, strict, true)? + || !validate_integer_arg(name, ¶ms[1], &args[1], &v2, strict, true)? + { + return Ok(Value::Undefined); + } + Ok(match v1.and(&v2) { Some(v) => Value::from(v), _ => Value::Undefined, }) } -fn lsh(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn lsh(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.lsh"; ensure_args_count(span, name, params, args, 2)?; let v1 = ensure_numeric(name, ¶ms[0], &args[0])?; let v2 = ensure_numeric(name, ¶ms[1], &args[1])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v1, strict, true)? + || !validate_integer_arg(name, ¶ms[1], &args[1], &v2, strict, false)? + { + return Ok(Value::Undefined); + } + Ok(match v1.lsh(&v2) { Some(v) => Value::from(v), _ => Value::Undefined, }) } -fn negate(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn negate(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.negate"; ensure_args_count(span, name, params, args, 1)?; let v = ensure_numeric(name, ¶ms[0], &args[0])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v, strict, true)? { + return Ok(Value::Undefined); + } + Ok(match v.neg() { Some(v) => Value::from(v), _ => Value::Undefined, }) } -fn or(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn or(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.or"; ensure_args_count(span, name, params, args, 2)?; let v1 = ensure_numeric(name, ¶ms[0], &args[0])?; let v2 = ensure_numeric(name, ¶ms[1], &args[1])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v1, strict, true)? + || !validate_integer_arg(name, ¶ms[1], &args[1], &v2, strict, true)? + { + return Ok(Value::Undefined); + } + Ok(match v1.or(&v2) { Some(v) => Value::from(v), _ => Value::Undefined, }) } -fn rsh(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn rsh(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.rsh"; ensure_args_count(span, name, params, args, 2)?; let v1 = ensure_numeric(name, ¶ms[0], &args[0])?; let v2 = ensure_numeric(name, ¶ms[1], &args[1])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v1, strict, true)? + || !validate_integer_arg(name, ¶ms[1], &args[1], &v2, strict, false)? + { + return Ok(Value::Undefined); + } + Ok(match v1.rsh(&v2) { Some(v) => Value::from(v), _ => Value::Undefined, }) } -fn xor(span: &Span, params: &[Ref], args: &[Value], _strict: bool) -> Result { +fn xor(span: &Span, params: &[Ref], args: &[Value], strict: bool) -> Result { let name = "bits.xor"; ensure_args_count(span, name, params, args, 2)?; let v1 = ensure_numeric(name, ¶ms[0], &args[0])?; let v2 = ensure_numeric(name, ¶ms[1], &args[1])?; + if !validate_integer_arg(name, ¶ms[0], &args[0], &v1, strict, true)? + || !validate_integer_arg(name, ¶ms[1], &args[1], &v2, strict, true)? + { + return Ok(Value::Undefined); + } + Ok(match v1.xor(&v2) { Some(v) => Value::from(v), _ => Value::Undefined, diff --git a/src/builtins/utils.rs b/src/builtins/utils.rs index 798532d..1cce846 100644 --- a/src/builtins/utils.rs +++ b/src/builtins/utils.rs @@ -45,6 +45,50 @@ pub fn ensure_numeric(fcn: &str, arg: &Expr, v: &Value) -> Result { }) } +pub fn validate_integer_arg( + fcn: &str, + param: &Ref, + original_value: &Value, + numeric_value: &Number, + strict: bool, + allow_negative: bool, +) -> Result { + if !numeric_value.is_integer() { + if strict { + bail!(param.span().error( + format!("`{fcn}` expects integer arguments. Got `{original_value}`").as_str() + )); + } + return Ok(false); + } + + if !allow_negative { + if let Some(int_value) = numeric_value.as_i128() { + if int_value < 0 { + if strict { + bail!(param.span().error( + format!("`{fcn}` expects non-negative integer arguments. Got `{original_value}`") + .as_str(), + )); + } + return Ok(false); + } + } else if !numeric_value.is_positive() { + if strict { + bail!(param.span().error( + format!( + "`{fcn}` expects non-negative integer arguments. Got `{original_value}`" + ) + .as_str(), + )); + } + return Ok(false); + } + } + + Ok(true) +} + pub fn ensure_string(fcn: &str, arg: &Expr, v: &Value) -> Result> { Ok(match &v { Value::String(s) => s.clone(), diff --git a/src/compiler.rs b/src/compiler.rs index 2999ca9..61bb013 100644 --- a/src/compiler.rs +++ b/src/compiler.rs @@ -7,4 +7,5 @@ //! the compilation phase to prepare policies for efficient execution. pub mod context; +pub mod destructuring_planner; pub mod hoist; diff --git a/src/compiler/context.rs b/src/compiler/context.rs index 6bb1c45..ffecbf4 100644 --- a/src/compiler/context.rs +++ b/src/compiler/context.rs @@ -32,19 +32,34 @@ pub enum ContextType { #[derive(Debug, Clone)] pub struct ScopeContext { /// Type of context (Rule, Comprehension, Every, Query) + #[allow(dead_code)] pub context_type: ContextType, /// Variables that are bound in the current scope pub bound_vars: BTreeSet, + /// Variables that are introduced in this scope (used for conflict detection) + pub current_scope_bound_vars: BTreeSet, + /// Variables that are explicitly marked as unbound (from `some` declarations) pub unbound_vars: BTreeSet, + /// Variables that are local to this scope and will become bound once assigned + pub local_vars: BTreeSet, + + /// Flag indicating whether scheduler scope information was available + pub has_scheduler_scope: bool, + /// Key expression from rule head or object comprehension (for output expression hoisting) + #[allow(dead_code)] pub key_expr: Option, /// Value expression from rule assignment or comprehension term (for output expression hoisting) + #[allow(dead_code)] pub value_expr: Option, + + /// Shared set of module-level globals available in this scope + pub module_globals: Option>>, } impl ScopeContext { @@ -53,9 +68,13 @@ impl ScopeContext { Self { context_type: ContextType::Query, bound_vars: BTreeSet::new(), + current_scope_bound_vars: BTreeSet::new(), unbound_vars: BTreeSet::new(), + local_vars: BTreeSet::new(), + has_scheduler_scope: false, key_expr: None, value_expr: None, + module_globals: None, } } @@ -65,9 +84,13 @@ impl ScopeContext { Self { context_type, bound_vars: BTreeSet::new(), + current_scope_bound_vars: BTreeSet::new(), unbound_vars: BTreeSet::new(), + local_vars: BTreeSet::new(), + has_scheduler_scope: false, key_expr: None, value_expr: None, + module_globals: None, } } @@ -81,9 +104,13 @@ impl ScopeContext { Self { context_type, bound_vars: BTreeSet::new(), + current_scope_bound_vars: BTreeSet::new(), unbound_vars: BTreeSet::new(), + local_vars: BTreeSet::new(), + has_scheduler_scope: false, key_expr, value_expr, + module_globals: None, } } @@ -97,9 +124,13 @@ impl ScopeContext { Self { context_type, bound_vars: self.bound_vars.clone(), + current_scope_bound_vars: BTreeSet::new(), unbound_vars: self.unbound_vars.clone(), + local_vars: self.local_vars.clone(), + has_scheduler_scope: self.has_scheduler_scope, key_expr, value_expr, + module_globals: self.module_globals.clone(), } } @@ -107,13 +138,18 @@ impl ScopeContext { pub fn bind_variable(&mut self, var_name: &str) { if var_name != "_" { self.bound_vars.insert(var_name.to_string()); + self.current_scope_bound_vars.insert(var_name.to_string()); self.unbound_vars.remove(var_name); + self.local_vars.remove(var_name); } } /// Mark a variable as unbound pub fn add_unbound_variable(&mut self, var_name: &str) { - if var_name != "_" && !self.bound_vars.contains(var_name) { + if var_name != "_" { + self.bound_vars.remove(var_name); + self.current_scope_bound_vars.remove(var_name); + self.local_vars.remove(var_name); self.unbound_vars.insert(var_name.to_string()); } } @@ -126,23 +162,27 @@ impl ScopeContext { /// Check if we can determine that a variable should be treated as a loop iterator /// (either it's unbound or explicitly marked as such) pub fn should_hoist_as_loop(&self, var_name: &str) -> bool { - if var_name == "_" || self.is_unbound(var_name) { - true - } else { - // Treat variables that haven't been bound in this scope as potential loop iterators - !self.bound_vars.contains(var_name) + if var_name == "_" { + return true; } - } - /// Create a child context inheriting parent bindings, output expressions, and context type - pub fn child(&self) -> Self { - Self { - context_type: self.context_type.clone(), - bound_vars: self.bound_vars.clone(), - unbound_vars: self.unbound_vars.clone(), - key_expr: self.key_expr.clone(), - value_expr: self.value_expr.clone(), + if self + .module_globals + .as_ref() + .is_some_and(|globals| globals.contains(var_name)) + { + return false; } + + if self.is_unbound(var_name) { + return true; + } + + if self.has_scheduler_scope { + return self.local_vars.contains(var_name); + } + + !self.bound_vars.contains(var_name) } } diff --git a/src/compiler/destructuring_planner/assignment.rs b/src/compiler/destructuring_planner/assignment.rs new file mode 100644 index 0000000..5947de0 --- /dev/null +++ b/src/compiler/destructuring_planner/assignment.rs @@ -0,0 +1,401 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Assignment-specific planning utilities. + +use alloc::collections::{BTreeMap, BTreeSet}; +use alloc::string::{String, ToString}; +use alloc::vec::Vec; + +use crate::ast::{AssignOp, Expr, ExprRef}; +use crate::compiler::destructuring_planner::create_destructuring_plan; +use crate::compiler::destructuring_planner::destructuring::create_destructuring_plan_with_tracking; +use crate::compiler::destructuring_planner::utils::{ + collect_plan_var_spans, ensure_literal_match, ensure_structural_compatibility, + extract_literal_key, format_literal_key_for_error, plan_only_if_binds, +}; +use crate::compiler::destructuring_planner::{ + AssignmentPlan, BindingPlan, BindingPlannerError, DestructuringPlan, Result, ScopingMode, + VariableBindingContext, WildcardSide, +}; +use crate::lexer::Span; +use crate::query::traversal::collect_expr_dependencies; +use crate::value::Value; + +/// Convenience function for assignment expressions with specific := and = rules. +pub fn create_assignment_binding_plan( + op: AssignOp, + lhs_expr: &ExprRef, + rhs_expr: &ExprRef, + context: &T, +) -> Result { + let assignment_plan = match op { + AssignOp::ColEq => { + // For :=, only LHS can be destructured + if let Some(lhs_plan) = plan_only_if_binds(create_destructuring_plan( + lhs_expr, + context, + ScopingMode::AllowShadowing, + )) { + let mut var_spans = Vec::new(); + collect_plan_var_spans(&lhs_plan, &mut var_spans); + let mut lhs_scope_bindings = BTreeSet::new(); + for span in var_spans { + let name = span.text().to_string(); + let is_duplicate = !lhs_scope_bindings.insert(name.clone()); + let has_same_scope_binding = context.has_same_scope_binding(&name); + + if is_duplicate || has_same_scope_binding { + return Err(BindingPlannerError::VariableAlreadyDefined { + var: name, + span, + }); + } + } + + ensure_structural_compatibility(lhs_expr, rhs_expr)?; + ensure_literal_match(&lhs_plan, rhs_expr)?; + AssignmentPlan::ColonEquals { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + lhs_plan, + } + } else { + return Err(BindingPlannerError::ColonEqualsRequiresBindableLeft { + span: lhs_expr.span().clone(), + }); + } + } + + AssignOp::Eq => { + let lhs_struct_plan = + create_destructuring_plan(lhs_expr, context, ScopingMode::RespectParent); + let rhs_struct_plan = + create_destructuring_plan(rhs_expr, context, ScopingMode::RespectParent); + + let lhs_plan = plan_only_if_binds(lhs_struct_plan.clone()); + let rhs_plan = plan_only_if_binds(rhs_struct_plan.clone()); + + let lhs_is_wildcard = + matches!(lhs_expr.as_ref(), Expr::Var { span, .. } if span.text() == "_"); + let rhs_is_wildcard = + matches!(rhs_expr.as_ref(), Expr::Var { span, .. } if span.text() == "_"); + + if lhs_is_wildcard || rhs_is_wildcard { + let wildcard_side = match (lhs_is_wildcard, rhs_is_wildcard) { + (true, true) => WildcardSide::Both, + (true, false) => WildcardSide::Lhs, + (false, true) => WildcardSide::Rhs, + (false, false) => unreachable!(), + }; + + AssignmentPlan::WildcardMatch { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + wildcard_side, + } + } else if lhs_plan.is_some() && rhs_plan.is_some() { + // Both sides have unbound vars - recursively flatten all nested structures + let mut element_pairs = Vec::new(); + let mut newly_bound = BTreeSet::new(); + flatten_assignment_pairs( + lhs_expr, + rhs_expr, + context, + &mut newly_bound, + &mut element_pairs, + )?; + order_element_pairs(&mut element_pairs, context); + + AssignmentPlan::EqualsBothSides { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + element_pairs, + } + } else if lhs_plan.is_none() && rhs_plan.is_none() { + AssignmentPlan::EqualityCheck { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + } + } else if let Some(lhs) = lhs_plan { + ensure_structural_compatibility(lhs_expr, rhs_expr)?; + ensure_literal_match(&lhs, rhs_expr)?; + + AssignmentPlan::EqualsBindLeft { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + lhs_plan: lhs, + } + } else if let Some(rhs) = rhs_plan { + ensure_structural_compatibility(rhs_expr, lhs_expr)?; + ensure_literal_match(&rhs, lhs_expr)?; + + AssignmentPlan::EqualsBindRight { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + rhs_plan: rhs, + } + } else if let Some(lhs) = lhs_struct_plan { + ensure_structural_compatibility(lhs_expr, rhs_expr)?; + ensure_literal_match(&lhs, rhs_expr)?; + + AssignmentPlan::EqualsBindLeft { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + lhs_plan: lhs, + } + } else if let Some(rhs) = rhs_struct_plan { + ensure_structural_compatibility(rhs_expr, lhs_expr)?; + ensure_literal_match(&rhs, lhs_expr)?; + + AssignmentPlan::EqualsBindRight { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + rhs_plan: rhs, + } + } else { + AssignmentPlan::EqualityCheck { + lhs_expr: lhs_expr.clone(), + rhs_expr: rhs_expr.clone(), + } + } + } + }; + + Ok(BindingPlan::Assignment { + plan: assignment_plan, + }) +} + +/// Recursively flatten assignment destructuring into (value_expr, pattern_plan) pairs. +fn flatten_assignment_pairs( + lhs_expr: &ExprRef, + rhs_expr: &ExprRef, + context: &T, + newly_bound: &mut BTreeSet, + pairs: &mut Vec<(ExprRef, DestructuringPlan)>, +) -> Result<()> { + let (lhs_plan, lhs_delta) = + preview_binding_plan(lhs_expr, context, ScopingMode::RespectParent, newly_bound); + let (rhs_plan, rhs_delta) = + preview_binding_plan(rhs_expr, context, ScopingMode::RespectParent, newly_bound); + + if lhs_plan.is_none() && rhs_plan.is_none() { + pairs.push(( + rhs_expr.clone(), + DestructuringPlan::EqualityExpr(lhs_expr.clone()), + )); + return Ok(()); + } + + let lhs_is_array = matches!(lhs_expr.as_ref(), Expr::Array { .. }); + let rhs_is_array = matches!(rhs_expr.as_ref(), Expr::Array { .. }); + let lhs_is_object = matches!(lhs_expr.as_ref(), Expr::Object { .. }); + let rhs_is_object = matches!(rhs_expr.as_ref(), Expr::Object { .. }); + + if (lhs_is_array && rhs_is_object) || (lhs_is_object && rhs_is_array) { + return Err(BindingPlannerError::IncompatibleDestructuringPatterns { + span: lhs_expr.span().clone(), + }); + } + + if lhs_is_array && rhs_is_array { + for (lhs_item, rhs_item) in collect_array_pairs(lhs_expr, rhs_expr)? { + flatten_assignment_pairs(&lhs_item, &rhs_item, context, newly_bound, pairs)?; + } + return Ok(()); + } + + if lhs_is_object && rhs_is_object { + if let Some(object_pairs) = collect_object_pairs(lhs_expr, rhs_expr)? { + for (lhs_value, rhs_value) in object_pairs { + flatten_assignment_pairs(&lhs_value, &rhs_value, context, newly_bound, pairs)?; + } + return Ok(()); + } + } + + match (lhs_plan, rhs_plan, lhs_expr.as_ref(), rhs_expr.as_ref()) { + // Case 1: LHS has pattern, RHS is value - add pair + (Some(lhs_pattern), None, _, _) => { + newly_bound.extend(lhs_delta); + pairs.push((rhs_expr.clone(), lhs_pattern)); + } + // Case 2: RHS has pattern, LHS is value - add pair + (None, Some(rhs_pattern), _, _) => { + newly_bound.extend(rhs_delta); + pairs.push((lhs_expr.clone(), rhs_pattern)); + } + // Case 5: Both have patterns but incompatible structures + (Some(_), Some(_), _, _) => { + return Err(BindingPlannerError::IncompatibleDestructuringPatterns { + span: lhs_expr.span().clone(), + }); + } + // Remaining cases are handled by earlier match arms and guard + (None, None, _, _) => { + unreachable!("handled by equality guard above"); + } + } + + Ok(()) +} + +fn collect_array_pairs(lhs_expr: &ExprRef, rhs_expr: &ExprRef) -> Result> { + let lhs_items = match lhs_expr.as_ref() { + Expr::Array { items, .. } => items, + _ => unreachable!(), + }; + let rhs_items = match rhs_expr.as_ref() { + Expr::Array { items, .. } => items, + _ => unreachable!(), + }; + + if lhs_items.len() != rhs_items.len() { + return Err(BindingPlannerError::ArraySizeMismatch { + left_size: lhs_items.len(), + right_size: rhs_items.len(), + span: lhs_expr.span().clone(), + }); + } + + Ok(lhs_items + .iter() + .cloned() + .zip(rhs_items.iter().cloned()) + .collect()) +} + +fn order_element_pairs( + element_pairs: &mut Vec<(ExprRef, DestructuringPlan)>, + context: &T, +) { + if element_pairs.len() <= 1 { + return; + } + + let mut remaining: Vec<_> = element_pairs + .drain(..) + .map(|(value_expr, plan)| { + let binds = plan.bound_vars().into_iter().collect::>(); + let deps = collect_expr_dependencies(&value_expr); + (value_expr, plan, deps, binds) + }) + .collect(); + + if remaining.iter().any(|(_, _, deps, _)| deps.is_none()) { + *element_pairs = remaining + .into_iter() + .map(|(value_expr, plan, _, _)| (value_expr, plan)) + .collect(); + return; + } + + let mut scheduled = BTreeSet::new(); + let mut ordered = Vec::with_capacity(remaining.len()); + + while !remaining.is_empty() { + let mut progress = false; + + for idx in 0..remaining.len() { + let (_, _, deps, _) = &remaining[idx]; + let deps = deps.as_ref().expect("checked above"); + let ready = deps.iter().all(|var| { + scheduled.contains(var) || !context.is_var_unbound(var, ScopingMode::RespectParent) + }); + + if ready { + let (value_expr, plan, _deps, binds) = remaining.remove(idx); + scheduled.extend(binds.into_iter()); + ordered.push((value_expr, plan)); + progress = true; + break; + } + } + + if !progress { + ordered.extend( + remaining + .into_iter() + .map(|(value_expr, plan, _, _)| (value_expr, plan)), + ); + break; + } + } + + *element_pairs = ordered; +} + +fn collect_object_pairs( + lhs_expr: &ExprRef, + rhs_expr: &ExprRef, +) -> Result>> { + let lhs_fields = match lhs_expr.as_ref() { + Expr::Object { fields, .. } => fields, + _ => unreachable!(), + }; + let rhs_fields = match rhs_expr.as_ref() { + Expr::Object { fields, .. } => fields, + _ => unreachable!(), + }; + + let mut lhs_map: BTreeMap = BTreeMap::new(); + for (_, key_expr, val_expr) in lhs_fields { + if let Some(key_value) = extract_literal_key(key_expr) { + lhs_map.insert(key_value, val_expr.clone()); + } else { + return Ok(None); + } + } + + let mut pairs = Vec::with_capacity(lhs_map.len()); + let mut rhs_literal_count = 0; + let mut missing_literal_key: Option<(String, Span)> = None; + + for (_, key_expr, val_expr) in rhs_fields { + if let Some(key_value) = extract_literal_key(key_expr) { + rhs_literal_count += 1; + + if let Some(lhs_value_expr) = lhs_map.get(&key_value) { + pairs.push((lhs_value_expr.clone(), val_expr.clone())); + } else if missing_literal_key.is_none() { + missing_literal_key = Some(( + format_literal_key_for_error(&key_value), + key_expr.span().clone(), + )); + } + } else { + return Ok(None); + } + } + + if rhs_literal_count != lhs_map.len() { + return Err(BindingPlannerError::ObjectFieldCountMismatch { + left_count: lhs_map.len(), + right_count: rhs_literal_count, + span: lhs_expr.span().clone(), + }); + } + + if let Some((key, span)) = missing_literal_key { + return Err(BindingPlannerError::ObjectKeyNotFound { key, span }); + } + + Ok(Some(pairs)) +} + +fn preview_binding_plan( + expr: &ExprRef, + context: &T, + scoping: ScopingMode, + already_bound: &BTreeSet, +) -> (Option, BTreeSet) { + let mut scratch = already_bound.clone(); + let plan = create_destructuring_plan_with_tracking(expr, context, scoping, &mut scratch); + let mut delta: BTreeSet = scratch.difference(already_bound).cloned().collect(); + let plan = plan_only_if_binds(plan); + if plan.is_none() { + delta.clear(); + } + (plan, delta) +} diff --git a/src/compiler/destructuring_planner/context.rs b/src/compiler/destructuring_planner/context.rs new file mode 100644 index 0000000..0a68068 --- /dev/null +++ b/src/compiler/destructuring_planner/context.rs @@ -0,0 +1,56 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Context traits shared across planner submodules. + +use alloc::collections::BTreeSet; +use alloc::string::String; + +/// Scoping mode for variable binding decisions. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum ScopingMode { + /// Respect existing bindings from parent scopes (normal scoping). + RespectParent, + /// Allow shadowing of parent scope bindings (local scoping). + AllowShadowing, +} + +/// Trait for determining variable binding status within a scope context. +pub trait VariableBindingContext { + /// Check if a variable is unbound in this context. + /// + /// # Arguments + /// * `var_name` - The name of the variable to check + /// * `scoping` - Whether to respect parent scopes or allow shadowing + fn is_var_unbound(&self, var_name: &str, scoping: ScopingMode) -> bool; + + /// Determine whether the current scope already bound this variable. + /// + /// This is used to detect same-scope rebinding conflicts when the planner + /// encounters `:=` assignments, while still allowing shadowing in nested scopes. + fn has_same_scope_binding(&self, var_name: &str) -> bool; +} + +/// Context overlay that tracks newly bound variables on top of an existing context. +pub(crate) struct OverlayBindingContext<'a, T: VariableBindingContext> { + pub(crate) base: &'a T, + pub(crate) newly_bound: &'a BTreeSet, +} + +impl<'a, T: VariableBindingContext> VariableBindingContext for OverlayBindingContext<'a, T> { + fn is_var_unbound(&self, var_name: &str, scoping: ScopingMode) -> bool { + if self.newly_bound.contains(var_name) { + return false; + } + + self.base.is_var_unbound(var_name, scoping) + } + + fn has_same_scope_binding(&self, var_name: &str) -> bool { + if self.newly_bound.contains(var_name) { + return true; + } + + self.base.has_same_scope_binding(var_name) + } +} diff --git a/src/compiler/destructuring_planner/destructuring.rs b/src/compiler/destructuring_planner/destructuring.rs new file mode 100644 index 0000000..98819d6 --- /dev/null +++ b/src/compiler/destructuring_planner/destructuring.rs @@ -0,0 +1,106 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Functions responsible for building destructuring plans. + +use alloc::collections::{BTreeMap, BTreeSet}; +use alloc::string::{String, ToString}; +use alloc::vec::Vec; + +use crate::ast::{Expr, ExprRef}; +use crate::compiler::destructuring_planner::context::OverlayBindingContext; +use crate::compiler::destructuring_planner::utils::extract_literal_key; +use crate::compiler::destructuring_planner::{ + DestructuringPlan, ScopingMode, VariableBindingContext, +}; + +/// Create a destructuring plan for an expression using a variable binding context. +pub fn create_destructuring_plan( + expr: &ExprRef, + context: &T, + scoping: ScopingMode, +) -> Option { + let mut newly_bound = BTreeSet::new(); + create_destructuring_plan_with_tracking(expr, context, scoping, &mut newly_bound) +} + +/// Create a destructuring plan while tracking newly bound variables. +pub(crate) fn create_destructuring_plan_with_tracking( + expr: &ExprRef, + context: &T, + scoping: ScopingMode, + newly_bound: &mut BTreeSet, +) -> Option { + let overlay = OverlayBindingContext { + base: context, + newly_bound, + }; + + match expr.as_ref() { + // Variable binding + Expr::Var { span: name, .. } => { + if name.text() == "_" { + return Some(DestructuringPlan::Ignore); + } + if overlay.is_var_unbound(name.text(), scoping) { + newly_bound.insert(name.text().to_string()); + Some(DestructuringPlan::Var(name.clone())) + } else { + // Already bound - treat as equality check + Some(DestructuringPlan::EqualityExpr(expr.clone())) + } + } + + Expr::String { value, .. } + | Expr::RawString { value, .. } + | Expr::Number { value, .. } + | Expr::Bool { value, .. } + | Expr::Null { value, .. } => Some(DestructuringPlan::EqualityValue(value.clone())), + + // Array destructuring + Expr::Array { items, .. } => { + let mut element_plans = Vec::new(); + for item in items { + if let Some(plan) = + create_destructuring_plan_with_tracking(item, context, scoping, newly_bound) + { + element_plans.push(plan); + } else { + // If any element can't be destructured, fail the whole array + return None; + } + } + Some(DestructuringPlan::Array { element_plans }) + } + + // Object destructuring + Expr::Object { fields, .. } => { + let mut field_plans = BTreeMap::new(); + let mut dynamic_fields = Vec::new(); + for (_, key_expr, value_expr) in fields { + if let Some(value_plan) = create_destructuring_plan_with_tracking( + value_expr, + context, + scoping, + newly_bound, + ) { + if let Some(key_value) = extract_literal_key(key_expr) { + field_plans.insert(key_value, value_plan); + } else { + dynamic_fields.push((key_expr.clone(), value_plan)); + } + } else { + return None; + } + } + + Some(DestructuringPlan::Object { + field_plans, + dynamic_fields, + }) + } + + // For all others, treat as equality checks + _ => Some(DestructuringPlan::EqualityExpr(expr.clone())), + } +} diff --git a/src/compiler/destructuring_planner/error.rs b/src/compiler/destructuring_planner/error.rs new file mode 100644 index 0000000..0145d62 --- /dev/null +++ b/src/compiler/destructuring_planner/error.rs @@ -0,0 +1,173 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Error definitions for the destructuring planner. + +use alloc::format; +use alloc::string::String; +use alloc::vec::Vec; +use anyhow::Error; +use core::error::Error as CoreError; +use core::fmt; + +use crate::lexer::Span; + +/// Errors produced while building binding plans. +#[derive(Debug)] +pub enum BindingPlannerError { + /// Assignment operator := requires left-hand side to have bindable variables. + ColonEqualsRequiresBindableLeft { span: Span }, + /// Array size mismatch in assignment destructuring. + ArraySizeMismatch { + left_size: usize, + right_size: usize, + span: Span, + }, + /// Array length mismatch detected while planning evaluation (no assignments). + ArrayLengthMismatch { + expected: usize, + actual: usize, + span: Span, + }, + /// Object literal keys mismatch detected while planning evaluation (no assignments). + ObjectLiteralKeysMismatch { + expected: Vec, + actual: Vec, + span: Span, + }, + /// Object field count mismatch in assignment destructuring. + ObjectFieldCountMismatch { + left_count: usize, + right_count: usize, + span: Span, + }, + /// Object key not found in destructuring. + ObjectKeyNotFound { key: String, span: Span }, + /// Variable reuse detected when a new binding is required. + VariableAlreadyDefined { var: String, span: Span }, + /// Incompatible destructuring patterns. + IncompatibleDestructuringPatterns { span: Span }, + /// Failed to create destructuring plan. + FailedToCreateDestructuringPlan { plan_type: String, span: Span }, +} + +/// Result alias used throughout the binding planner. +pub type Result = core::result::Result; + +/// Convert planner errors into diagnostic-rich anyhow errors for compiler callers. +pub fn map_binding_error(err: BindingPlannerError) -> Error { + match err { + BindingPlannerError::ColonEqualsRequiresBindableLeft { span } => span + .error("assignment operator := requires left-hand side to have bindable variables"), + BindingPlannerError::ArraySizeMismatch { span, .. } + | BindingPlannerError::ArrayLengthMismatch { span, .. } => { + span.error("mismatch in number of array elements") + } + BindingPlannerError::ObjectLiteralKeysMismatch { + expected, + actual, + span, + } => span.error(&format!( + "object literal keys mismatch. Expected keys {:?} got {:?}.", + expected, actual + )), + BindingPlannerError::ObjectFieldCountMismatch { + left_count, + right_count, + span, + } => span.error(&format!( + "object field count mismatch in assignment: left has {left_count} fields, right has {right_count} fields" + )), + BindingPlannerError::ObjectKeyNotFound { key, span } => span + .error(&format!("key \"{key}\" not found in left-hand side object during destructuring")), + BindingPlannerError::VariableAlreadyDefined { var, span } => { + span.error(&format!("var `{var}` used before definition below")) + } + BindingPlannerError::IncompatibleDestructuringPatterns { span } => span.error( + "incompatible destructuring patterns: both sides must be arrays or objects with matching structure", + ), + BindingPlannerError::FailedToCreateDestructuringPlan { plan_type, span } => { + span.error(&format!("failed to create {plan_type} destructuring plan")) + } + } +} + +impl BindingPlannerError { + pub(crate) fn to_span_message(&self) -> String { + match self { + BindingPlannerError::ColonEqualsRequiresBindableLeft { span } => span + .message( + "error", + "assignment operator := requires left-hand side to have bindable variables", + ), + BindingPlannerError::ArraySizeMismatch { + left_size, + right_size, + span, + } => { + let detail = format!( + "mismatch in number of array elements (left has {left_size}, right has {right_size})" + ); + span.message("error", detail.as_str()) + } + BindingPlannerError::ArrayLengthMismatch { + expected, + actual, + span, + } => { + let detail = format!( + "array length mismatch. Expected {expected} got {actual}." + ); + span.message("error", detail.as_str()) + } + BindingPlannerError::ObjectLiteralKeysMismatch { + expected, + actual, + span, + } => { + let detail = format!( + "object literal keys mismatch. Expected keys {:?} got {:?}.", + expected, actual + ); + span.message("error", detail.as_str()) + } + BindingPlannerError::ObjectFieldCountMismatch { + left_count, + right_count, + span, + } => { + let detail = format!( + "object field count mismatch in assignment: left has {left_count} fields, right has {right_count} fields" + ); + span.message("error", detail.as_str()) + } + BindingPlannerError::ObjectKeyNotFound { key, span } => { + let detail = format!( + "key \"{key}\" not found in left-hand side object during destructuring" + ); + span.message("error", detail.as_str()) + } + BindingPlannerError::VariableAlreadyDefined { var, span } => { + let detail = format!("var `{var}` used before definition below"); + span.message("error", detail.as_str()) + } + BindingPlannerError::IncompatibleDestructuringPatterns { span } => span + .message( + "error", + "incompatible destructuring patterns: both sides must be arrays or objects with matching structure", + ), + BindingPlannerError::FailedToCreateDestructuringPlan { plan_type, span } => { + let detail = format!("failed to create {plan_type} destructuring plan"); + span.message("error", detail.as_str()) + } + } + } +} + +impl fmt::Display for BindingPlannerError { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + write!(f, "{}", self.to_span_message()) + } +} + +impl CoreError for BindingPlannerError {} diff --git a/src/compiler/destructuring_planner/mod.rs b/src/compiler/destructuring_planner/mod.rs new file mode 100644 index 0000000..cb7bcec --- /dev/null +++ b/src/compiler/destructuring_planner/mod.rs @@ -0,0 +1,25 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Destructuring and binding planner utilities. +//! +//! This module hierarchy reorganizes the binding planner into smaller components +//! so the compiler can evolve without ambiguity with FFI bindings. Submodules +//! will be filled in as code is migrated from the legacy `bindings` module. + +pub mod assignment; +pub mod context; +pub mod destructuring; +pub mod error; +pub mod parameters; +pub mod plans; +pub mod some_in; +pub mod utils; + +pub use assignment::create_assignment_binding_plan; +pub use context::{ScopingMode, VariableBindingContext}; +pub use destructuring::create_destructuring_plan; +pub use error::{map_binding_error, BindingPlannerError, Result}; +pub use parameters::{create_loop_index_binding_plan, create_parameter_binding_plan}; +pub use plans::{AssignmentPlan, BindingPlan, DestructuringPlan, WildcardSide}; +pub use some_in::create_some_in_binding_plan; diff --git a/src/compiler/destructuring_planner/parameters.rs b/src/compiler/destructuring_planner/parameters.rs new file mode 100644 index 0000000..fd5574a --- /dev/null +++ b/src/compiler/destructuring_planner/parameters.rs @@ -0,0 +1,59 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Planner helpers for function parameters and loop indices. + +use alloc::collections::BTreeSet; +use alloc::string::ToString; + +use crate::ast::ExprRef; +use crate::compiler::destructuring_planner::create_destructuring_plan; +use crate::compiler::destructuring_planner::destructuring::create_destructuring_plan_with_tracking; +use crate::compiler::destructuring_planner::utils::validate_pattern_bindings; +use crate::compiler::destructuring_planner::{ + BindingPlan, BindingPlannerError, Result, ScopingMode, VariableBindingContext, +}; + +/// Convenience function for loop index expressions (respects parent scope). +pub fn create_loop_index_binding_plan( + index_expr: &ExprRef, + context: &T, +) -> Result { + let destructuring_plan = + create_destructuring_plan(index_expr, context, ScopingMode::RespectParent).ok_or_else( + || BindingPlannerError::FailedToCreateDestructuringPlan { + plan_type: "loop index".to_string(), + span: index_expr.span().clone(), + }, + )?; + + Ok(BindingPlan::LoopIndex { + index_expr: index_expr.clone(), + destructuring_plan, + }) +} + +/// Convenience function for function parameters (always allow shadowing). +pub fn create_parameter_binding_plan( + param_expr: &ExprRef, + context: &T, +) -> Result { + let mut newly_bound = BTreeSet::new(); + let destructuring_plan = create_destructuring_plan_with_tracking( + param_expr, + context, + ScopingMode::AllowShadowing, + &mut newly_bound, + ) + .ok_or_else(|| BindingPlannerError::FailedToCreateDestructuringPlan { + plan_type: "parameter".to_string(), + span: param_expr.span().clone(), + })?; + + validate_pattern_bindings(param_expr, &newly_bound, context)?; + + Ok(BindingPlan::Parameter { + param_expr: param_expr.clone(), + destructuring_plan, + }) +} diff --git a/src/compiler/destructuring_planner/plans.rs b/src/compiler/destructuring_planner/plans.rs new file mode 100644 index 0000000..aec9e14 --- /dev/null +++ b/src/compiler/destructuring_planner/plans.rs @@ -0,0 +1,241 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Core data structures used by the destructuring planner. + +use alloc::collections::BTreeMap; +use alloc::string::{String, ToString}; +use alloc::vec::Vec; + +use crate::ast::ExprRef; +use crate::lexer::Span; +use crate::value::Value; + +/// Strategy for how to destructure an expression. +#[derive(Debug, Clone)] +pub enum DestructuringPlan { + /// Bind the entire value to a variable. + Var(Span), + + /// Ignore the value completely. + Ignore, + + /// Check equality with the value of the dynamic expression. + EqualityExpr(ExprRef), + + /// Check equality against a literal value captured at planning time. + EqualityValue(Value), + + /// Destructure an array. + Array { + element_plans: Vec, + }, + + /// Destructure an object. + Object { + field_plans: BTreeMap, + dynamic_fields: Vec<(ExprRef, DestructuringPlan)>, + }, +} + +impl DestructuringPlan { + fn collect_bound_vars(&self, vars: &mut Vec) { + match self { + DestructuringPlan::Var(name) => vars.push(name.text().to_string()), + DestructuringPlan::Array { element_plans } => { + for element in element_plans { + element.collect_bound_vars(vars); + } + } + DestructuringPlan::Object { + field_plans, + dynamic_fields, + } => { + for plan in field_plans.values() { + plan.collect_bound_vars(vars); + } + for (_, plan) in dynamic_fields { + plan.collect_bound_vars(vars); + } + } + DestructuringPlan::Ignore + | DestructuringPlan::EqualityExpr(_) + | DestructuringPlan::EqualityValue(_) => {} + } + } + + pub(crate) fn contains_wildcards(&self) -> bool { + match self { + DestructuringPlan::Ignore => true, + DestructuringPlan::Array { element_plans } => element_plans + .iter() + .any(DestructuringPlan::contains_wildcards), + DestructuringPlan::Object { + field_plans, + dynamic_fields, + } => { + field_plans + .values() + .any(DestructuringPlan::contains_wildcards) + || dynamic_fields + .iter() + .any(|(_, plan)| plan.contains_wildcards()) + } + DestructuringPlan::Var(_) + | DestructuringPlan::EqualityExpr(_) + | DestructuringPlan::EqualityValue(_) => false, + } + } + + pub(crate) fn bound_vars(&self) -> Vec { + let mut vars = Vec::new(); + self.collect_bound_vars(&mut vars); + vars + } + + pub(crate) fn introduces_binding(&self) -> bool { + match self { + DestructuringPlan::Var(_) | DestructuringPlan::Ignore => true, + DestructuringPlan::Array { element_plans } => element_plans + .iter() + .any(DestructuringPlan::introduces_binding), + DestructuringPlan::Object { + field_plans, + dynamic_fields, + } => { + field_plans + .values() + .any(DestructuringPlan::introduces_binding) + || dynamic_fields + .iter() + .any(|(_, plan)| plan.introduces_binding()) + } + DestructuringPlan::EqualityExpr(_) | DestructuringPlan::EqualityValue(_) => false, + } + } +} + +/// Strategy-based assignment plan with specific rules for := and = operators. +#[derive(Debug, Clone)] +#[allow(dead_code)] +pub enum AssignmentPlan { + /// For := (ColEq) - only LHS can have patterns. + ColonEquals { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + lhs_plan: DestructuringPlan, + }, + + /// For = (Eq) - only one side has unbound variables. + EqualsBindLeft { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + lhs_plan: DestructuringPlan, + }, + + /// For = (Eq) - only one side has unbound variables. + EqualsBindRight { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + rhs_plan: DestructuringPlan, + }, + + /// For = (Eq) - both sides have unbound vars, flattened to pairs. + /// Each pair is (value_expr, destructuring_plan_for_pattern). + EqualsBothSides { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + element_pairs: Vec<(ExprRef, DestructuringPlan)>, + }, + + /// No variables to bind - simple equality check. + EqualityCheck { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + }, + + /// No variables to bind and at least one side is a wildcard `_`. + WildcardMatch { + lhs_expr: ExprRef, + rhs_expr: ExprRef, + wildcard_side: WildcardSide, + }, +} + +impl AssignmentPlan { + pub(crate) fn bound_vars(&self) -> Vec { + match self { + AssignmentPlan::ColonEquals { lhs_plan, .. } + | AssignmentPlan::EqualsBindLeft { lhs_plan, .. } => lhs_plan.bound_vars(), + AssignmentPlan::EqualsBindRight { rhs_plan, .. } => rhs_plan.bound_vars(), + AssignmentPlan::EqualsBothSides { element_pairs, .. } => { + let mut vars = Vec::new(); + for (_, plan) in element_pairs { + vars.extend(plan.bound_vars()); + } + vars + } + AssignmentPlan::EqualityCheck { .. } | AssignmentPlan::WildcardMatch { .. } => { + Vec::new() + } + } + } +} + +/// Indicates which side of an equality expression contains a wildcard `_`. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +#[allow(dead_code)] +pub enum WildcardSide { + Lhs, + Rhs, + Both, +} + +/// High-level plan describing how bindings are produced in various contexts. +#[derive(Debug, Clone)] +#[allow(dead_code)] +pub enum BindingPlan { + Assignment { + plan: AssignmentPlan, + }, + LoopIndex { + index_expr: ExprRef, + destructuring_plan: DestructuringPlan, + }, + Parameter { + param_expr: ExprRef, + destructuring_plan: DestructuringPlan, + }, + SomeIn { + collection_expr: ExprRef, + key_plan: Option, + value_plan: DestructuringPlan, + }, +} + +impl BindingPlan { + /// Return the set of variables newly bound by this plan. + pub fn bound_vars(&self) -> Vec { + match self { + BindingPlan::Assignment { plan } => plan.bound_vars(), + BindingPlan::LoopIndex { + destructuring_plan, .. + } => destructuring_plan.bound_vars(), + BindingPlan::Parameter { + destructuring_plan, .. + } => destructuring_plan.bound_vars(), + BindingPlan::SomeIn { + key_plan, + value_plan, + .. + } => { + let mut vars = Vec::new(); + if let Some(key_destructuring) = key_plan { + vars.extend(key_destructuring.bound_vars()); + } + vars.extend(value_plan.bound_vars()); + vars + } + } + } +} diff --git a/src/compiler/destructuring_planner/some_in.rs b/src/compiler/destructuring_planner/some_in.rs new file mode 100644 index 0000000..d7eba39 --- /dev/null +++ b/src/compiler/destructuring_planner/some_in.rs @@ -0,0 +1,92 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Planner support for `some .. in` expressions. + +use alloc::collections::BTreeSet; +use alloc::string::String; + +use crate::ast::{Expr, ExprRef}; +use crate::compiler::destructuring_planner::destructuring::create_destructuring_plan_with_tracking; +use crate::compiler::destructuring_planner::utils::{ + check_literal_structure, validate_pattern_bindings, LiteralStructureCheck, +}; +use crate::compiler::destructuring_planner::{ + BindingPlan, BindingPlannerError, Result, ScopingMode, VariableBindingContext, +}; + +/// Convenience function for some..in expressions (always allow shadowing for new bindings). +pub fn create_some_in_binding_plan( + key_expr: &Option, + value_expr: &ExprRef, + collection_expr: &ExprRef, + context: &T, +) -> Result { + let key_plan = if let Some(key) = key_expr { + let mut newly_bound = BTreeSet::new(); + let plan = create_destructuring_plan_with_tracking( + key, + context, + ScopingMode::RespectParent, + &mut newly_bound, + ); + if let Some(plan) = plan { + validate_pattern_bindings(key, &newly_bound, context)?; + Some(plan) + } else { + None + } + } else { + None + }; + + let mut newly_bound_value = BTreeSet::new(); + let value_plan = create_destructuring_plan_with_tracking( + value_expr, + context, + ScopingMode::RespectParent, + &mut newly_bound_value, + ) + .ok_or_else(|| BindingPlannerError::FailedToCreateDestructuringPlan { + plan_type: String::from("some-in value"), + span: value_expr.span().clone(), + })?; + + validate_pattern_bindings(value_expr, &newly_bound_value, context)?; + + if let Expr::Array { items, .. } = collection_expr.as_ref() { + let mut found_match = false; + let mut mismatch_error: Option = None; + let mut saw_unknown = false; + + for collection_item in items { + match check_literal_structure(&value_plan, collection_item) { + LiteralStructureCheck::Match => { + found_match = true; + } + LiteralStructureCheck::Unknown => { + saw_unknown = true; + } + mismatch => { + if let Some(err) = mismatch.into_error() { + if mismatch_error.is_none() { + mismatch_error = Some(err); + } + } + } + } + } + + if !found_match && !saw_unknown { + if let Some(err) = mismatch_error { + return Err(err); + } + } + } + + Ok(BindingPlan::SomeIn { + collection_expr: collection_expr.clone(), + key_plan, + value_plan, + }) +} diff --git a/src/compiler/destructuring_planner/utils.rs b/src/compiler/destructuring_planner/utils.rs new file mode 100644 index 0000000..3f1afaa --- /dev/null +++ b/src/compiler/destructuring_planner/utils.rs @@ -0,0 +1,271 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +//! Shared helper routines for the destructuring planner modules. + +use alloc::collections::BTreeSet; +use alloc::string::{String, ToString}; +use alloc::vec::Vec; + +use crate::ast::{Expr, ExprRef}; +use crate::compiler::destructuring_planner::{ + BindingPlannerError, DestructuringPlan, Result, ScopingMode, VariableBindingContext, +}; +use crate::lexer::Span; +use crate::value::Value; + +/// Result of statically comparing a destructuring plan with a literal expression. +pub(crate) enum LiteralStructureCheck { + Match, + Unknown, + ArrayMismatch { + expected: usize, + actual: usize, + span: Span, + }, + ObjectMismatch { + expected: Vec, + actual: Vec, + span: Span, + }, +} + +impl LiteralStructureCheck { + pub(crate) fn into_error(self) -> Option { + match self { + LiteralStructureCheck::Match | LiteralStructureCheck::Unknown => None, + LiteralStructureCheck::ArrayMismatch { + expected, + actual, + span, + } => Some(BindingPlannerError::ArrayLengthMismatch { + expected, + actual, + span, + }), + LiteralStructureCheck::ObjectMismatch { + expected, + actual, + span, + } => Some(BindingPlannerError::ObjectLiteralKeysMismatch { + expected, + actual, + span, + }), + } + } +} + +/// Compare a destructuring plan against a literal expression. +pub(crate) fn check_literal_structure( + plan: &DestructuringPlan, + expr: &ExprRef, +) -> LiteralStructureCheck { + match (plan, expr.as_ref()) { + (DestructuringPlan::Array { element_plans }, Expr::Array { items, .. }) => { + if items.len() != element_plans.len() { + return LiteralStructureCheck::ArrayMismatch { + expected: element_plans.len(), + actual: items.len(), + span: expr.span().clone(), + }; + } + + for (nested_plan, nested_expr) in element_plans.iter().zip(items.iter()) { + match check_literal_structure(nested_plan, nested_expr) { + LiteralStructureCheck::Match => {} + LiteralStructureCheck::Unknown => return LiteralStructureCheck::Unknown, + mismatch => return mismatch, + } + } + + LiteralStructureCheck::Match + } + ( + DestructuringPlan::Object { + field_plans, + dynamic_fields, + }, + Expr::Object { fields, .. }, + ) => { + if field_plans.is_empty() && dynamic_fields.is_empty() { + return LiteralStructureCheck::Match; + } + + if !field_plans.is_empty() { + let expected_keys: Vec = field_plans + .keys() + .map(format_literal_key_for_error) + .collect(); + + let mut literal_fields: Vec<(Value, &ExprRef)> = Vec::new(); + let mut actual_keys: Vec = Vec::new(); + for (_, key_expr, value_expr) in fields { + if let Some(key_value) = extract_literal_key(key_expr) { + actual_keys.push(format_literal_key_for_error(&key_value)); + literal_fields.push((key_value, value_expr)); + } + } + + let mut expected_sorted = expected_keys.clone(); + expected_sorted.sort(); + let mut actual_sorted = actual_keys.clone(); + actual_sorted.sort(); + + if expected_sorted != actual_sorted { + return LiteralStructureCheck::ObjectMismatch { + expected: expected_keys, + actual: actual_keys, + span: expr.span().clone(), + }; + } + + for (key_value, value_expr) in literal_fields { + if let Some(field_plan) = field_plans.get(&key_value) { + match check_literal_structure(field_plan, value_expr) { + LiteralStructureCheck::Match => {} + LiteralStructureCheck::Unknown => { + return LiteralStructureCheck::Unknown + } + mismatch => return mismatch, + } + } + } + + if dynamic_fields.is_empty() { + return LiteralStructureCheck::Match; + } + } + + LiteralStructureCheck::Unknown + } + _ => LiteralStructureCheck::Unknown, + } +} + +pub(crate) fn ensure_literal_match(plan: &DestructuringPlan, expr: &ExprRef) -> Result<()> { + match check_literal_structure(plan, expr).into_error() { + Some(err) => Err(err), + None => Ok(()), + } +} + +pub(crate) fn collect_pattern_var_spans(expr: &ExprRef, spans: &mut Vec) { + match expr.as_ref() { + Expr::Var { span, .. } => { + let name = span.text(); + if name != "_" && name != "input" && name != "data" { + spans.push(span.clone()); + } + } + Expr::Array { items, .. } => { + for item in items { + collect_pattern_var_spans(item, spans); + } + } + Expr::Set { items, .. } => { + for item in items { + collect_pattern_var_spans(item, spans); + } + } + Expr::Object { fields, .. } => { + for (_, _, value_expr) in fields { + collect_pattern_var_spans(value_expr, spans); + } + } + _ => {} + } +} + +pub(crate) fn validate_pattern_bindings( + expr: &ExprRef, + newly_bound: &BTreeSet, + context: &T, +) -> Result<()> { + let mut candidate_vars = Vec::new(); + collect_pattern_var_spans(expr, &mut candidate_vars); + + for span in candidate_vars { + let name = span.text().to_string(); + if newly_bound.contains(&name) { + continue; + } + if !context.is_var_unbound(&name, ScopingMode::RespectParent) { + return Err(BindingPlannerError::VariableAlreadyDefined { var: name, span }); + } + } + + Ok(()) +} + +pub(crate) fn collect_plan_var_spans(plan: &DestructuringPlan, spans: &mut Vec) { + match plan { + DestructuringPlan::Var(span) => spans.push(span.clone()), + DestructuringPlan::Array { element_plans } => { + for nested in element_plans { + collect_plan_var_spans(nested, spans); + } + } + DestructuringPlan::Object { + field_plans, + dynamic_fields, + } => { + for nested in field_plans.values() { + collect_plan_var_spans(nested, spans); + } + for (_, nested) in dynamic_fields { + collect_plan_var_spans(nested, spans); + } + } + DestructuringPlan::Ignore + | DestructuringPlan::EqualityExpr(_) + | DestructuringPlan::EqualityValue(_) => {} + } +} + +pub(crate) fn ensure_structural_compatibility( + lhs_expr: &ExprRef, + rhs_expr: &ExprRef, +) -> Result<()> { + let lhs_is_array = matches!(lhs_expr.as_ref(), Expr::Array { .. }); + let rhs_is_array = matches!(rhs_expr.as_ref(), Expr::Array { .. }); + let lhs_is_object = matches!(lhs_expr.as_ref(), Expr::Object { .. }); + let rhs_is_object = matches!(rhs_expr.as_ref(), Expr::Object { .. }); + + if (lhs_is_array && rhs_is_object) || (lhs_is_object && rhs_is_array) { + return Err(BindingPlannerError::IncompatibleDestructuringPatterns { + span: lhs_expr.span().clone(), + }); + } + + Ok(()) +} + +/// Helper that discards destructuring plans which do not bind any variables. +pub(crate) fn plan_only_if_binds(plan: Option) -> Option { + plan.and_then(|plan| { + if plan.introduces_binding() || plan.contains_wildcards() { + Some(plan) + } else { + None + } + }) +} + +pub(crate) fn extract_literal_key(expr: &ExprRef) -> Option { + match expr.as_ref() { + Expr::String { value, .. } => Some(value.clone()), + Expr::RawString { value, .. } => Some(value.clone()), + Expr::Number { value, .. } => Some(value.clone()), + Expr::Bool { value, .. } => Some(value.clone()), + Expr::Null { .. } => Some(Value::Null), + _ => None, + } +} + +pub(crate) fn format_literal_key_for_error(value: &Value) -> String { + match value { + Value::String(s) => s.as_ref().to_string(), + _ => value.to_string(), + } +} diff --git a/src/compiler/hoist.rs b/src/compiler/hoist.rs index 0b15fa5..801d024 100644 --- a/src/compiler/hoist.rs +++ b/src/compiler/hoist.rs @@ -7,14 +7,66 @@ //! pre-computing loop hoisting information that can be stored in the //! compiled policy and reused by the interpreter. +use super::destructuring_planner::{ + map_binding_error, BindingPlan, ScopingMode, VariableBindingContext, +}; use crate::ast::{Expr, ExprRef, Literal, LiteralStmt, Module, Query, Ref, Rule, RuleHead}; use crate::compiler::context::{ContextType, ScopeContext}; use crate::lookup::Lookup; +use crate::scheduler::compute_module_globals; use crate::*; -use anyhow::Result; +use anyhow::{anyhow, Result}; +use alloc::collections::BTreeSet; use alloc::vec::Vec; +/// Implementation of VariableBindingContext for ScopeContext +impl VariableBindingContext for ScopeContext { + fn is_var_unbound(&self, var_name: &str, scoping: ScopingMode) -> bool { + if var_name == "_" { + return false; + } + + if self.unbound_vars.contains(var_name) { + return true; + } + + if self.has_scheduler_scope && self.local_vars.contains(var_name) { + return true; + } + + match scoping { + ScopingMode::AllowShadowing => { + // Allow shadowing - always consider variables as potentially unbound + true + } + ScopingMode::RespectParent => { + // Respect parent scope bindings + if self + .module_globals + .as_ref() + .is_some_and(|globals| globals.contains(var_name)) + { + return false; + } + + if self.bound_vars.contains(var_name) { + return false; + } + true + } + } + } + + fn has_same_scope_binding(&self, var_name: &str) -> bool { + if var_name == "_" { + return false; + } + + self.current_scope_bound_vars.contains(var_name) + } +} + /// Type of loop that was hoisted #[derive(Debug, Clone, PartialEq, Eq)] pub enum LoopType { @@ -54,6 +106,10 @@ pub struct HoistedLoopsLookup { /// For output expressions in comprehensions and rule values expr_loops: Lookup>, + /// Maps (module_index, expr_index) -> BindingPlan + /// Stores pre-computed binding plans for assignment-style expressions + expr_binding_plans: Lookup, + /// Maps (module_index, query_index) -> ScopeContext /// Stores compilation contexts for queries (rules, comprehensions, every) query_contexts: Lookup, @@ -69,6 +125,7 @@ impl HoistedLoopsLookup { pub fn ensure_statement_capacity(&mut self, module_idx: u32, stmt_idx: u32) { self.statement_loops.ensure_capacity(module_idx, stmt_idx); self.expr_loops.ensure_capacity(module_idx, 0); + self.expr_binding_plans.ensure_capacity(module_idx, 0); self.query_contexts.ensure_capacity(module_idx, 0); } @@ -76,6 +133,8 @@ impl HoistedLoopsLookup { pub fn ensure_expr_capacity(&mut self, module_idx: u32, expr_idx: u32) { self.expr_loops.ensure_capacity(module_idx, expr_idx); self.statement_loops.ensure_capacity(module_idx, 0); + self.expr_binding_plans + .ensure_capacity(module_idx, expr_idx); self.query_contexts.ensure_capacity(module_idx, 0); } @@ -84,6 +143,7 @@ impl HoistedLoopsLookup { self.query_contexts.ensure_capacity(module_idx, query_idx); self.statement_loops.ensure_capacity(module_idx, 0); self.expr_loops.ensure_capacity(module_idx, 0); + self.expr_binding_plans.ensure_capacity(module_idx, 0); } /// Store hoisted loops for a statement @@ -111,11 +171,22 @@ impl HoistedLoopsLookup { self.query_contexts.set(module_idx, query_idx, context); } + /// Store a binding plan for an expression + pub fn set_expr_binding_plan(&mut self, module_idx: u32, expr_idx: u32, plan: BindingPlan) { + self.expr_binding_plans.set(module_idx, expr_idx, plan); + } + /// Get the compilation context for a query + #[allow(dead_code)] pub fn get_query_context(&self, module_idx: u32, query_idx: u32) -> Option<&ScopeContext> { self.query_contexts.get_checked(module_idx, query_idx) } + /// Get the binding plan for an expression + pub fn get_expr_binding_plan(&self, module_idx: u32, expr_idx: u32) -> Option<&BindingPlan> { + self.expr_binding_plans.get_checked(module_idx, expr_idx) + } + /// Merge another loop hoisting table into this one /// This is used to add query module loops to the existing table pub fn merge_query_loops(&mut self, mut other: HoistedLoopsLookup, module_idx: usize) { @@ -127,6 +198,10 @@ impl HoistedLoopsLookup { self.expr_loops.push_module(Vec::new()); } + while self.expr_binding_plans.module_len() < module_idx { + self.expr_binding_plans.push_module(Vec::new()); + } + while self.query_contexts.module_len() < module_idx { self.query_contexts.push_module(Vec::new()); } @@ -141,6 +216,10 @@ impl HoistedLoopsLookup { self.expr_loops.push_module(module); } + if let Some(module) = other.expr_binding_plans.remove_module(query_module_idx) { + self.expr_binding_plans.push_module(module); + } + if let Some(module) = other.query_contexts.remove_module(query_module_idx) { self.query_contexts.push_module(module); } @@ -149,6 +228,7 @@ impl HoistedLoopsLookup { pub fn truncate_modules(&mut self, module_count: usize) { self.statement_loops.truncate_modules(module_count); self.expr_loops.truncate_modules(module_count); + self.expr_binding_plans.truncate_modules(module_count); self.query_contexts.truncate_modules(module_count); } @@ -163,6 +243,7 @@ impl HoistedLoopsLookup { pub struct LoopHoister { lookup: HoistedLoopsLookup, schedule: Option>, + module_globals: Lookup>>, } impl LoopHoister { @@ -171,6 +252,7 @@ impl LoopHoister { Self { lookup: HoistedLoopsLookup::new(), schedule: None, + module_globals: Lookup::new(), } } @@ -179,18 +261,30 @@ impl LoopHoister { Self { lookup: HoistedLoopsLookup::new(), schedule: Some(schedule), + module_globals: Lookup::new(), } } /// Populate loop hoisting information for all modules /// Returns the populated lookup table pub fn populate(mut self, modules: &[Ref]) -> Result { + self.module_globals = compute_module_globals(modules).map_err(|err| anyhow!(err))?; for (module_idx, module) in modules.iter().enumerate() { self.populate_module(module_idx as u32, module)?; } Ok(self.lookup) } + fn create_scope_context(&self, module_idx: u32) -> ScopeContext { + let mut context = ScopeContext::new(); + + if let Some(globals) = self.module_globals.get_checked(module_idx, 0) { + context.module_globals = Some(globals.clone()); + } + + context + } + /// Populate loop hoisting information for all modules, with extra capacity /// for additional modules that will be added later (e.g., query modules) /// @@ -202,6 +296,7 @@ impl LoopHoister { modules: &[Ref], extra_capacity: u32, ) -> Result { + self.module_globals = compute_module_globals(modules).map_err(|err| anyhow!(err))?; for (module_idx, module) in modules.iter().enumerate() { self.populate_module(module_idx as u32, module)?; } @@ -212,6 +307,9 @@ impl LoopHoister { self.lookup .ensure_statement_capacity(last_module_idx + i, 0); self.lookup.ensure_expr_capacity(last_module_idx + i, 0); + self.module_globals.ensure_capacity(last_module_idx + i, 0); + self.module_globals + .set(last_module_idx + i, 0, crate::Rc::new(BTreeSet::new())); } Ok(self.lookup) } @@ -257,10 +355,18 @@ impl LoopHoister { .ensure_expr_capacity(module_idx, num_expressions - 1); } + self.module_globals.ensure_capacity(module_idx, 0); + let mut reserved_globals = BTreeSet::new(); + reserved_globals.insert("data".to_string()); + reserved_globals.insert("input".to_string()); + self.module_globals + .set(module_idx, 0, crate::Rc::new(reserved_globals)); + // Populate the query with default context - let context = ScopeContext::new(); + let context = self.create_scope_context(module_idx); self.lookup.ensure_query_capacity(module_idx, query.qidx); - self.populate_query(module_idx, query, &context).map(|_| ()) + self.populate_query(module_idx, query, &context)?; + Ok(()) } /// Populate loop information for a single rule @@ -268,11 +374,31 @@ impl LoopHoister { match rule { Rule::Spec { head, bodies, .. } => { // Create a context for this rule - let mut context = ScopeContext::new(); + let mut context = self.create_scope_context(module_idx); // Bind function parameters if this is a function rule if let RuleHead::Func { args, .. } = head { for param in args { + // Create binding plan for function parameter + match super::destructuring_planner::create_parameter_binding_plan( + param, &context, + ) { + Ok(binding_plan) => { + let expr_idx = param.as_ref().eidx(); + self.lookup.ensure_expr_capacity(module_idx, expr_idx); + + // Immediately bind variables from the plan to context + Self::bind_vars_from_plan_to_context(&binding_plan, &mut context); + + self.lookup.set_expr_binding_plan( + module_idx, + expr_idx, + binding_plan, + ); + } + Err(err) => return Err(map_binding_error(err)), + } + // Extract variable name from parameter expression if let Expr::Var { span, .. } = param.as_ref() { context.bind_variable(span.text()); @@ -309,11 +435,13 @@ impl LoopHoister { // Process each rule body (definitions) for body in bodies { // Create a context with the output expressions (using Rule context type) - let body_context = context.child_with_output_exprs( + let mut body_context = context.child_with_output_exprs( ContextType::Rule, key_expr.clone(), value_expr.clone(), ); + body_context.current_scope_bound_vars = + context.current_scope_bound_vars.clone(); // Store the context for this query let populated_body_context = @@ -348,11 +476,13 @@ impl LoopHoister { // Handle rules with head assignments but no bodies (e.g., `y := "string"`) if bodies.is_empty() { - let body_context = context.child_with_output_exprs( + let mut body_context = context.child_with_output_exprs( ContextType::Rule, key_expr.clone(), value_expr.clone(), ); + body_context.current_scope_bound_vars = + context.current_scope_bound_vars.clone(); if let Some(ref key) = key_expr { self.populate_output_expr(module_idx, key, &body_context)?; @@ -365,7 +495,7 @@ impl LoopHoister { } Rule::Default { value, .. } => { // For default rules, just process the value expression - let context = ScopeContext::new(); + let context = self.create_scope_context(module_idx); self.populate_output_expr(module_idx, value, &context)?; } } @@ -380,7 +510,8 @@ impl LoopHoister { query: &Query, parent_context: &ScopeContext, ) -> Result { - let mut context = parent_context.child(); + let mut context = parent_context.clone(); + context.current_scope_bound_vars = parent_context.current_scope_bound_vars.clone(); // Get the scheduled order if available let stmt_order: Vec = if let Some(ref schedule) = self.schedule { @@ -423,93 +554,26 @@ impl LoopHoister { } } - // Traverse literal expressions to populate nested contexts (comprehensions, every, etc.) - self.process_literal_for_contexts(module_idx, &stmt.literal, context)?; + let mut loops = Vec::new(); + self.analyze_literal(module_idx, &stmt.literal, context, &mut loops)?; + for with_mod in &stmt.with_mods { - self.process_expr_for_contexts(module_idx, &with_mod.refr, context)?; - self.process_expr_for_contexts(module_idx, &with_mod.r#as, context)?; + self.analyze_expr(module_idx, &with_mod.refr, context, &mut loops)?; + self.analyze_expr(module_idx, &with_mod.r#as, context, &mut loops)?; } - // Hoist loops from this statement using populated contexts - let loops = - self.hoist_loops_from_literal_with_context(module_idx, &stmt.literal, context)?; - - // Always store in lookup table, even if no loops (store empty vec) - // This ensures the interpreter can always find an entry self.lookup.ensure_statement_capacity(module_idx, stmt_idx); self.lookup.set_statement_loops(module_idx, stmt_idx, loops); - // Update context based on variable bindings in this statement - self.update_context_from_literal(&stmt.literal, context); - Ok(()) } - /// Hoist loops from a literal with variable binding context - fn hoist_loops_from_literal_with_context( - &self, - module_idx: u32, - literal: &Literal, - context: &ScopeContext, - ) -> Result> { - let mut loops = Vec::new(); - - use Literal::*; - match literal { - SomeIn { - key, - value, - collection, - .. - } => { - // Recursively hoist from sub-expressions first - if let Some(key) = key { - self.hoist_loops_from_expr_with_context(module_idx, key, &mut loops, context)?; - } - self.hoist_loops_from_expr_with_context(module_idx, value, &mut loops, context)?; - self.hoist_loops_from_expr_with_context( - module_idx, collection, &mut loops, context, - )?; - } - Expr { expr, .. } => { - // Hoist loops from expressions (like array[_] patterns) - self.hoist_loops_from_expr_with_context(module_idx, expr, &mut loops, context)?; - } - Every { domain, query, .. } => { - // Hoist from domain expression - self.hoist_loops_from_expr_with_context(module_idx, domain, &mut loops, context)?; - - // Process the Every query in a child context - let child_context = self - .lookup - .get_query_context(module_idx, query.qidx) - .cloned() - .unwrap_or_else(|| context.child()); - for stmt in &query.stmts { - self.hoist_loops_from_literal_with_context( - module_idx, - &stmt.literal, - &child_context, - )?; - } - } - NotExpr { expr, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, expr, &mut loops, context)?; - } - _ => { - // Other literal types don't have loops to hoist - } - } - - Ok(loops) - } - - /// Traverse literals to populate nested contexts (comprehensions, every, etc.) - fn process_literal_for_contexts( + fn analyze_literal( &mut self, module_idx: u32, literal: &Literal, - context: &ScopeContext, + context: &mut ScopeContext, + loops: &mut Vec, ) -> Result<()> { use Literal::*; @@ -520,31 +584,38 @@ impl LoopHoister { collection, .. } => { + let binding_plan = super::destructuring_planner::create_some_in_binding_plan( + key, value, collection, context, + ) + .map_err(map_binding_error)?; + + let expr_idx = collection.as_ref().eidx(); + self.lookup.ensure_expr_capacity(module_idx, expr_idx); + Self::bind_vars_from_plan_to_context(&binding_plan, context); + self.lookup + .set_expr_binding_plan(module_idx, expr_idx, binding_plan); + if let Some(key_expr) = key { - self.process_expr_for_contexts(module_idx, key_expr, context)?; + self.analyze_expr(module_idx, key_expr, context, loops)?; } - self.process_expr_for_contexts(module_idx, value, context)?; - self.process_expr_for_contexts(module_idx, collection, context)?; + self.analyze_expr(module_idx, value, context, loops)?; + self.analyze_expr(module_idx, collection, context, loops)?; } - Expr { expr, .. } | NotExpr { expr, .. } => { - self.process_expr_for_contexts(module_idx, expr, context)?; + Expr { expr, .. } => { + self.analyze_expr(module_idx, expr, context, loops)?; } Every { domain, query, .. } => { - // Process the domain expression for nested contexts - self.process_expr_for_contexts(module_idx, domain, context)?; + self.analyze_expr(module_idx, domain, context, loops)?; - // Create a child context for the Every quantifier let every_context = context.child_with_output_exprs(ContextType::Every, None, None); - let populated_every_context = + let populated_context = self.populate_query(module_idx, query.as_ref(), &every_context)?; self.lookup.ensure_query_capacity(module_idx, query.qidx); - self.lookup.set_query_context( - module_idx, - query.qidx, - populated_every_context.clone(), - ); - - // Nested query already processed for hoisting via populated context + self.lookup + .set_query_context(module_idx, query.qidx, populated_context); + } + NotExpr { expr, .. } => { + self.analyze_expr(module_idx, expr, context, loops)?; } _ => {} } @@ -552,25 +623,31 @@ impl LoopHoister { Ok(()) } - /// Traverse expressions to populate nested contexts (comprehensions, function params, etc.) - fn process_expr_for_contexts( + fn analyze_expr( &mut self, module_idx: u32, expr: &ExprRef, - context: &ScopeContext, + context: &mut ScopeContext, + loops: &mut Vec, ) -> Result<()> { use crate::ast::Expr as E; match expr.as_ref() { + E::String { .. } + | E::RawString { .. } + | E::Number { .. } + | E::Bool { .. } + | E::Null { .. } + | E::Var { .. } => {} E::Array { items, .. } | E::Set { items, .. } => { for item in items { - self.process_expr_for_contexts(module_idx, item, context)?; + self.analyze_expr(module_idx, item, context, loops)?; } } E::Object { fields, .. } => { for (_, key_expr, value_expr) in fields { - self.process_expr_for_contexts(module_idx, key_expr, context)?; - self.process_expr_for_contexts(module_idx, value_expr, context)?; + self.analyze_expr(module_idx, key_expr, context, loops)?; + self.analyze_expr(module_idx, value_expr, context, loops)?; } } E::ArrayCompr { term, query, .. } | E::SetCompr { term, query, .. } => { @@ -579,17 +656,12 @@ impl LoopHoister { None, Some(term.clone()), ); - - let populated_compr_context = + let populated_context = self.populate_query(module_idx, query.as_ref(), &compr_context)?; self.lookup.ensure_query_capacity(module_idx, query.qidx); - self.lookup.set_query_context( - module_idx, - query.qidx, - populated_compr_context.clone(), - ); - - self.populate_output_expr(module_idx, term, &populated_compr_context)?; + self.lookup + .set_query_context(module_idx, query.qidx, populated_context.clone()); + self.populate_output_expr_with_context(module_idx, term, &populated_context)?; } E::ObjectCompr { key, value, query, .. @@ -599,154 +671,21 @@ impl LoopHoister { Some(key.clone()), Some(value.clone()), ); - - let populated_compr_context = + let populated_context = self.populate_query(module_idx, query.as_ref(), &compr_context)?; self.lookup.ensure_query_capacity(module_idx, query.qidx); - self.lookup.set_query_context( - module_idx, - query.qidx, - populated_compr_context.clone(), - ); - - self.populate_output_expr(module_idx, key, &populated_compr_context)?; - self.populate_output_expr(module_idx, value, &populated_compr_context)?; + self.lookup + .set_query_context(module_idx, query.qidx, populated_context.clone()); + self.populate_output_expr_with_context(module_idx, key, &populated_context)?; + self.populate_output_expr_with_context(module_idx, value, &populated_context)?; } E::Call { fcn, params, .. } => { - self.process_expr_for_contexts(module_idx, fcn, context)?; + self.analyze_expr(module_idx, fcn, context, loops)?; for param in params { - self.process_expr_for_contexts(module_idx, param, context)?; - } - } - E::UnaryExpr { expr, .. } => { - self.process_expr_for_contexts(module_idx, expr, context)?; - } - E::RefDot { refr, .. } => { - self.process_expr_for_contexts(module_idx, refr, context)?; - } - E::RefBrack { refr, index, .. } => { - self.process_expr_for_contexts(module_idx, refr, context)?; - self.process_expr_for_contexts(module_idx, index, context)?; - } - E::BinExpr { lhs, rhs, .. } - | E::BoolExpr { lhs, rhs, .. } - | E::ArithExpr { lhs, rhs, .. } => { - self.process_expr_for_contexts(module_idx, lhs, context)?; - self.process_expr_for_contexts(module_idx, rhs, context)?; - } - E::AssignExpr { lhs, rhs, .. } => { - self.process_expr_for_contexts(module_idx, lhs, context)?; - self.process_expr_for_contexts(module_idx, rhs, context)?; - } - E::Membership { - key, - value, - collection, - .. - } => { - if let Some(key_expr) = key { - self.process_expr_for_contexts(module_idx, key_expr, context)?; - } - self.process_expr_for_contexts(module_idx, value, context)?; - self.process_expr_for_contexts(module_idx, collection, context)?; - } - #[cfg(feature = "rego-extensions")] - E::OrExpr { lhs, rhs, .. } => { - self.process_expr_for_contexts(module_idx, lhs, context)?; - self.process_expr_for_contexts(module_idx, rhs, context)?; - } - _ => {} - } - - Ok(()) - } - - /// Hoist loops from expressions with variable binding context - fn hoist_loops_from_expr_with_context( - &self, - module_idx: u32, - expr: &ExprRef, - loops: &mut Vec, - context: &ScopeContext, - ) -> Result<()> { - use Expr::*; - match expr.as_ref() { - // Primitive types - no loops to hoist - String { .. } - | RawString { .. } - | Number { .. } - | Bool { .. } - | Null { .. } - | Var { .. } => { - // No sub-expressions to process - } - - // Collection types - hoist from items - Array { items, .. } => { - for item in items { - self.hoist_loops_from_expr_with_context(module_idx, item, loops, context)?; - } - } - Set { items, .. } => { - for item in items { - self.hoist_loops_from_expr_with_context(module_idx, item, loops, context)?; - } - } - Object { fields, .. } => { - for (_, key_expr, value_expr) in fields { - self.hoist_loops_from_expr_with_context(module_idx, key_expr, loops, context)?; - self.hoist_loops_from_expr_with_context( - module_idx, value_expr, loops, context, - )?; - } - } - - // Comprehensions - process their queries - // Note: Comprehension contexts and output expressions will be handled - // by populate_comprehension called from the parent expression processing - ArrayCompr { term, query, .. } | SetCompr { term, query, .. } => { - let child_context = self - .lookup - .get_query_context(module_idx, query.qidx) - .cloned() - .unwrap_or_else(|| context.child()); - for stmt in &query.stmts { - self.hoist_loops_from_literal_with_context( - module_idx, - &stmt.literal, - &child_context, - )?; - } - self.hoist_loops_from_expr_with_context(module_idx, term, loops, &child_context)?; - } - ObjectCompr { - key, value, query, .. - } => { - let child_context = self - .lookup - .get_query_context(module_idx, query.qidx) - .cloned() - .unwrap_or_else(|| context.child()); - for stmt in &query.stmts { - self.hoist_loops_from_literal_with_context( - module_idx, - &stmt.literal, - &child_context, - )?; - } - self.hoist_loops_from_expr_with_context(module_idx, key, loops, &child_context)?; - self.hoist_loops_from_expr_with_context(module_idx, value, loops, &child_context)?; - } - - // Function calls - check for walk() builtin which generates loops - Call { fcn, params, .. } => { - // First hoist loops in parameters. - for param in params { - self.hoist_loops_from_expr_with_context(module_idx, param, loops, context)?; + self.analyze_expr(module_idx, param, context, loops)?; } - // Check if this is a walk() call - let is_walk = if let Var { + let is_walk = if let E::Var { value: Value::String(name), .. } = fcn.as_ref() @@ -759,35 +698,55 @@ impl LoopHoister { if is_walk { loops.push(HoistedLoop { loop_expr: Some(expr.clone()), - key: None, // walk doesn't have an index + key: None, value: expr.clone(), - collection: expr.clone(), // The walk call itself + collection: expr.clone(), loop_type: LoopType::Walk, }); - return Ok(()); } + // If the last parameter expression contains unbound vars, create a binding plan + if let Some(last_param) = params.last() { + match super::destructuring_planner::create_parameter_binding_plan( + last_param, context, + ) { + Ok(binding_plan) => { + let expr_idx = last_param.as_ref().eidx(); + self.lookup.ensure_expr_capacity(module_idx, expr_idx); - // For other function calls, hoist loops in parameters + // Immediately bind variables from the plan to context + Self::bind_vars_from_plan_to_context(&binding_plan, context); + + self.lookup + .set_expr_binding_plan(module_idx, expr_idx, binding_plan); + } + Err(err) => return Err(map_binding_error(err)), + } + } } - - // Unary expressions - hoist from operand - UnaryExpr { expr, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, expr, loops, context)?; + E::UnaryExpr { expr, .. } => { + self.analyze_expr(module_idx, expr, context, loops)?; } - - // Reference expressions - check for array[_] patterns - RefDot { refr, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, refr, loops, context)?; + E::RefDot { refr, .. } => { + self.analyze_expr(module_idx, refr, context, loops)?; } - RefBrack { refr, index, .. } => { - // Recursively hoist from sub-expressions - self.hoist_loops_from_expr_with_context(module_idx, refr, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, index, loops, context)?; + E::RefBrack { refr, index, .. } => { + self.analyze_expr(module_idx, refr, context, loops)?; + self.analyze_expr(module_idx, index, context, loops)?; - // Check if the index expression contains unbound variables - // This handles both simple cases like array[x] and complex cases like array[[x, y]] if Self::expr_contains_unbound_vars(index, context) { - // This index contains unbound variables - create a loop to iterate + match super::destructuring_planner::create_loop_index_binding_plan( + index, context, + ) { + Ok(binding_plan) => { + let expr_idx = index.as_ref().eidx(); + self.lookup.ensure_expr_capacity(module_idx, expr_idx); + Self::bind_vars_from_plan_to_context(&binding_plan, context); + self.lookup + .set_expr_binding_plan(module_idx, expr_idx, binding_plan); + } + Err(err) => return Err(map_binding_error(err)), + } + loops.push(HoistedLoop { loop_expr: Some(expr.clone()), key: Some(index.clone()), @@ -795,97 +754,59 @@ impl LoopHoister { collection: refr.clone(), loop_type: LoopType::IndexIteration, }); - return Ok(()); } } + E::BinExpr { lhs, rhs, .. } + | E::BoolExpr { lhs, rhs, .. } + | E::ArithExpr { lhs, rhs, .. } => { + self.analyze_expr(module_idx, lhs, context, loops)?; + self.analyze_expr(module_idx, rhs, context, loops)?; + } + E::AssignExpr { op, lhs, rhs, .. } => { + let binding_plan = super::destructuring_planner::create_assignment_binding_plan( + op.clone(), + lhs, + rhs, + context, + ) + .map_err(map_binding_error)?; - // Binary expressions - hoist from both operands - BinExpr { lhs, rhs, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, lhs, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, rhs, loops, context)?; - } - BoolExpr { lhs, rhs, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, lhs, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, rhs, loops, context)?; - } - ArithExpr { lhs, rhs, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, lhs, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, rhs, loops, context)?; - } - AssignExpr { lhs, rhs, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, lhs, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, rhs, loops, context)?; - } + let expr_idx = expr.as_ref().eidx(); + self.lookup.ensure_expr_capacity(module_idx, expr_idx); + Self::bind_vars_from_plan_to_context(&binding_plan, context); + self.lookup + .set_expr_binding_plan(module_idx, expr_idx, binding_plan); - // Membership expressions - hoist from key, value, and collection - Membership { + self.analyze_expr(module_idx, lhs, context, loops)?; + self.analyze_expr(module_idx, rhs, context, loops)?; + } + E::Membership { key, value, collection, .. } => { if let Some(key_expr) = key { - self.hoist_loops_from_expr_with_context(module_idx, key_expr, loops, context)?; + self.analyze_expr(module_idx, key_expr, context, loops)?; } - self.hoist_loops_from_expr_with_context(module_idx, value, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, collection, loops, context)?; + self.analyze_expr(module_idx, value, context, loops)?; + self.analyze_expr(module_idx, collection, context, loops)?; } - - // Handle conditionally compiled expression types #[cfg(feature = "rego-extensions")] - OrExpr { lhs, rhs, .. } => { - self.hoist_loops_from_expr_with_context(module_idx, lhs, loops, context)?; - self.hoist_loops_from_expr_with_context(module_idx, rhs, loops, context)?; + E::OrExpr { lhs, rhs, .. } => { + self.analyze_expr(module_idx, lhs, context, loops)?; + self.analyze_expr(module_idx, rhs, context, loops)?; } } + Ok(()) } - /// Update context based on variable bindings in a literal - fn update_context_from_literal(&self, literal: &Literal, context: &mut ScopeContext) { - use crate::ast::Expr as E; - use Literal::*; - match literal { - SomeIn { key, value, .. } => { - // Bind the loop variables - if let Some(key_expr) = key { - if let E::Var { span, .. } = key_expr.as_ref() { - context.bind_variable(span.text()); - } - } - if let E::Var { span, .. } = value.as_ref() { - context.bind_variable(span.text()); - } - } - Expr { expr, .. } => { - // Look for assignment expressions that bind variables - if let E::AssignExpr { lhs, .. } = expr.as_ref() { - Self::bind_variables_from_expr(lhs, context); - } - } - _ => {} - } - } - - /// Recursively bind variables from an expression (for assignments) - fn bind_variables_from_expr(expr: &ExprRef, context: &mut ScopeContext) { - use crate::ast::Expr as E; - match expr.as_ref() { - E::Var { span, .. } => { - context.bind_variable(span.text()); - } - E::Array { items, .. } => { - for item in items { - Self::bind_variables_from_expr(item, context); - } - } - E::Object { fields, .. } => { - for (_, key_expr, value_expr) in fields { - Self::bind_variables_from_expr(key_expr, context); - Self::bind_variables_from_expr(value_expr, context); - } - } - _ => {} + /// Bind variables from a binding plan into the context + fn bind_vars_from_plan_to_context(binding_plan: &BindingPlan, context: &mut ScopeContext) { + let bound_vars = binding_plan.bound_vars(); + for var in bound_vars { + context.bind_variable(&var); } } @@ -916,18 +837,23 @@ impl LoopHoister { expr: &ExprRef, context: &ScopeContext, ) -> Result<()> { - let mut loops = Vec::new(); - self.hoist_loops_from_expr_with_context(module_idx, expr, &mut loops, context)?; + self.populate_output_expr_with_context(module_idx, expr, context) + } + + fn populate_output_expr_with_context( + &mut self, + module_idx: u32, + expr: &ExprRef, + context: &ScopeContext, + ) -> Result<()> { + let mut loops = Vec::new(); + let mut expr_context = context.clone(); + self.analyze_expr(module_idx, expr, &mut expr_context, &mut loops)?; - // Always store expression loops, even if empty - // This ensures the interpreter can always find an entry let expr_idx = expr.as_ref().eidx(); self.lookup.ensure_expr_capacity(module_idx, expr_idx); self.lookup.set_expr_loops(module_idx, expr_idx, loops); - // Traverse child expressions to populate any nested contexts (e.g., comprehensions) - self.process_expr_for_contexts(module_idx, expr, context)?; - Ok(()) } } diff --git a/src/engine.rs b/src/engine.rs index 81df4c3..3f90f04 100644 --- a/src/engine.rs +++ b/src/engine.rs @@ -852,27 +852,24 @@ impl Engine { let module_idx = self.modules.len() as u32; use crate::compiler::hoist::LoopHoister; + let query_schedule_rc = Rc::new(query_schedule.clone()); - let mut hoister = LoopHoister::new_with_schedule(query_schedule_rc); + + // Run loop hoisting for query snippet + let mut hoister = LoopHoister::new_with_schedule(query_schedule_rc.clone()); hoister.populate_query_snippet( module_idx, &query_node, query_module.num_statements, query_module.num_expressions, )?; - let query_loops = hoister.finalize(); + let query_lookup = hoister.finalize(); #[cfg(debug_assertions)] { - debug_assert_eq!( - query_loops.module_len(), - module_idx as usize + 1, - "query hoisting table missing expected module slot {}", - module_idx - ); for stmt in &query_node.stmts { debug_assert!( - query_loops + query_lookup .get_statement_loops(module_idx, stmt.sidx) .is_some(), "missing hoisted loop entry for query statement index {}", @@ -891,7 +888,7 @@ impl Engine { "loop hoisting table should not retain extra modules before merge" ); } - existing_table.merge_query_loops(query_loops, self.modules.len()); + existing_table.merge_query_loops(query_lookup, self.modules.len()); #[cfg(debug_assertions)] { for stmt in &query_node.stmts { @@ -952,8 +949,11 @@ impl Engine { // Populate loop hoisting table for efficient evaluation // Reserve capacity for 1 extra module (for query modules) use crate::compiler::hoist::LoopHoister; + + // Run loop hoisting pass first let hoister = LoopHoister::new_with_schedule(schedule.clone()); let loop_lookup = hoister.populate_with_extra_capacity(&self.modules, 0)?; + self.interpreter.set_loop_hoisting_table(loop_lookup); // Set schedule after hoisting completes diff --git a/src/interpreter.rs b/src/interpreter.rs index 2c70d02..5a8f0a7 100644 --- a/src/interpreter.rs +++ b/src/interpreter.rs @@ -6,7 +6,10 @@ use crate::builtins::{self, BuiltinFcn}; use crate::compiled_policy::CompiledPolicyData; #[cfg(feature = "azure_policy")] use crate::compiled_policy::TargetInfo; -use crate::compiler::hoist::HoistedLoop; +use crate::compiler::destructuring_planner::{ + AssignmentPlan, BindingPlan, DestructuringPlan, WildcardSide, +}; +use crate::compiler::hoist::{HoistedLoop, LoopType}; use crate::lexer::*; use crate::lookup::Lookup; use crate::parser::Parser; @@ -16,6 +19,9 @@ use crate::value::*; use crate::*; use crate::{Expression, Extension, Location, QueryResult, QueryResults}; +#[cfg(feature = "coverage")] +use crate::query::traversal::traverse; + use alloc::collections::btree_map::Entry as BTreeMapEntry; use alloc::collections::{BTreeMap, BTreeSet}; use anyhow::{anyhow, bail, Result}; @@ -40,7 +46,7 @@ type State = ( BTreeSet>, Value, BTreeMap, - BTreeMap, (Value, Ref)>, + RuleValues, ); #[derive(Debug, Clone)] @@ -68,7 +74,6 @@ pub struct Interpreter { prints: Vec, extensions: Map>)>, - module: Option>, current_module_path: String, current_module_index: u32, @@ -372,10 +377,6 @@ impl Interpreter { self.loop_var_values.clear(module_idx, expr_idx); } - fn has_loop_var_value(&self, expr: &ExprRef) -> bool { - self.get_loop_var_value(expr).is_some() - } - #[inline] fn loop_assignment_expr(loop_info: &HoistedLoop) -> &ExprRef { loop_info.loop_expr.as_ref().unwrap_or(&loop_info.value) @@ -396,6 +397,36 @@ impl Interpreter { loop_info.value.span().clone() } + fn get_walk_binding_plan(&self, loop_info: &HoistedLoop) -> Result> { + if loop_info.loop_type != LoopType::Walk { + return Ok(None); + } + + if let Expr::Call { params, .. } = Self::loop_assignment_expr(loop_info).as_ref() { + if let Some(last_param) = params.last() { + let module_idx = self.current_module_index; + let expr_idx = last_param.as_ref().eidx(); + return match self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() + { + Some(BindingPlan::Parameter { + destructuring_plan, .. + }) => Ok(Some(destructuring_plan)), + Some(other_plan) => bail!( + "internal error: expected Parameter for walk output parameter, got {:?}", + other_plan + ), + None => bail!("internal error: missing binding plan for walk output parameter"), + }; + } + } + + bail!("internal error: walk loop missing output parameter expression") + } + fn ensure_loop_var_values_capacity(&mut self) { for (module_idx, module) in self.compiled_policy.modules.iter().enumerate() { self.loop_var_values @@ -448,18 +479,6 @@ impl Interpreter { Ok(Value::Undefined) } - // TODO: optimize this - fn variables_assignment(&mut self, name: &SourceStr, value: &Value) -> Result<()> { - if let Some(variable) = self.current_scope_mut()?.get_mut(name) { - *variable = value.clone(); - Ok(()) - } else if name.text() == "_" { - Ok(()) - } else { - bail!("variable {} is undefined", name) - } - } - fn eval_chained_ref_dot_or_brack(&mut self, mut expr: &ExprRef) -> Result { // Collect a chaing of '.field' or '["field"]' let mut path = vec![]; @@ -592,161 +611,6 @@ impl Interpreter { } } - fn eval_assign_expr(&mut self, op: &AssignOp, lhs: &ExprRef, rhs: &ExprRef) -> Result { - let (name, value) = match op { - AssignOp::Eq => { - match (lhs.as_ref(), rhs.as_ref()) { - (_, Expr::Var { span: var, .. }) - if var.source_str().text() != "input" - && self.lookup_var(var, &[], true)? == Value::Undefined => - { - (var.source_str(), self.eval_expr(lhs)?) - } - (Expr::Var { span: var, .. }, _) - if var.source_str().text() != "input" - && self.lookup_var(var, &[], true)? == Value::Undefined => - { - (var.source_str(), self.eval_expr(rhs)?) - } - ( - Expr::Array { - items: lhs_items, .. - }, - Expr::Array { - items: rhs_items, - span: rhs_span, - .. - }, - ) => { - if lhs_items.len() != rhs_items.len() { - bail!(rhs_span - .error("mismatch in number of array elements in lhs and rhs")); - } - for (lhs, rhs) in core::iter::zip(lhs_items.iter(), rhs_items.iter()) { - if self.eval_assign_expr(&AssignOp::Eq, lhs, rhs)? != Value::Bool(true) - { - return Ok(Value::Bool(false)); - } - } - return Ok(Value::Bool(true)); - } - ( - Expr::Object { - fields: lhs_fields, .. - }, - Expr::Object { - fields: rhs_fields, - span: rhs_span, - .. - }, - ) => { - if lhs_fields.len() != rhs_fields.len() { - bail!(rhs_span.error("mismatch in number of object keysin lhs and rhs")); - } - - for ((_, lhs_key, lhs_value), (_, rhs_key, rhs_value)) in - core::iter::zip(lhs_fields.iter(), rhs_fields.iter()) - { - if self.eval_bool_expr(&BoolOp::Eq, lhs_key, rhs_key)? - != Value::Bool(true) - { - return Ok(Value::Bool(false)); - } - - if self.eval_assign_expr(&AssignOp::Eq, lhs_value, rhs_value)? - != Value::Bool(true) - { - return Ok(Value::Bool(false)); - } - } - return Ok(Value::Bool(true)); - } - (Expr::Array { .. }, _) => { - let value = self.eval_expr(rhs)?; - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - return self - .make_bindings(false, &mut type_match, &mut cache, lhs, &value, false) - .map(Value::Bool); - } - (_, Expr::Array { .. }) => { - let value = self.eval_expr(lhs)?; - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - return self - .make_bindings(false, &mut type_match, &mut cache, rhs, &value, false) - .map(Value::Bool); - } - (Expr::Object { .. }, _) => { - let value = self.eval_expr(rhs)?; - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - return self - .make_bindings(false, &mut type_match, &mut cache, lhs, &value, false) - .map(Value::Bool); - } - (_, Expr::Object { .. }) => { - let value = self.eval_expr(lhs)?; - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - return self - .make_bindings(false, &mut type_match, &mut cache, rhs, &value, false) - .map(Value::Bool); - } - // Treat the assignment as comparison if neither lhs nor rhs is a variable - _ => { - let r = self.eval_bool_expr(&BoolOp::Eq, lhs, rhs)?; - if r == Value::Bool(false) { - return Ok(Value::Undefined); - } - return Ok(r); - } - } - } - AssignOp::ColEq => { - let rhs_value = self.eval_expr(rhs)?; - if rhs_value == Value::Undefined { - return Ok(rhs_value); - } - - let name = if let Expr::Var { span: s, .. } = lhs.as_ref() { - s.source_str() - } else { - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - return self - .make_bindings(false, &mut type_match, &mut cache, lhs, &rhs_value, false) - .map(Value::Bool); - }; - - // TODO: Check this - // Allow variable overwritten inside a loop - let lhs_val = self.lookup_local_var(&name); - if !matches!(lhs_val, None | Some(Value::Undefined)) - && !self.has_loop_var_value(rhs) - { - bail!(rhs - .span() - .error(&format!("redefinition for variable {name}"))); - } - - (name, rhs_value) - } - }; - - // Omit recording undefined values. - if value == Value::Undefined { - return Ok(value); //Ok(Value::Bool(false)); - } - - self.add_variable_or(&name)?; - - // TODO: optimize this - self.variables_assignment(&name, &value)?; - - Ok(Value::Bool(true)) - } - fn eval_every( &mut self, _span: &Span, @@ -809,268 +673,338 @@ impl Interpreter { Ok(r) } - fn lookup_or_eval_expr( + /// Execute a destructuring plan against a value, binding variables as needed + pub fn execute_destructuring_plan( &mut self, - cache: &mut BTreeMap, - expr: &ExprRef, + plan: &DestructuringPlan, + value: &Value, ) -> Result { - match cache.get(expr) { - Some(v) => Ok(v.clone()), - _ => { - let v = self.eval_expr(expr)?; - cache.insert(expr.clone(), v.clone()); - Ok(v) - } - } - } - - fn make_bindings_impl( - &mut self, - is_last: bool, - type_match: &mut BTreeSet, - cache: &mut BTreeMap, - expr: &ExprRef, - value: &Value, - check_existing_value: bool, - ) -> Result { - // Propagate undefined. if value == &Value::Undefined { - return Ok(false); + return Ok(Value::Undefined); } - let span = expr.span(); - let raise_error = is_last && type_match.get(expr).is_none(); - match (expr.as_ref(), value) { - (Expr::Var { span: ident, .. }, _) if ident.text() == "_" => Ok(true), - (Expr::Var { span: ident, .. }, _) - if check_existing_value - && self.lookup_local_var(&ident.source_str()) == Some(value.clone()) => - { - Ok(false) + fn compare(v1: &Value, v2: &Value) -> Result { + if v1 != v2 || v1 == &Value::Undefined { + Ok(Value::Undefined) + } else { + Ok(Value::from(true)) + } + } + + match plan { + DestructuringPlan::Var(var_name) => { + // Bind the variable to the value + self.add_variable(&var_name.source_str(), value.clone())?; + Ok(Value::Bool(true)) } - (Expr::Var { span: ident, .. }, _) => { - self.add_variable(&ident.source_str(), value.clone())?; - Ok(true) + DestructuringPlan::Ignore => Ok(Value::Bool(true)), + + DestructuringPlan::EqualityExpr(expected_expr) => { + let expected = self.eval_expr(expected_expr)?; + compare(value, &expected) } - // Destructure arrays - (Expr::Array { items, .. }, Value::Array(a)) => { - if items.len() != a.len() { - if raise_error { - return Err(span.error( - format!( - "array length mismatch. Expected {} got {}.", - items.len(), - a.len() - ) - .as_str(), - )); + DestructuringPlan::EqualityValue(expected) => Ok(Value::from(value == expected)), + + DestructuringPlan::Array { element_plans } => { + // Value must be an array with matching length + if let Value::Array(arr) = value { + if arr.len() != element_plans.len() { + return Ok(Value::Undefined); } - return Ok(false); - } - type_match.insert(expr.clone()); - let mut r = true; - for (idx, item) in items.iter().enumerate() { - r = self.make_bindings( - is_last, - type_match, - cache, - item, - &a[idx], - check_existing_value, - )? && r; - } - - Ok(r) - } - // Destructure objects - (Expr::Object { fields, .. }, Value::Object(_)) => { - let mut r = true; - for (_, key_expr, value_expr) in fields.iter() { - // Rego does not support bindings in keys. - // Therefore, just eval key_expr. - let key = self.lookup_or_eval_expr(cache, key_expr)?; - let field_value = &value[&key]; - - if field_value == &Value::Undefined { - if raise_error { - return Err(span.error("Expected value, got undefined.")); + // Recursively execute each element plan + for (i, element_plan) in element_plans.iter().enumerate() { + if self.execute_destructuring_plan(element_plan, &arr[i])? + != Value::from(true) + { + return Ok(Value::Undefined); } - return Ok(false); } - - // Match patterns in value_expr - r = r - && self.make_bindings( - is_last, - type_match, - cache, - value_expr, - field_value, - check_existing_value, - )?; + Ok(Value::from(true)) + } else { + Ok(Value::Undefined) // Not an array } - type_match.insert(expr.clone()); - - Ok(r) } - // TODO: This suppresses errors in case of type mismatches. - // OPA raises the error sometimes in static scenarios, but doesn't - // raise in scenarios due to data/input - (Expr::Array { .. }, _) | (Expr::Object { .. }, _) => Ok(false), - _ => { - let expr_value = self.lookup_or_eval_expr(cache, expr)?; - if expr_value == Value::Undefined { - return Ok(false); - } - - if raise_error { - let expr_t = builtins::types::get_type(&expr_value); - let value_t = builtins::types::get_type(value); - - if expr_t != value_t { - return Err(span.error( - format!("Cannot bind pattern of type `{expr_t}` with value of type `{value_t}`. Value is {value}.").as_str())); + DestructuringPlan::Object { + field_plans, + dynamic_fields, + } => { + // Value must be an object with matching fields + if let Value::Object(obj) = value { + // Check that all required fields are present and match + for (key, field_plan) in field_plans { + if let Some(field_value) = obj.get(key) { + if self.execute_destructuring_plan(field_plan, field_value)? + != Value::from(true) + { + return Ok(Value::Undefined); + } + } else { + return Ok(Value::Undefined); // Required field missing + } } - } - type_match.insert(expr.clone()); - Ok(&expr_value == value) + if !dynamic_fields.is_empty() { + for (key_expr, field_plan) in dynamic_fields { + let key_value = self.eval_expr(key_expr)?; + if key_value == Value::Undefined { + return Ok(Value::Undefined); + } + + if let Some(field_value) = obj.get(&key_value) { + if self.execute_destructuring_plan(field_plan, field_value)? + != Value::from(true) + { + return Ok(Value::Undefined); + } + } else { + return Ok(Value::Undefined); + } + } + } + Ok(Value::from(true)) + } else { + Ok(Value::Undefined) // Not an object + } } } } - fn make_bindings( - &mut self, - is_last: bool, - type_match: &mut BTreeSet, - cache: &mut BTreeMap, - expr: &ExprRef, - value: &Value, - check_existing_value: bool, - ) -> Result { - let prev = self.no_rules_lookup; - self.no_rules_lookup = true; - let r = self.make_bindings_impl( - is_last, - type_match, - cache, - expr, - value, - check_existing_value, - ); - self.no_rules_lookup = prev; - r - } + fn execute_assignment_plan(&mut self, plan: &AssignmentPlan) -> Result { + match plan { + AssignmentPlan::ColonEquals { + lhs_expr: _, + rhs_expr, + lhs_plan, + } => { + // For :=, evaluate RHS and bind to LHS pattern + let rhs_value = self.eval_expr(rhs_expr)?; + self.execute_destructuring_plan(lhs_plan, &rhs_value) + } - fn make_key_value_bindings( - &mut self, - is_last: bool, - type_match: &mut BTreeSet, - cache: &mut BTreeMap, - exprs: (&Option, &ExprRef), - values: (&Value, &Value), - ) -> Result { - let (key_expr, value_expr) = exprs; - let (key, value) = values; - if let Some(key_expr) = key_expr { - if !self.make_bindings(is_last, type_match, cache, key_expr, key, false)? { - return Ok(false); + AssignmentPlan::EqualsBindLeft { + lhs_expr: _, + rhs_expr, + lhs_plan, + } => { + // For = with LHS binding, evaluate RHS and bind to LHS pattern + let rhs_value = self.eval_expr(rhs_expr)?; + self.execute_destructuring_plan(lhs_plan, &rhs_value) + } + + AssignmentPlan::EqualsBindRight { + lhs_expr, + rhs_expr: _, + rhs_plan, + } => { + // For = with RHS binding, evaluate LHS and bind to RHS pattern + let lhs_value = self.eval_expr(lhs_expr)?; + self.execute_destructuring_plan(rhs_plan, &lhs_value) + } + + AssignmentPlan::EqualsBothSides { + lhs_expr: _, + rhs_expr: _, + element_pairs, + } => { + // For = with both sides having patterns, execute each flattened pair + for (value_expr, pattern_plan) in element_pairs { + let value = self.eval_expr(value_expr)?; + if self.execute_destructuring_plan(pattern_plan, &value)? != Value::from(true) { + return Ok(Value::Undefined); + } + } + Ok(Value::from(true)) + } + + AssignmentPlan::WildcardMatch { + lhs_expr, + rhs_expr, + wildcard_side, + } => match wildcard_side { + WildcardSide::Both => Ok(Value::Bool(true)), + WildcardSide::Lhs => { + let rhs_value = self.eval_expr(rhs_expr)?; + if rhs_value == Value::Undefined { + Ok(Value::Undefined) + } else { + Ok(Value::Bool(true)) + } + } + WildcardSide::Rhs => { + let lhs_value = self.eval_expr(lhs_expr)?; + if lhs_value == Value::Undefined { + Ok(Value::Undefined) + } else { + Ok(Value::Bool(true)) + } + } + }, + + AssignmentPlan::EqualityCheck { lhs_expr, rhs_expr } => { + let lhs_value = self.eval_expr(lhs_expr)?; + let rhs_value = self.eval_expr(rhs_expr)?; + + if lhs_value == Value::Undefined || rhs_value == Value::Undefined { + return Ok(Value::Undefined); + } + + if lhs_value == rhs_value { + Ok(Value::Bool(true)) + } else { + Ok(Value::Undefined) + } } } - self.make_bindings(is_last, type_match, cache, value_expr, value, false) } fn eval_some_in( &mut self, _span: &Span, - key_expr: &Option, - value_expr: &ExprRef, + _key_expr: &Option, + _value_expr: &ExprRef, collection: &ExprRef, stmts: &[&LiteralStmt], ) -> Result { let scope_saved = self.current_scope()?.clone(); - let mut type_match = BTreeSet::new(); - let mut cache = BTreeMap::new(); let mut count = 0; + + // Fetch the binding plan for this some..in expression + let module_idx = self.current_module_index; + let expr_idx = collection.as_ref().eidx(); + + let Some(BindingPlan::SomeIn { + key_plan, + value_plan, + .. + }) = self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() + else { + bail!("internal error: missing binding plan for some..in expression"); + }; + match self.eval_expr(collection)? { Value::Array(a) => { for (idx, value) in a.iter().enumerate() { - if !self.make_key_value_bindings( - idx == a.len() - 1, - &mut type_match, - &mut cache, - (key_expr, value_expr), - (&Value::from(idx), value), - )? { + *self.current_scope_mut()? = scope_saved.clone(); + + let mut success = true; + // Execute key binding if present + if let Some(key_plan) = &key_plan { + success = self.execute_destructuring_plan(key_plan, &Value::from(idx))? + == Value::from(true); + } + + // Execute value binding + success = success + && self.execute_destructuring_plan(&value_plan, value)? + == Value::from(true); + + if !success { + *self.current_scope_mut()? = scope_saved.clone(); continue; } + let mut should_break = false; if self.eval_stmts(stmts)? { count += 1; if let Some(ctx) = self.contexts.last() { if ctx.early_return { - break; + should_break = true; } } } *self.current_scope_mut()? = scope_saved.clone(); + + if should_break { + break; + } } } Value::Set(s) => { - for (idx, value) in s.iter().enumerate() { - if !self.make_key_value_bindings( - idx == s.len() - 1, - &mut type_match, - &mut cache, - (key_expr, value_expr), - (value, value), - )? { + for value in s.iter() { + *self.current_scope_mut()? = scope_saved.clone(); + + let mut success = true; + // Execute key binding if present + if let Some(key_plan) = &key_plan { + success = + self.execute_destructuring_plan(key_plan, value)? == Value::from(true); + } + + // Execute value binding + success = success + && self.execute_destructuring_plan(&value_plan, value)? + == Value::from(true); + + if !success { + *self.current_scope_mut()? = scope_saved.clone(); continue; } + let mut should_break = false; if self.eval_stmts(stmts)? { count += 1; if let Some(ctx) = self.contexts.last() { if ctx.early_return { - break; + should_break = true; } } } *self.current_scope_mut()? = scope_saved.clone(); + + if should_break { + break; + } } } Value::Object(o) => { - for (idx, (key, value)) in o.iter().enumerate() { - if !self.make_key_value_bindings( - idx == o.len() - 1, - &mut type_match, - &mut cache, - (key_expr, value_expr), - (key, value), - )? { + for (key, value) in o.iter() { + *self.current_scope_mut()? = scope_saved.clone(); + + let mut success = true; + // Execute key binding if present + if let Some(key_plan) = &key_plan { + success = + self.execute_destructuring_plan(key_plan, key)? == Value::from(true); + } + + // Execute value binding + success = success + && self.execute_destructuring_plan(&value_plan, value)? + == Value::from(true); + + if !success { + *self.current_scope_mut()? = scope_saved.clone(); continue; } + let mut should_break = false; if self.eval_stmts(stmts)? { count += 1; if let Some(ctx) = self.contexts.last() { if ctx.early_return { - break; + should_break = true; } } } *self.current_scope_mut()? = scope_saved.clone(); + + if should_break { + break; + } } } Value::Undefined => (), v => { - let span = collection.span(); - bail!(span.error( + bail!(collection.span().error( format!("`some .. in collection` expects array/set/object. Got `{v}`").as_str() )) } @@ -1423,6 +1357,47 @@ impl Interpreter { // If the loop's index variable h { + for item in items.iter() { + self.set_loop_var_value(loop_target_expr, item.clone()); + + if self.execute_destructuring_plan(&walk_plan, item)? + == Value::from(true) + { + walk_result = + self.eval_stmts_in_loop(stmts, &loops[1..])? || walk_result; + } + + Self::clear_scope(self.current_scope_mut()?); + if let Some(ctx) = self.contexts.last_mut() { + ctx.result.clone_from(&query_result); + if ctx.early_return { + break; + } + } + } + } + Value::Undefined => (), + other => { + let span = Self::loop_span(loop_info); + bail!(span + .error(format!("walk expected array result, got `{other}`").as_str())); + } + } + + self.scopes.pop(); + self.remove_loop_var_value(loop_target_expr); + return Ok(walk_result); + } + let index_expr = Self::loop_index_expr(loop_info); if let Some(Expr::Var { span: index_var, .. @@ -1439,6 +1414,29 @@ impl Interpreter { } } + let index_plan = if let Some(index) = index_expr { + let module_idx = self.current_module_index; + let expr_idx = index.as_ref().eidx(); + match self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() + { + Some(BindingPlan::LoopIndex { + destructuring_plan, .. + }) => destructuring_plan, + Some(other_plan) => { + bail!("internal error: expected LoopIndex for loop index expression, got {:?}", other_plan); + } + None => { + bail!("internal error: no binding plan found for loop index expression"); + } + } + } else { + bail!("internal error: no binding plan found for loop index expression"); + }; + // Create a new scope. self.scopes.push(Scope::default()); @@ -1449,21 +1447,9 @@ impl Interpreter { for (idx, v) in items.iter().enumerate() { self.set_loop_var_value(loop_target_expr, v.clone()); - let exec = if let Some(index) = index_expr { - let mut type_match = BTreeSet::new(); - let mut cache = BTreeMap::new(); - self.make_bindings( - false, - &mut type_match, - &mut cache, - index, - &Value::from(idx), - true, - )? - } else { - true - }; - if exec { + if self.execute_destructuring_plan(&index_plan, &Value::from(idx))? + == Value::from(true) + { result = self.eval_stmts_in_loop(stmts, &loops[1..])? || result; } @@ -1483,14 +1469,7 @@ impl Interpreter { self.set_loop_var_value(loop_target_expr, v.clone()); // For sets, index is also the value. - let exec = if let Some(index) = index_expr { - let mut type_match = BTreeSet::new(); - let mut cache = BTreeMap::new(); - self.make_bindings(false, &mut type_match, &mut cache, index, v, true)? - } else { - true - }; - if exec { + if self.execute_destructuring_plan(&index_plan, v)? == Value::from(true) { result = self.eval_stmts_in_loop(stmts, &loops[1..])? || result; } @@ -1508,14 +1487,7 @@ impl Interpreter { for (k, v) in obj.iter() { self.set_loop_var_value(loop_target_expr, v.clone()); // For objects, index is key. - let exec = if let Some(index) = index_expr { - let mut type_match = BTreeSet::new(); - let mut cache = BTreeMap::new(); - self.make_bindings(false, &mut type_match, &mut cache, index, k, true)? - } else { - true - }; - if exec { + if self.execute_destructuring_plan(&index_plan, k)? == Value::from(true) { result = self.eval_stmts_in_loop(stmts, &loops[1..])? || result; } @@ -2507,20 +2479,38 @@ impl Interpreter { let args_scope = Scope::new(); self.scopes.push(args_scope); - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); + // Determine the module index for the callee so we can fetch binding plans + let callee_module_idx = fcn_module + .as_ref() + .map(|module| self.find_module_index(module)) + .unwrap_or(self.current_module_index); for (idx, a) in args.iter().enumerate() { - let b = self.make_bindings( - false, - &mut type_match, - &mut cache, - a, - ¶m_values[idx], - false, - ); + // Fetch the binding plan for this function parameter + let module_idx = callee_module_idx; + let expr_idx = a.as_ref().eidx(); - if b.ok() != Some(true) { + let binding_success = if let Some(BindingPlan::Parameter { + destructuring_plan, + .. + }) = self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() + { + // Execute the destructuring plan with the parameter value + self.execute_destructuring_plan(&destructuring_plan, ¶m_values[idx])? + == Value::from(true) + } else { + // Raise error if binding plan is not found + return Err(span.error(&format!( + "binding plan not found for parameter {}", + a.span().text() + ))); + }; + + if !binding_success { self.scopes = scopes; continue 'outer; } @@ -2640,32 +2630,33 @@ impl Interpreter { allow_return_arg: bool, ) -> Result { // TODO: global var check; interop with `some var` - if let Some(ea) = extra_arg { - match ea.as_ref() { - Expr::Var { span: var, .. } - if allow_return_arg && self.lookup_local_var(&var.source_str()).is_none() => + if extra_arg.is_some() { + let value = self.eval_call_impl(span, expr, fcn, ¶ms[..params.len() - 1])?; + if allow_return_arg { + let last_param = ¶ms[params.len() - 1]; + let module_idx = self.current_module_index; + let expr_idx = last_param.as_ref().eidx(); + if let Some(BindingPlan::Parameter { + destructuring_plan, .. + }) = self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() { - let value = - self.eval_call_impl(span, expr, fcn, ¶ms[..params.len() - 1])?; - if var.text() != "_" { - self.add_variable(&var.source_str(), value)?; - } - Ok(Value::Bool(true)) - } - _ if allow_return_arg => { - let ret_value = - self.eval_call_impl(span, expr, fcn, ¶ms[..params.len() - 1])?; - let mut cache = BTreeMap::new(); - let mut type_match = BTreeSet::new(); - self.make_bindings(false, &mut type_match, &mut cache, &ea, &ret_value, false) - .map(Value::Bool) - } - _ => { - let expected = self.eval_expr(¶ms[params.len() - 1])?; - let ret_value = - self.eval_call_impl(span, expr, fcn, ¶ms[..params.len() - 1])?; - Ok(Value::Bool(ret_value == expected)) + // Execute the destructuring plan with the return value + let result = self.execute_destructuring_plan(&destructuring_plan, &value)?; + Ok(result) + } else { + // Raise error if binding plan is not found + Err(span.error(&format!( + "binding plan not found for parameter {}", + last_param.span().text() + ))) } + } else { + let expected = self.eval_expr(¶ms[params.len() - 1])?; + Ok(Value::Bool(value == expected)) } } else { self.eval_call_impl(span, expr, fcn, params) @@ -2934,7 +2925,28 @@ impl Interpreter { // Expressions with operators Expr::ArithExpr { op, lhs, rhs, .. } => self.eval_arith_expr(expr.span(), op, lhs, rhs), - Expr::AssignExpr { op, lhs, rhs, .. } => self.eval_assign_expr(op, lhs, rhs), + Expr::AssignExpr { .. } => { + let module_idx = self.current_module_index; + let expr_idx = expr.as_ref().eidx(); + let expr_text = expr.span().text().to_string(); + let binding_plan = self + .compiled_policy + .loop_hoisting_table + .get_expr_binding_plan(module_idx, expr_idx) + .cloned() + .ok_or_else(|| { + expr.span().error( + format!( + "binding plan missing for assignment expression (module_idx={module_idx}, expr_idx={expr_idx}, expr='{expr_text}')" + ) + .as_str(), + ) + })?; + let BindingPlan::Assignment { plan } = binding_plan else { + bail!(expr.span().error("internal error: not an assignment plan")); + }; + self.execute_assignment_plan(&plan) + } Expr::BinExpr { op, lhs, rhs, .. } => self.eval_bin_expr(op, lhs, rhs), Expr::BoolExpr { op, lhs, rhs, .. } => self.eval_bool_expr(op, lhs, rhs), Expr::Membership { @@ -2990,7 +3002,6 @@ impl Interpreter { fn make_rule_context(&self, head: &RuleHead) -> Result<(Context, Vec)> { let mut path = Parser::get_path_ref_components(&self.module.clone().unwrap().package.refr)?; - match head { RuleHead::Compr { refr, assign, .. } => { let output_expr = assign.as_ref().map(|assign| assign.value.clone()); diff --git a/src/lib.rs b/src/lib.rs index 6e8bfcd..269d1e1 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -35,6 +35,7 @@ mod lookup; mod number; mod parser; mod policy_info; +mod query; #[cfg(feature = "azure_policy")] pub mod registry; mod scheduler; diff --git a/src/number.rs b/src/number.rs index e725c40..3c84b1d 100644 --- a/src/number.rs +++ b/src/number.rs @@ -489,4 +489,15 @@ mod test { let n = Number::from(123456f64); assert_eq!(format!("{}", n.format_decimal()), "123456"); } + + #[test] + fn division_matches_high_precision_decimal() { + let one = Number::from(1u64); + let three = Number::from(3u64); + let div = one.divide(&three).unwrap(); + let from_str: Number = "0.3333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333333" + .parse() + .unwrap(); + assert_eq!(div, from_str); + } } diff --git a/src/query/mod.rs b/src/query/mod.rs new file mode 100644 index 0000000..f8e56a4 --- /dev/null +++ b/src/query/mod.rs @@ -0,0 +1 @@ +pub mod traversal; diff --git a/src/query/traversal.rs b/src/query/traversal.rs new file mode 100644 index 0000000..56d42a6 --- /dev/null +++ b/src/query/traversal.rs @@ -0,0 +1,224 @@ +// Copyright (c) Microsoft Corporation. +// Licensed under the MIT License. + +use alloc::collections::{BTreeMap, BTreeSet}; +use alloc::string::{String, ToString}; + +use anyhow::Result; + +use crate::ast::Expr::{self, *}; +use crate::ast::{AssignOp, ExprRef}; +use crate::lexer::{SourceStr, Span}; +use crate::value::Value; + +#[derive(Clone, Default, Debug)] +pub struct Scope { + pub locals: BTreeMap, + pub unscoped: BTreeSet, + pub inputs: BTreeSet, + pub uses_input: bool, +} + +pub fn traverse(expr: &ExprRef, f: &mut dyn FnMut(&ExprRef) -> Result) -> Result<()> { + if !f(expr)? { + return Ok(()); + } + + match expr.as_ref() { + Expr::String { .. } + | RawString { .. } + | Number { .. } + | Bool { .. } + | Null { .. } + | Var { .. } => (), + + Array { items, .. } | Set { items, .. } => { + for item in items { + traverse(item, f)?; + } + } + Object { fields, .. } => { + for (_, key, value) in fields { + traverse(key, f)?; + traverse(value, f)?; + } + } + + ArrayCompr { .. } | SetCompr { .. } | ObjectCompr { .. } => (), + + Call { params, .. } => { + for param in params { + traverse(param, f)?; + } + } + + UnaryExpr { expr, .. } => traverse(expr, f)?, + + RefDot { refr, .. } => traverse(refr, f)?, + + RefBrack { refr, index, .. } => { + traverse(refr, f)?; + traverse(index, f)?; + } + + BinExpr { lhs, rhs, .. } + | BoolExpr { lhs, rhs, .. } + | ArithExpr { lhs, rhs, .. } + | AssignExpr { lhs, rhs, .. } => { + traverse(lhs, f)?; + traverse(rhs, f)?; + } + + #[cfg(feature = "rego-extensions")] + OrExpr { lhs, rhs, .. } => { + traverse(lhs, f)?; + traverse(rhs, f)?; + } + + Membership { + key, + value, + collection, + .. + } => { + if let Some(key) = key.as_ref() { + traverse(key, f)?; + } + traverse(value, f)?; + traverse(collection, f)?; + } + } + + Ok(()) +} + +pub fn var_exists(var: &Span, parent_scopes: &[Scope]) -> bool { + let name = var.source_str(); + + for scope in parent_scopes.iter().rev() { + if scope.unscoped.contains(&name) { + return true; + } + + if let Some(span) = scope.locals.get(&name) { + if span.line <= var.line { + return true; + } + } + } + + false +} + +pub fn gather_assigned_vars( + expr: &ExprRef, + can_shadow: bool, + parent_scopes: &[Scope], + scope: &mut Scope, +) -> Result<()> { + traverse(expr, &mut |node| match node.as_ref() { + Var { span, .. } if matches!(span.text(), "_" | "input" | "data") => { + if span.text() == "input" { + scope.uses_input = true; + } + Ok(false) + } + Var { span, .. } if can_shadow => { + scope.locals.insert(span.source_str(), span.clone()); + Ok(false) + } + Var { span, .. } if var_exists(span, parent_scopes) => { + scope.inputs.insert(span.source_str()); + Ok(false) + } + Var { span, .. } => { + scope.unscoped.insert(span.source_str()); + Ok(false) + } + Array { .. } | Object { .. } => Ok(true), + _ => Ok(false), + }) +} + +pub fn gather_input_vars(expr: &ExprRef, parent_scopes: &[Scope], scope: &mut Scope) -> Result<()> { + traverse(expr, &mut |node| match node.as_ref() { + Var { span, .. } => { + let name = span.source_str(); + if name.text() == "input" { + scope.uses_input = true; + } else if !scope.unscoped.contains(&name) && var_exists(span, parent_scopes) { + scope.inputs.insert(name); + } + Ok(false) + } + _ => Ok(true), + }) +} + +pub fn gather_loop_vars(expr: &ExprRef, parent_scopes: &[Scope], scope: &mut Scope) -> Result<()> { + traverse(expr, &mut |node| match node.as_ref() { + Var { span, .. } if span.text() == "input" => { + scope.uses_input = true; + Ok(false) + } + RefBrack { index, .. } => { + gather_assigned_vars(index, false, parent_scopes, scope)?; + Ok(true) + } + _ => Ok(true), + }) +} + +pub fn gather_vars( + expr: &ExprRef, + can_shadow: bool, + parent_scopes: &[Scope], + scope: &mut Scope, +) -> Result<()> { + if let AssignExpr { op, lhs, rhs, .. } = expr.as_ref() { + gather_assigned_vars(lhs, *op == AssignOp::ColEq, parent_scopes, scope)?; + gather_assigned_vars(rhs, false, parent_scopes, scope)?; + } else { + gather_assigned_vars(expr, can_shadow, parent_scopes, scope)?; + } + + gather_input_vars(expr, parent_scopes, scope)?; + gather_loop_vars(expr, parent_scopes, scope) +} + +pub fn collect_expr_dependencies(expr: &ExprRef) -> Option> { + let mut deps = BTreeSet::new(); + let mut valid = true; + + if traverse(expr, &mut |node| match node.as_ref() { + Var { value, .. } => { + if let Value::String(name) = value { + let var = name.as_ref(); + if var != "_" { + deps.insert(var.to_string()); + } + } + Ok(false) + } + ArrayCompr { .. } | SetCompr { .. } | ObjectCompr { .. } => { + valid = false; + Ok(false) + } + #[cfg(feature = "rego-extensions")] + OrExpr { .. } => { + valid = false; + Ok(false) + } + _ => Ok(true), + }) + .is_err() + { + return None; + } + + if valid { + Some(deps) + } else { + None + } +} diff --git a/src/scheduler.rs b/src/scheduler.rs index 17cab72..888228c 100644 --- a/src/scheduler.rs +++ b/src/scheduler.rs @@ -1,10 +1,14 @@ // Copyright (c) Microsoft Corporation. // Licensed under the MIT License. -use crate::ast::Expr::{Set, *}; +use crate::ast::Expr::*; use crate::ast::*; use crate::lexer::*; use crate::lookup::*; +pub use crate::query::traversal::Scope; +use crate::query::traversal::{ + gather_assigned_vars, gather_input_vars, gather_loop_vars, gather_vars, traverse, +}; use crate::utils::*; use crate::*; @@ -208,205 +212,12 @@ pub fn schedule( Ok(SortResult::Order(order)) } -#[derive(Clone, Default, Debug)] -pub struct Scope { - pub locals: BTreeMap, - pub unscoped: BTreeSet, - pub inputs: BTreeSet, - pub uses_input: bool, -} - #[derive(Clone, Default, Debug)] pub struct QuerySchedule { pub scope: Scope, pub order: Vec, } -pub fn traverse(expr: &Ref, f: &mut dyn FnMut(&Ref) -> Result) -> Result<()> { - if !f(expr)? { - return Ok(()); - } - match expr.as_ref() { - Expr::String { .. } - | RawString { .. } - | Number { .. } - | Bool { .. } - | Null { .. } - | Var { .. } => (), - - Array { items, .. } | Set { items, .. } => { - for i in items { - traverse(i, f)?; - } - } - Object { fields, .. } => { - for (_, k, v) in fields { - traverse(k, f)?; - traverse(v, f)?; - } - } - - ArrayCompr { .. } | SetCompr { .. } | ObjectCompr { .. } => (), - - Call { params, .. } => { - for p in params { - traverse(p, f)?; - } - } - - UnaryExpr { expr, .. } => traverse(expr, f)?, - - RefDot { refr, .. } => traverse(refr, f)?, - - RefBrack { refr, index, .. } => { - traverse(refr, f)?; - traverse(index, f)?; - } - - BinExpr { lhs, rhs, .. } - | BoolExpr { lhs, rhs, .. } - | ArithExpr { lhs, rhs, .. } - | AssignExpr { lhs, rhs, .. } => { - traverse(lhs, f)?; - traverse(rhs, f)?; - } - - #[cfg(feature = "rego-extensions")] - OrExpr { lhs, rhs, .. } => { - traverse(lhs, f)?; - traverse(rhs, f)?; - } - - Membership { - key, - value, - collection, - .. - } => { - if let Some(key) = key.as_ref() { - traverse(key, f)?; - } - traverse(value, f)?; - traverse(collection, f)?; - } - } - Ok(()) -} - -fn var_exists(var: &Span, parent_scopes: &[Scope]) -> bool { - let name = var.source_str(); - - for pscope in parent_scopes.iter().rev() { - if pscope.unscoped.contains(&name) { - return true; - } - // Check parent scope vars defined using :=. - if let Some(s) = pscope.locals.get(&name) { - // Note: Since a rule cannot span multiple files, it is safe to check only - // the line numbers. - if s.line <= var.line { - // The variable was defined in parent scope prior to current comprehension. - return true; - } - } - } - false -} - -fn gather_assigned_vars( - expr: &Ref, - can_shadow: bool, - parent_scopes: &[Scope], - scope: &mut Scope, -) -> Result<()> { - traverse(expr, &mut |e| match e.as_ref() { - // Ignore _, input, data. - Var { span: v, .. } if matches!(v.text(), "_" | "input" | "data") => { - if v.text() == "input" { - scope.uses_input = true; - } - Ok(false) - } - - // Record local var that can shadow input var. - Var { span: v, .. } if can_shadow => { - scope.locals.insert(v.source_str(), v.clone()); - Ok(false) - } - - // Record input vars. - Var { span: v, .. } if var_exists(v, parent_scopes) => { - scope.inputs.insert(v.source_str()); - Ok(false) - } - - // Record local var. - Var { span: v, .. } => { - scope.unscoped.insert(v.source_str()); - Ok(false) - } - - // TODO: key vs value for object binding - Array { .. } | Object { .. } => Ok(true), - _ => Ok(false), - }) -} - -fn gather_input_vars(expr: &Ref, parent_scopes: &[Scope], scope: &mut Scope) -> Result<()> { - traverse(expr, &mut |e| match e.as_ref() { - Var { span: v, .. } => { - let name = v.source_str(); - if name.text() == "input" { - scope.uses_input = true; - } else if !scope.unscoped.contains(&name) && var_exists(v, parent_scopes) { - scope.inputs.insert(name); - } - Ok(false) - } - _ => Ok(true), - }) -} - -fn gather_loop_vars(expr: &Ref, parent_scopes: &[Scope], scope: &mut Scope) -> Result<()> { - traverse(expr, &mut |e| match e.as_ref() { - Var { span: v, .. } if v.text() == "input" => { - scope.uses_input = true; - Ok(false) - } - RefBrack { index, .. } => { - gather_assigned_vars(index, false, parent_scopes, scope)?; - Ok(true) - } - _ => Ok(true), - }) -} - -// TODO: start opa discussion -// k = "k" -// t = {"k": 5} -// {k:y} = t -// Try inlining value of t -fn gather_vars( - expr: &Ref, - can_shadow: bool, - parent_scopes: &[Scope], - scope: &mut Scope, -) -> Result<()> { - // Process assignment expressions to gather vars that are defined/assigned - // in current scope. - if let AssignExpr { op, lhs, rhs, .. } = expr.as_ref() { - gather_assigned_vars(lhs, *op == AssignOp::ColEq, parent_scopes, scope)?; - gather_assigned_vars(rhs, false, parent_scopes, scope)?; - } else { - gather_assigned_vars(expr, can_shadow, parent_scopes, scope)?; - } - - // Process all expressions to gather loop index vars and inputs. - // TODO: := assignment and use in same statement. - gather_input_vars(expr, parent_scopes, scope)?; - gather_loop_vars(expr, parent_scopes, scope) -} - pub struct Analyzer { packages: BTreeMap, scopes: Vec, @@ -1011,7 +822,7 @@ impl Analyzer { expr: &Ref, scope: &Scope, _first_use: &BTreeMap, - vars: &mut Vec, + vars: &mut Vec<(SourceStr, Span)>, non_vars: &mut Vec>, ) -> Result<()> { traverse(expr, &mut |e| match e.as_ref() { @@ -1021,7 +832,7 @@ impl Analyzer { Ok(false) } Var { span: v, .. } if scope.locals.contains_key(&v.source_str()) => { - vars.push(v.source_str()); + vars.push((v.source_str(), v.clone())); Ok(false) } // TODO: Object key/value @@ -1114,13 +925,17 @@ impl Analyzer { )?; // Add dependency between some-vars and vars used in collection. - for var in &some_vars { + for (var, _) in &some_vars { definitions.push(Definition { var: var.clone(), used_vars: col_used_vars.clone(), }) } + for (var, span) in &some_vars { + first_use.entry(var.clone()).or_insert(span.clone()); + } + let mut used_vars = vec![]; for e in non_vars { let mut definitions = vec![]; @@ -1296,3 +1111,71 @@ impl Analyzer { Ok(()) } } + +/// Compute module globals for each module. +/// +/// For each module, the globals are: +/// 1) The set of rule names defined in the package that the module defines +/// 2) Additionally, the set of aliases imported by the module +pub fn compute_module_globals( + modules: &[Ref], +) -> Result>>> { + let mut result = Lookup::new(); + let mut packages: BTreeMap>> = BTreeMap::new(); + + // First pass: collect all rule names by package + for m in modules { + let path = get_path_string(&m.package.refr, Some("data"))?; + let package_globals: &mut crate::Rc> = packages.entry(path).or_default(); + + for r in &m.policy { + let var = match r.as_ref() { + Rule::Default { refr, .. } + | Rule::Spec { + head: + RuleHead::Compr { refr, .. } + | RuleHead::Set { refr, .. } + | RuleHead::Func { refr, .. }, + .. + } => get_root_var(refr)?, + }; + crate::Rc::make_mut(package_globals).insert(var.text().to_string()); + } + } + + // Second pass: for each module, combine package globals with module-specific imports + for (module_idx, m) in modules.iter().enumerate() { + let path = get_path_string(&m.package.refr, Some("data"))?; + let mut module_globals = packages.get(&path).cloned().unwrap_or_default(); + + // Add import aliases specific to this module + for import in &m.imports { + if let Some(var) = &import.r#as { + crate::Rc::make_mut(&mut module_globals).insert(var.text().to_string()); + } + } + + // Ensure reserved root documents are always treated as globals. + for &reserved in ["input", "data"].iter() { + crate::Rc::make_mut(&mut module_globals).insert(reserved.to_string()); + } + + // Reserved documents are always available in every module. + let reserved_docs = ["input", "data"]; + for doc in reserved_docs { + crate::Rc::make_mut(&mut module_globals).insert(doc.to_string()); + } + + // Seed with reserved document roots that are always globally accessible. + { + let globals = crate::Rc::make_mut(&mut module_globals); + globals.insert("input".to_string()); + globals.insert("data".to_string()); + } + + result.ensure_capacity(module_idx as u32, 0); + result.set(module_idx as u32, 0, module_globals); + } + + Ok(result) +} diff --git a/src/tests/interpreter/mod.rs b/src/tests/interpreter/mod.rs index 984d184..13d3204 100644 --- a/src/tests/interpreter/mod.rs +++ b/src/tests/interpreter/mod.rs @@ -253,9 +253,10 @@ pub fn eval_file( query: &str, enable_tracing: bool, strict: bool, + v0: bool, ) -> Result<(Vec, Vec)> { let mut engine: Engine = Engine::new(); - engine.set_rego_v0(true); + engine.set_rego_v0(v0); engine.set_strict_builtin_errors(strict); engine.set_gather_prints(true); @@ -333,9 +334,10 @@ pub fn eval_file_with_rule_evaluation( query: &str, _enable_tracing: bool, strict: bool, + v0: bool, ) -> Result<(Vec, Vec)> { let mut engine: Engine = Engine::new(); - engine.set_rego_v0(true); + engine.set_rego_v0(v0); engine.set_strict_builtin_errors(strict); engine.set_gather_prints(true); @@ -484,9 +486,19 @@ fn yaml_test_impl(file: &str) -> Result<()> { } } } + #[cfg(not(feature = "graph"))] + { + // Skip tests that depend on graph builtin that need graph feature. + if file.contains("walk.yaml") { + std::println!("skipped {file} without graph feature."); + return Ok(()); + } + } std::println!("running {file}"); + let v0 = !file.contains("bindings.yaml"); + for case in test.cases { std::print!("case {} ", case.note); if case.skip == Some(true) { @@ -516,6 +528,7 @@ fn yaml_test_impl(file: &str) -> Result<()> { case.query.as_str(), enable_tracing, case.strict, + v0, ) } #[cfg(not(feature = "azure_policy"))] @@ -530,6 +543,7 @@ fn yaml_test_impl(file: &str) -> Result<()> { case.query.as_str(), enable_tracing, case.strict, + v0, ) }; diff --git a/tests/interpreter/cases/binding/bindings.yaml b/tests/interpreter/cases/binding/bindings.yaml new file mode 100644 index 0000000..517b4d8 --- /dev/null +++ b/tests/interpreter/cases/binding/bindings.yaml @@ -0,0 +1,432 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. + +cases: + - note: assignment-colonequals-nested-pattern + data: {} + modules: + - | + package test + + result := {"outer": outer, "inner": inner, "tag": tag} if { + [outer, {"meta": {"inner": inner, "tag": tag}}] := ["alpha", {"meta": {"inner": "omega", "tag": "v1"}}] + } + query: data.test.result + want_result: + outer: "alpha" + inner: "omega" + tag: "v1" + + - note: assignment-colonequals-cannot-rebind + data: {} + modules: + - | + package test + + result := true if { + value := "initial" + value := "shadowed" + } + query: data.test.result + error: "ar `value` used before definition below" + + - note: assignment-colonequals-requires-bindable-left + data: {} + modules: + - | + package test + + result := true if { + 1 := value + value = 1 + } + query: data.test.result + error: "assignment operator := requires left-hand side to have bindable variables" + + - note: equals-binds-left-nested-literal + data: {} + modules: + - | + package test + import future.keywords + + result := {"first": first, "second": second, "deep": deep} if { + [first, {"details": [second, deep]}] = ["foo", {"details": ["bar", "baz"]}] + } + query: data.test.result + want_result: + first: "foo" + second: "bar" + deep: "baz" + + - note: equals-binds-right-nested-literal + data: {} + modules: + - | + package test + + result := {"first": first, "second": second, "deep": deep} if { + payload := ["foo", {"details": ["bar", "baz"]}] + payload = [first, {"details": [second, deep]}] + } + query: data.test.result + want_result: + first: "foo" + second: "bar" + deep: "baz" + + - note: equals-both-sides-nested-dependent-order + data: + transitions: + - [{"id": 1, "next": {"target": 2}}, {"id": 2, "payload": {"value": "beta"}}] + - [{"id": 2, "next": {"target": 3}}, {"id": 3, "payload": {"value": "gamma"}}] + modules: + - | + package test + + result := [[left_id, right_id, value] | + some left, right, left_id, right_id, value + data.transitions[_] = [left, right] + [{"id": left_id, "next": {"target": right_id}}, {"id": right_id, "payload": {"value": value}}] = [left, right] + ] + query: data.test.result + want_result: [[1, 2, "beta"], [2, 3, "gamma"]] + + - note: equals-non-shadowing-success + data: {} + modules: + - | + package test + + default result = false + + result = true if { + user_id := "user-1" + [user_id, role] = ["user-1", "admin"] + role = "admin" + } + query: data.test.result + want_result: true + + - note: equals-non-shadowing-mismatch + data: {} + modules: + - | + package test + + default result = false + + result = true if { + user_id := "user-1" + [user_id, role] = ["user-2", "admin"] + } + query: data.test.result + want_result: false + + - note: colon-equals-shadowing-allowed + data: {} + modules: + - | + package test + + x := 10 + + y if { + x := 5 + } + query: data.test.y + want_result: true + + - note: equals-wildcard-both-sides + data: {} + modules: + - | + package test + + result := true if { + [_, _] = [1, 2] + } + query: data.test.result + want_result: true + + - note: equals-incompatible-patterns-error + data: {} + modules: + - | + package test + + result := true if { + [x] = {"key": 1} + } + query: data.test.result + error: "incompatible destructuring patterns: both sides must be arrays or objects with matching structure" + + - note: equals-array-size-mismatch-error + data: {} + modules: + - | + package test + + result := true if { + some a, b, c + [x, y] = [a, b, c] + } + query: data.test.result + error: "mismatch in number of array elements" + + - note: equals-array-literal-length-mismatch-error + data: {} + modules: + - | + package test + + result := true if { + [x, y] = [1, 2, 3] + } + query: data.test.result + error: "mismatch in number of array elements" + + - note: equals-object-literal-keys-mismatch-error + data: {} + modules: + - | + package test + + result := true if{ + {"a": first, "b": second} = {"a": 1, "c": 2} + } + query: data.test.result + error: "object literal keys mismatch. Expected keys [\"a\", \"b\"] got [\"a\", \"c\"]." + + - note: equals-object-key-not-found-error + data: {} + modules: + - | + package test + + result := true if { + some value + {"a": first, "b": second} = {"a": 1, "c": value} + } + query: data.test.result + error: "key \"c\" not found in left-hand side object during destructuring" + + - note: equals-object-field-count-mismatch-error + data: {} + modules: + - | + package test + + result := true if { + some a, b + {"a": first} = {"a": a, "b": b} + } + query: data.test.result + error: "object field count mismatch in assignment: left has 1 fields, right has 2 fields" + + - note: dynamic-object-field-binding + data: {} + modules: + - | + package test + + result := {"target": target, "captured": captured} if { + source := {"target": {"value": {"inner": 1}}, "alt": {"value": {"inner": 2}}} + {"target": {"value": {"inner": target}}, chosen: {"value": {"inner": captured}}} = source + chosen = "alt" + } + query: data.test.result + want_result: + target: 1 + captured: 2 + + - note: some-in-nested-binding + data: {} + modules: + - | + package test + + result := { + [letter, code] | + some letter, detail in {"a": {"info": {"code": 1}}, "b": {"info": {"code": 2}}} + detail.info.code = code + } + query: data.test.result + want_result: + set!: + - ["a", 1] + - ["b", 2] + + - note: some-in-array-length-mismatch-error + data: {} + modules: + - | + package test + + result := true if { + some [x, y] in [[1, 2, 3]] + } + query: data.test.result + error: "mismatch in number of array elements" + + - note: some-in-shadowing + data: {} + modules: + - | + package test + + result := {"before": before, "after": value} if { + value := 0 + before := value + some value in [1, 2] + value == 2 + } + query: data.test.result + error: "var `value` used before definition below" + + - note: some-in-shadowing-equals + data: {} + modules: + - | + package test + + result := true if { + value = 0 + some value in [1, 2] + } + query: data.test.result + error: "var `value` used before definition below" + + - note: some-in-after-shadows + data: {} + modules: + - | + package test + + result := true if { + some value in [1, 2] + value := 1 + } + query: data.test.result + error: "var `value` used before definition below" + + - note: comprehension-nested-binding + data: + records: + - {"type": "user", "profile": {"name": "alice", "roles": ["admin", "user"]}} + - {"type": "user", "profile": {"name": "bob", "roles": ["user"]}} + - {"type": "service", "profile": {"name": "svc"}} + modules: + - | + package test + + result := [[name, role] | + some record, role + data.records[_] = record + record = {"type": "user", "profile": {"name": name, "roles": roles}} + roles[_] = role + role = "admin" + ] + query: data.test.result + want_result: [["alice", "admin"]] + + - note: comprehension-scalar-filter + data: + scores: + - {"value": 10} + - {"value": 20} + - {"value": 30} + modules: + - | + package test + + result := [score | + some entry, score + data.scores[_] = entry + entry = {"value": score} + score >= 20 + ] + query: data.test.result + want_result: [20, 30] + + + - note: parameter-nested-destructuring + data: {} + modules: + - | + package test + + pair([[name, {"primary": role}], {"meta": {"active": active}}]) := [ + [name, {"primary": role}], + {"meta": {"active": active}}, + ] if { + name = "alice" + role = "admin" + active = true + } + + result := {"name": name, "role": role, "active": active} if { + [[name, {"primary": role}], {"meta": {"active": active}}] := pair([ + ["alice", {"primary": "admin"}], + {"meta": {"active": true}}, + ]) + } + query: data.test.result + want_result: + name: "alice" + role: "admin" + active: true + + - note: parameter-nested-destructuring-colonequals-error + data: {} + modules: + - | + package test + + pair([[name, {"primary": role}], {"meta": {"active": active}}]) if { + name := "alice" + role := "admin" + active := true + } + query: data.test.pair + error: "var `name` used before definition below" + + - note: colon-equals-parameter-rebinding-error + data: {} + modules: + - | + package test + + pair([[name, {"primary": role}], {"meta": {"active": active}}]) if { + name := "alice" + role := "admin" + active := true + } + + result := true if { + pair([ + ["alice", {"primary": "admin"}], + {"meta": {"active": true}}, + ]) + } + query: data.test.result + error: "var `name` used before definition below" + + - note: parameter-shadowing + data: {} + modules: + - | + package test + + global_name := "global" + + capture([name, {"role": role}]) if { + name := "local" + role := "admin" + } + + result := {"global": global_name, "param": name, "role": role} if { + capture([name, {"role": role}]) + } + query: data.test.result + error: "var `name` used before definition below" \ No newline at end of file diff --git a/tests/interpreter/cases/binding/walk.yaml b/tests/interpreter/cases/binding/walk.yaml new file mode 100644 index 0000000..ac6a7a7 --- /dev/null +++ b/tests/interpreter/cases/binding/walk.yaml @@ -0,0 +1,20 @@ +# Copyright (c) Microsoft Corporation. +# Licensed under the MIT License. +cases: + - note: walk-loop-index-destructuring + data: + doc: + team: + ops: + members: ["alice"] + dev: + members: ["bob"] + modules: + - | + package test + + result := [[section, role, members] | + walk(data.doc, [[section, role], {"members": members}]) + ] + query: data.test.result + want_result: [["team", "dev", ["bob"]], ["team", "ops", ["alice"]]] \ No newline at end of file diff --git a/tests/interpreter/cases/builtins/numbers/div.yaml b/tests/interpreter/cases/builtins/numbers/div.yaml index 068d025..f2e6da3 100644 --- a/tests/interpreter/cases/builtins/numbers/div.yaml +++ b/tests/interpreter/cases/builtins/numbers/div.yaml @@ -27,7 +27,6 @@ cases: want_result: d: 5.1 e: 3.25 - z: true - note: non-numeric data: {} diff --git a/tests/interpreter/cases/some/tests.yaml b/tests/interpreter/cases/some/tests.yaml index 7041b25..9a8f22b 100644 --- a/tests/interpreter/cases/some/tests.yaml +++ b/tests/interpreter/cases/some/tests.yaml @@ -76,7 +76,7 @@ cases: import future.keywords x { some [1] in [[1, 2]] } query: data.test - error: "array length mismatch. Expected 1 got 2." + error: "mismatch in number of array elements" - note: array-length-mismatch-skipped data: {}