mirror of
https://github.com/microsoft/regorus.git
synced 2026-08-05 02:16:11 +00:00
Add knowledge docs, agent definitions, and skill files
Add comprehensive documentation and GitHub Copilot configuration: - docs/knowledge/: 17 deep-dive knowledge files covering value semantics, RVM architecture, builtins, FFI boundary, feature composition, error handling migration, policy evaluation security, Rego semantics, interpreter/compiler architecture, Azure Policy/RBAC, engine API, time builtins, language extension guide, tooling architecture, causality/partial eval, Rego compiler, Azure Policy aliases, and telemetry/diagnostics - .github/agents/: 16 role-specific AI agent definitions (red-teamer, semantics-expert, architect, performance-engineer, test-engineer, verification-engineer, security-auditor, reliability-engineer, support-engineer, ci-engineer, refactorer, api-steward, program-manager, demo-engineer, dx-engineer, tech-lead) - .github/skills/: 6 workflow skill definitions (thorough-review, design-alternatives, add-builtin, opa-conformance, security-review, verification) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: anakrish <35780660+anakrish@users.noreply.github.com>
This commit is contained in:
copilot-swe-agent[bot]
committed by
GitHub
GitHub
parent
3d16489ec6
commit
524aab5528
@@ -0,0 +1,164 @@
|
||||
---
|
||||
name: add-builtin
|
||||
description: >-
|
||||
Guide for adding new builtin functions to regorus. Use this skill when asked
|
||||
to add a new builtin, implement a missing OPA builtin, or extend the builtin
|
||||
system.
|
||||
allowed-tools: shell
|
||||
---
|
||||
|
||||
# Add Builtin Skill
|
||||
|
||||
Adding a builtin to regorus requires changes in multiple places and careful
|
||||
attention to feature gating, type safety, and OPA conformance.
|
||||
|
||||
## Overview
|
||||
|
||||
Read `docs/knowledge/builtin-system.md` first for the full registration
|
||||
architecture.
|
||||
|
||||
## Steps to Add a Builtin
|
||||
|
||||
### 1. Choose the Right Module
|
||||
|
||||
Builtins are organized by category in `src/builtins/`:
|
||||
|
||||
```
|
||||
src/builtins/
|
||||
aggregates.rs # count, sum, max, min, sort
|
||||
arrays.rs # array.concat, array.slice, array.reverse
|
||||
bitwise.rs # bits.and, bits.or, bits.negate, etc.
|
||||
casts.rs # to_number
|
||||
comparison.rs # opa.runtime
|
||||
conversions.rs # units.parse, units.parse_bytes
|
||||
crypto.rs # crypto.sha256, crypto.x509, etc.
|
||||
encoding.rs # base64, json, yaml, hex, urlquery
|
||||
graphs.rs # graph.reachable, graph.reachable_paths
|
||||
numbers.rs # rand.intn, numbers.range, ceil, floor
|
||||
objects.rs # object.get, object.union, object.filter
|
||||
regex.rs # regex.match, regex.split, regex.find
|
||||
semver.rs # semver.compare, semver.is_valid
|
||||
sets.rs # intersection, union
|
||||
strings.rs # concat, contains, sprintf, etc.
|
||||
time/ # time.now_ns, time.parse_ns, etc.
|
||||
types.rs # is_string, is_number, type_name
|
||||
azure_policy/ # Azure Policy-specific builtins
|
||||
```
|
||||
|
||||
Add your builtin to the appropriate existing module, or create a new module
|
||||
if it represents a new category.
|
||||
|
||||
### 2. Implement the Function
|
||||
|
||||
```rust
|
||||
fn my_builtin(span: &Span, params: &[Ref<Expr>], args: &[Value], strict: bool) -> Result<Value> {
|
||||
// Validate argument count
|
||||
ensure_args_count(span, "my_builtin", params, args, expected_count)?;
|
||||
|
||||
// Type-check arguments — return Undefined for type mismatches (not errors)
|
||||
let arg0 = match &args[0] {
|
||||
Value::String(s) => s,
|
||||
_ => return Ok(Value::Undefined),
|
||||
};
|
||||
|
||||
// Implement the logic
|
||||
// ...
|
||||
|
||||
Ok(result)
|
||||
}
|
||||
```
|
||||
|
||||
Key patterns:
|
||||
- **Return `Value::Undefined`** for type mismatches (OPA semantics)
|
||||
- **Return `Err`** only for genuine errors (wrong arg count, internal failure)
|
||||
- **Use `strict` parameter** for strict mode behavior differences
|
||||
- **Handle `Value::Undefined` inputs** — decide: propagate or treat as error
|
||||
|
||||
### 3. Register the Builtin
|
||||
|
||||
In the same module, add to the registration function:
|
||||
|
||||
```rust
|
||||
pub fn register(m: &mut HashMap<&'static str, BuiltinFcn>) {
|
||||
m.insert("my_category.my_builtin", (my_builtin, 2));
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
The tuple is `(function_pointer, expected_arg_count)`.
|
||||
|
||||
### 4. Feature Gate (if needed)
|
||||
|
||||
If the builtin depends on an optional crate or is language-specific:
|
||||
|
||||
```rust
|
||||
#[cfg(feature = "my-feature")]
|
||||
pub fn register(m: &mut HashMap<&'static str, BuiltinFcn>) {
|
||||
m.insert("my_category.my_builtin", (my_builtin, 2));
|
||||
}
|
||||
```
|
||||
|
||||
Update `Cargo.toml` if adding a new feature flag. Update
|
||||
`docs/knowledge/feature-composition.md` with the new flag.
|
||||
|
||||
### 5. Add Tests
|
||||
|
||||
```rust
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn test_my_builtin_basic() { /* ... */ }
|
||||
|
||||
#[test]
|
||||
fn test_my_builtin_undefined_input() {
|
||||
// Verify Undefined propagation behavior
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_my_builtin_type_mismatch() {
|
||||
// Verify returns Undefined, not error
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_my_builtin_edge_cases() {
|
||||
// Empty inputs, null, very large values, etc.
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### 6. Verify OPA Conformance
|
||||
|
||||
```bash
|
||||
# Run conformance tests
|
||||
cargo test --test opa --features opa-testutil
|
||||
|
||||
# If OPA test data exists for this builtin, verify it passes
|
||||
cargo test --test opa --features opa-testutil -- my_builtin
|
||||
```
|
||||
|
||||
### 7. Update Documentation
|
||||
|
||||
- Add the builtin to `docs/builtins.md`
|
||||
- If it's complex, consider updating `docs/knowledge/builtin-system.md`
|
||||
|
||||
## Checklist
|
||||
|
||||
- [ ] Function implemented with correct signature
|
||||
- [ ] Returns Undefined for type mismatches (not errors)
|
||||
- [ ] Handles Undefined inputs correctly
|
||||
- [ ] Registered with correct name and arg count
|
||||
- [ ] Feature-gated if needed
|
||||
- [ ] Unit tests cover: basic, undefined, type mismatch, edge cases
|
||||
- [ ] OPA conformance tests pass
|
||||
- [ ] Works in both interpreter and RVM
|
||||
- [ ] Documentation updated
|
||||
- [ ] Compiles with `--no-default-features` (if not feature-gated)
|
||||
|
||||
## Reference
|
||||
|
||||
- `docs/knowledge/builtin-system.md` — Full registration architecture
|
||||
- `docs/knowledge/value-semantics.md` — Undefined propagation rules
|
||||
- `docs/knowledge/feature-composition.md` — Feature flag guidance
|
||||
- `src/builtins/` — Existing builtins as examples
|
||||
@@ -0,0 +1,120 @@
|
||||
---
|
||||
name: design-alternatives
|
||||
description: >-
|
||||
Explore multiple design alternatives for a feature or change in regorus.
|
||||
Use this skill when asked to consider different approaches, evaluate
|
||||
tradeoffs, compare implementations, or when facing a non-trivial design
|
||||
decision. Generates and evaluates multiple candidates before recommending.
|
||||
---
|
||||
|
||||
# Design Alternatives Skill
|
||||
|
||||
When facing a non-trivial design decision in regorus, don't commit to the
|
||||
first approach that comes to mind. Generate multiple alternatives, evaluate
|
||||
their tradeoffs against regorus's constraints, and recommend the best option.
|
||||
|
||||
## Strategy
|
||||
|
||||
### Phase 1: Understand the Problem
|
||||
|
||||
Before generating alternatives:
|
||||
|
||||
1. **Clarify the requirement** — what exactly must this achieve?
|
||||
2. **Identify constraints** — which of regorus's constraints apply?
|
||||
- no_std compatibility
|
||||
- 9 FFI binding targets
|
||||
- Dual execution paths (interpreter + RVM)
|
||||
- Feature flag composition
|
||||
- Security-critical correctness
|
||||
- Performance at scale
|
||||
3. **Read relevant knowledge files** from `docs/knowledge/`
|
||||
4. **Study existing patterns** — how does the codebase solve similar problems?
|
||||
|
||||
### Phase 2: Generate Alternatives
|
||||
|
||||
Generate **at least 3 meaningfully different approaches**. Don't generate
|
||||
trivial variations — each alternative should represent a genuinely different
|
||||
design philosophy or tradeoff.
|
||||
|
||||
For each alternative, describe:
|
||||
- **Approach**: what it does and how
|
||||
- **Key design choice**: what makes this different from the others
|
||||
|
||||
Push yourself to consider:
|
||||
- The obvious approach everyone would try first
|
||||
- A simpler approach that sacrifices some capability
|
||||
- A more sophisticated approach that handles more edge cases
|
||||
- An approach that reuses existing infrastructure differently
|
||||
- An approach from a different domain that could apply here
|
||||
|
||||
### Phase 3: Evaluate
|
||||
|
||||
Evaluate each alternative against these dimensions (weight by relevance
|
||||
to the specific problem):
|
||||
|
||||
| Dimension | Description |
|
||||
|-----------|-------------|
|
||||
| **Correctness** | Can this be implemented correctly? How many edge cases? |
|
||||
| **Security** | Attack surface? Resource bounds? Panic safety? |
|
||||
| **Complexity** | How much code? How hard to understand and maintain? |
|
||||
| **Performance** | Runtime cost? Memory cost? Scales with what? |
|
||||
| **Compatibility** | Works with no_std? All FFI targets? All feature combos? |
|
||||
| **Extensibility** | Easy to extend later? Blocks future plans? |
|
||||
| **Testability** | Easy to test? Property-testable? |
|
||||
| **Migration cost** | How much existing code must change? |
|
||||
| **Risk** | What could go wrong? How bad is the failure mode? |
|
||||
|
||||
Be honest about tradeoffs. Every approach has weaknesses — name them
|
||||
explicitly rather than advocating for a favorite.
|
||||
|
||||
### Phase 4: Recommend
|
||||
|
||||
1. **Rank** the alternatives
|
||||
2. **Recommend** one with clear reasoning
|
||||
3. **Identify risks** in the recommended approach
|
||||
4. **Suggest mitigations** for those risks
|
||||
5. **Note what to revisit** — decisions that should be reconsidered
|
||||
if assumptions change
|
||||
|
||||
If no alternative is clearly best, say so. Present the decision to the
|
||||
user with the tradeoffs clearly laid out so they can make an informed choice.
|
||||
|
||||
## Example Decision Framework
|
||||
|
||||
For a decision like "how should we implement partial evaluation":
|
||||
|
||||
**Alternative A: AST-level transformation**
|
||||
- Walk AST, evaluate ground subexpressions, leave symbolic ones
|
||||
- Simple, reuses parser, but loses RVM optimizations
|
||||
|
||||
**Alternative B: RVM-level symbolic execution**
|
||||
- Extend registers with symbolic values, execute normally
|
||||
- Complex, but preserves all optimizations and is more precise
|
||||
|
||||
**Alternative C: Hybrid — compile then reduce**
|
||||
- Compile to RVM, then do a simplification pass on bytecode
|
||||
- Medium complexity, preserves compilation optimizations
|
||||
|
||||
Evaluate each against correctness (Undefined propagation!), complexity,
|
||||
performance, and extensibility. The right answer depends on which
|
||||
constraints matter most for this specific decision.
|
||||
|
||||
## Anti-Patterns
|
||||
|
||||
- **Don't generate strawmen** — every alternative should be genuinely viable
|
||||
- **Don't evaluate only on your preferred dimension** — consider all
|
||||
- **Don't hide tradeoffs** — if an approach is risky, say so clearly
|
||||
- **Don't over-engineer** — sometimes the simplest approach is best
|
||||
- **Don't ignore existing patterns** — the codebase has established idioms
|
||||
|
||||
## Reference
|
||||
|
||||
All knowledge files in `docs/knowledge/` are potentially relevant —
|
||||
choose based on the subsystem being designed for. Key files:
|
||||
|
||||
- `docs/knowledge/rvm-architecture.md` — RVM design constraints
|
||||
- `docs/knowledge/ffi-boundary.md` — FFI compatibility requirements
|
||||
- `docs/knowledge/feature-composition.md` — Feature flag constraints
|
||||
- `docs/knowledge/value-semantics.md` — Value type constraints
|
||||
- `docs/knowledge/language-extension-guide.md` — Extensibility patterns
|
||||
- `docs/knowledge/causality-and-partial-eval.md` — Future architecture vision
|
||||
@@ -0,0 +1,76 @@
|
||||
---
|
||||
name: opa-conformance
|
||||
description: >-
|
||||
Check OPA conformance for regorus changes. Use this skill when modifying
|
||||
Rego evaluation, builtins, or anything that could affect OPA compatibility.
|
||||
Runs conformance tests and analyzes failures.
|
||||
allowed-tools: shell
|
||||
---
|
||||
|
||||
# OPA Conformance Skill
|
||||
|
||||
regorus aims for high conformance with the Open Policy Agent (OPA) reference
|
||||
implementation. This skill helps verify that changes don't break conformance
|
||||
and diagnose any failures.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Modifying Rego evaluation (interpreter or RVM compiler)
|
||||
- Adding or changing builtin functions
|
||||
- Changing the Value type or its operations
|
||||
- Modifying the parser or scheduler
|
||||
- Any change where you're unsure if it affects Rego semantics
|
||||
|
||||
## Running Conformance Tests
|
||||
|
||||
```bash
|
||||
# Full OPA conformance suite
|
||||
cargo test --test opa --features opa-testutil
|
||||
|
||||
# Run with verbose output to see which tests pass/fail
|
||||
cargo test --test opa --features opa-testutil -- --nocapture
|
||||
|
||||
# Run a specific conformance test category
|
||||
cargo test --test opa --features opa-testutil -- test_name_pattern
|
||||
```
|
||||
|
||||
## Analyzing Failures
|
||||
|
||||
When conformance tests fail:
|
||||
|
||||
1. **Read the test case** — OPA conformance tests are in `tests/opa/` and
|
||||
follow a standard structure: input, data, policy, expected result
|
||||
2. **Identify the Rego feature** — which language feature does the failing
|
||||
test exercise? (comprehensions, `with`, negation, builtins, etc.)
|
||||
3. **Check both execution paths** — run the failing test against both the
|
||||
interpreter and RVM to see if the failure is path-specific
|
||||
4. **Compare with OPA spec** — the expected result comes from the OPA
|
||||
reference implementation. Understand why OPA produces that result.
|
||||
5. **Check Undefined propagation** — the most common conformance failure
|
||||
is incorrect Undefined handling. Review `docs/knowledge/value-semantics.md`.
|
||||
|
||||
## Known Non-Conformance
|
||||
|
||||
Some OPA features are intentionally not supported or have known gaps.
|
||||
Before investigating a failure, check if it's in a known category:
|
||||
|
||||
- Check `tests/` for any skip lists or known-failure annotations
|
||||
- Check GitHub issues for tracked conformance gaps
|
||||
- Some builtins may be feature-gated — ensure the right features are enabled
|
||||
|
||||
## After Fixing
|
||||
|
||||
After fixing a conformance issue:
|
||||
|
||||
1. Run the full conformance suite to ensure no regressions
|
||||
2. Run `cargo test` for general test suite
|
||||
3. Verify the fix works in both interpreter and RVM paths
|
||||
4. Update `docs/knowledge/` if the fix reveals a subtle semantic rule
|
||||
|
||||
## Reference
|
||||
|
||||
- `docs/knowledge/rego-semantics.md` — Rego evaluation model
|
||||
- `docs/knowledge/value-semantics.md` — Value type and Undefined
|
||||
- `docs/knowledge/builtin-system.md` — Builtin registration and conformance
|
||||
- `docs/knowledge/interpreter-architecture.md` — Interpreter details
|
||||
- `docs/knowledge/rego-compiler.md` — RVM compiler details
|
||||
@@ -0,0 +1,119 @@
|
||||
---
|
||||
name: security-review
|
||||
description: >-
|
||||
Security-focused review for regorus changes. Use this skill when asked to
|
||||
do a security review, threat analysis, or when reviewing changes to FFI
|
||||
boundaries, resource limits, policy evaluation, or dependency updates.
|
||||
allowed-tools: shell
|
||||
---
|
||||
|
||||
# Security Review Skill
|
||||
|
||||
regorus is a security-critical policy evaluation engine. Policy evaluation
|
||||
bugs can lead to incorrect access control decisions at Azure scale. This skill
|
||||
provides a security-focused review lens.
|
||||
|
||||
## Threat Model
|
||||
|
||||
regorus evaluates **untrusted policies and inputs** provided by external users.
|
||||
The engine must:
|
||||
|
||||
1. **Produce correct results** — a wrong allow/deny is a security bug
|
||||
2. **Not crash** — panics in FFI contexts poison the engine permanently
|
||||
3. **Bound resource usage** — adversarial inputs must not cause DoS
|
||||
4. **Maintain isolation** — evaluation of one policy must not affect another
|
||||
5. **Protect the host** — no arbitrary code execution, file access, or network access
|
||||
|
||||
## Review Approach
|
||||
|
||||
Think adversarially. For each change, ask:
|
||||
|
||||
### Policy Evaluation Correctness
|
||||
|
||||
- Could this change cause a policy to evaluate to a different result?
|
||||
- If the result changes, is that the correct behavior per specification?
|
||||
- What happens with edge-case inputs: empty, null, very large, deeply nested?
|
||||
- What happens when values are Undefined? (`not Undefined = true`)
|
||||
- Are default rules affected?
|
||||
|
||||
### Resource Exhaustion
|
||||
|
||||
- Does this introduce unbounded iteration (no instruction budget check)?
|
||||
- Does this allocate memory proportional to untrusted input size?
|
||||
- Does this add recursion without depth bounds?
|
||||
- Can an adversarial policy trigger O(n²) or worse behavior?
|
||||
- RVM instruction budget is 25,000 — does this change affect instruction
|
||||
count significantly for common policies?
|
||||
|
||||
### Panic Safety
|
||||
|
||||
- Can this code path panic? (`.unwrap()`, `.expect()`, index `[i]`,
|
||||
integer overflow via `as` casts, slice out of bounds)
|
||||
- Is this reachable from FFI? (If so, panic = permanent engine poisoning)
|
||||
- Are all match arms exhaustive?
|
||||
- Are arithmetic operations checked? (`checked_add`, `saturating_mul`, etc.)
|
||||
|
||||
### FFI Boundary
|
||||
|
||||
If the change touches public API or FFI:
|
||||
- Does the handle pattern remain safe? (`Box::into_raw` / `Box::from_raw`)
|
||||
- Is `with_unwind_guard()` used for panic containment?
|
||||
- Do all 9 binding languages handle the change correctly?
|
||||
- Are error codes and status values consistent?
|
||||
- Could a binding language misuse the new API in a way that causes UB?
|
||||
|
||||
### Supply Chain
|
||||
|
||||
If dependencies change:
|
||||
- Is the new dependency necessary?
|
||||
- Does it have known vulnerabilities? (`cargo audit`)
|
||||
- Does it use `unsafe`? How much?
|
||||
- Is it maintained? How many maintainers?
|
||||
- Does it support `no_std` with `default-features = false`?
|
||||
- Could it be replaced with a smaller, more focused crate?
|
||||
|
||||
Run: `cargo audit` and `cargo deny check` after dependency changes.
|
||||
|
||||
### Feature Flag Safety
|
||||
|
||||
- Does this compile with `--all-features`?
|
||||
- Does this compile with `--no-default-features`?
|
||||
- Does the `arc` feature (Rc→Arc) work correctly with this change?
|
||||
- Are `#[cfg(...)]` guards correct and complete?
|
||||
|
||||
## Automated Security Checks
|
||||
|
||||
```bash
|
||||
# Dependency audit
|
||||
cargo audit
|
||||
|
||||
# Dependency policy check
|
||||
cargo deny check
|
||||
|
||||
# Clippy with all features (catches unsafe patterns)
|
||||
cargo clippy --all-features -- -D warnings
|
||||
|
||||
# Clippy with no features (no_std safety)
|
||||
cargo clippy --no-default-features -- -D warnings
|
||||
|
||||
# Miri for memory safety (if nightly available)
|
||||
cargo +nightly miri test
|
||||
```
|
||||
|
||||
## Severity Assessment
|
||||
|
||||
For each finding, assess:
|
||||
|
||||
- **Impact**: what's the worst case if exploited?
|
||||
- **Exploitability**: can an external user trigger this?
|
||||
- **Scope**: how many deployments are affected?
|
||||
|
||||
In regorus, most evaluation bugs are high-impact because they affect
|
||||
policy decisions across all deployments using the engine.
|
||||
|
||||
## Reference
|
||||
|
||||
- `docs/knowledge/policy-evaluation-security.md` — DoS protection, limits
|
||||
- `docs/knowledge/ffi-boundary.md` — Handle pattern, panic containment
|
||||
- `docs/knowledge/feature-composition.md` — Feature flag interactions
|
||||
- `docs/knowledge/value-semantics.md` — Undefined propagation (security-relevant)
|
||||
@@ -0,0 +1,172 @@
|
||||
---
|
||||
name: thorough-review
|
||||
description: >-
|
||||
Multi-agent thorough code review for regorus. Use this skill when asked to
|
||||
do a thorough review, deep review, or comprehensive review of code changes.
|
||||
Orchestrates parallel focused review agents for correctness, security, and
|
||||
polish, then synthesizes findings.
|
||||
allowed-tools: shell
|
||||
---
|
||||
|
||||
# Thorough Review Skill
|
||||
|
||||
You are orchestrating a multi-agent code review of a regorus change. regorus is
|
||||
a security-critical multi-policy-language evaluation engine used in production
|
||||
at Azure scale. Behavioral bugs are security bugs.
|
||||
|
||||
## Strategy
|
||||
|
||||
Run **automated checks first**, then launch **parallel focused review agents**,
|
||||
then **synthesize** their findings into a unified report. You decide the
|
||||
best approach based on the change — the guidance below is a starting point,
|
||||
not a rigid script.
|
||||
|
||||
## Phase 1: Understand the Change
|
||||
|
||||
Before reviewing, understand what changed and why:
|
||||
|
||||
1. Get the diff: `git diff` (unstaged), `git diff --cached` (staged), or
|
||||
`git diff main...HEAD` (branch diff)
|
||||
2. Read the changed files and their surrounding context
|
||||
3. Identify which subsystems are affected
|
||||
4. Read relevant knowledge files from `docs/knowledge/` — consult the
|
||||
reference table in `.github/copilot-instructions.md`
|
||||
|
||||
## Phase 2: Automated Checks
|
||||
|
||||
Run these before the AI review passes. Fix any failures before proceeding.
|
||||
|
||||
```bash
|
||||
# Format check
|
||||
cargo fmt --check
|
||||
|
||||
# Lint with all features
|
||||
cargo clippy --all-features -- -D warnings
|
||||
|
||||
# Lint with no features (no_std)
|
||||
cargo clippy --no-default-features -- -D warnings
|
||||
|
||||
# Run tests
|
||||
cargo test
|
||||
|
||||
# OPA conformance (if Rego evaluation changed)
|
||||
cargo test --test opa --features opa-testutil
|
||||
```
|
||||
|
||||
Report any automated check failures immediately — they take priority over
|
||||
review findings.
|
||||
|
||||
## Phase 3: Parallel Focused Reviews
|
||||
|
||||
Launch multiple focused review agents in parallel. Each agent reviews the
|
||||
same diff but with a different perspective. Select agents based on what
|
||||
changed — not every PR needs all agents.
|
||||
|
||||
### Agent Selection Guide
|
||||
|
||||
Choose agents based on the change type:
|
||||
|
||||
| Change type | Always invoke | Also consider |
|
||||
|-------------|--------------|---------------|
|
||||
| **Rego evaluation** | `semantics-expert`, `test-engineer` | `red-teamer`, `performance-engineer` |
|
||||
| **RVM/compiler** | `semantics-expert`, `verification-engineer` | `performance-engineer`, `reliability-engineer` |
|
||||
| **FFI/bindings** | `architect`, `api-steward` | `security-auditor`, `test-engineer` |
|
||||
| **New feature** | `architect`, `program-manager`, `test-engineer` | `semantics-expert`, `demo-engineer` |
|
||||
| **Security-sensitive** | `red-teamer`, `security-auditor` | `reliability-engineer`, `verification-engineer` |
|
||||
| **Performance** | `performance-engineer`, `test-engineer` | `reliability-engineer` |
|
||||
| **Refactoring** | `refactorer`, `test-engineer` | `architect` |
|
||||
| **CI/build** | `ci-engineer` | `dx-engineer` |
|
||||
| **API change** | `api-steward`, `architect` | `dx-engineer`, `demo-engineer` |
|
||||
| **Any significant PR** | `tech-lead` (after other agents) | — |
|
||||
|
||||
### Invoking Agents
|
||||
|
||||
For each selected agent, launch it as a subagent with:
|
||||
1. The full diff
|
||||
2. A summary of what changed and why
|
||||
3. The relevant knowledge file context (from Phase 1)
|
||||
|
||||
Agents are defined in `.github/agents/`. Each has specific focus areas,
|
||||
knowledge file references, and output formats. Let them do their work
|
||||
independently — diversity of perspective is the goal.
|
||||
|
||||
### Cross-Agent Context
|
||||
|
||||
To enable agents to build on each other's findings, use a shared context
|
||||
document. After each agent completes, append its key findings to the context
|
||||
so subsequent agents can reference them.
|
||||
|
||||
**Context structure:**
|
||||
|
||||
```markdown
|
||||
## Shared Review Context
|
||||
|
||||
### Change Summary
|
||||
(Your Phase 1 analysis — shared with all agents)
|
||||
|
||||
### Subsystems Affected
|
||||
(List of modules, features, and boundaries touched)
|
||||
|
||||
### Agent Findings
|
||||
#### [agent-name] — [timestamp]
|
||||
- Key findings: ...
|
||||
- Concerns raised: ...
|
||||
- Questions for other agents: ...
|
||||
```
|
||||
|
||||
**Context flow:**
|
||||
1. Start with your Phase 1 analysis as the seed context
|
||||
2. Launch the first wave of agents (e.g., semantics-expert + red-teamer)
|
||||
3. Append their findings to the context
|
||||
4. Launch the second wave with the enriched context (e.g., test-engineer
|
||||
can now see what the semantics-expert flagged)
|
||||
5. Pass the full context to tech-lead for final synthesis
|
||||
|
||||
This is optional — for simple changes, parallel-only is fine. Use the
|
||||
context protocol when agents' findings might inform each other (e.g.,
|
||||
the red-teamer finds an attack vector that the test-engineer should
|
||||
write a test for).
|
||||
|
||||
## Phase 4: Synthesize
|
||||
|
||||
Invoke the **tech-lead** agent with all agent findings to produce a unified
|
||||
assessment. The tech-lead will:
|
||||
|
||||
1. **Collect** all findings from all agents
|
||||
2. **Deduplicate** — multiple agents may flag the same issue
|
||||
3. **Resolve conflicts** — when agents disagree, apply the priority framework
|
||||
(correctness > security > reliability > stability > performance > maintainability > DX)
|
||||
4. **Categorize** every finding:
|
||||
- 🔴 **Correctness** — wrong result, logic error, behavioral bug
|
||||
- 🟠 **Security** — could affect policy evaluation, resource limits, DoS
|
||||
- 🟡 **Robustness** — panic path, missing error handling, unchecked arithmetic
|
||||
- 🔵 **Polish** — duplication, naming, style, documentation, dead code
|
||||
- ⚪ **Nit** — minor style preference
|
||||
5. **Sort** by severity (🔴 first, then 🟠, 🟡, 🔵, ⚪)
|
||||
6. **Present** the unified report with clear context for each finding:
|
||||
- File and line reference
|
||||
- What the issue is
|
||||
- Why it matters
|
||||
- Suggested fix (if not obvious)
|
||||
7. **Make the call**: Ship / Ship with follow-ups / Revise / Redesign
|
||||
|
||||
## Phase 5: Iterate
|
||||
|
||||
If 🔴 or 🟠 findings exist:
|
||||
- Help the author fix them
|
||||
- After fixes, re-run the relevant focused review
|
||||
- Repeat until no significant findings remain
|
||||
|
||||
A change is ready when you would trust it in production at scale.
|
||||
|
||||
## Adapting the Strategy
|
||||
|
||||
Not every change needs all agents. Use your judgment:
|
||||
|
||||
- **Tiny fix** (1-2 lines): a single correctness pass may suffice
|
||||
- **New feature**: all three agents, plus extra attention to test coverage
|
||||
- **Refactor**: polish agent is primary, correctness verifies behavior preservation
|
||||
- **Dependency update**: security agent is primary
|
||||
- **FFI change**: security agent with heavy focus on `ffi-boundary.md`
|
||||
|
||||
The goal is thoroughness, not ceremony. Skip what doesn't add value.
|
||||
@@ -0,0 +1,143 @@
|
||||
---
|
||||
name: verification
|
||||
description: >-
|
||||
Formal verification and memory safety verification for regorus. Use this
|
||||
skill when asked about Miri, formal verification, Z3, Verus, property
|
||||
testing, or when verifying safety properties of regorus code.
|
||||
allowed-tools: shell
|
||||
---
|
||||
|
||||
# Verification Skill
|
||||
|
||||
regorus uses multiple verification approaches to ensure correctness and
|
||||
memory safety. This skill guides verification efforts.
|
||||
|
||||
## Verification Tiers
|
||||
|
||||
### Tier 1: Miri (Active — in CI)
|
||||
|
||||
Miri detects undefined behavior in unsafe code, memory leaks, and
|
||||
concurrency bugs. regorus runs Miri in CI.
|
||||
|
||||
```bash
|
||||
# Run Miri on the test suite
|
||||
cargo +nightly miri test
|
||||
|
||||
# Run Miri on specific tests
|
||||
cargo +nightly miri test -- test_name
|
||||
|
||||
# Run with stricter checks
|
||||
MIRIFLAGS="-Zmiri-strict-provenance" cargo +nightly miri test
|
||||
```
|
||||
|
||||
**What Miri catches:**
|
||||
- Use-after-free, double-free
|
||||
- Out-of-bounds memory access
|
||||
- Uninitialized memory reads
|
||||
- Data races (with `-Zmiri-check-stacked-borrows`)
|
||||
- Memory leaks
|
||||
|
||||
**regorus context:** The core crate is `#![forbid(unsafe_code)]`, so Miri
|
||||
is most relevant for FFI binding crates (`bindings/ffi/`) where unsafe is
|
||||
allowed. Also useful for verifying `Rc::make_mut()` patterns.
|
||||
|
||||
### Tier 2: Property Testing (Recommended)
|
||||
|
||||
Use `proptest` or `quickcheck` to test properties that must hold for all
|
||||
inputs:
|
||||
|
||||
```rust
|
||||
use proptest::prelude::*;
|
||||
|
||||
proptest! {
|
||||
#[test]
|
||||
fn value_roundtrip(v in arb_value()) {
|
||||
let json = v.to_json_str();
|
||||
let parsed = Value::from_json_str(&json)?;
|
||||
prop_assert_eq!(v, parsed);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn eval_deterministic(policy in arb_policy(), input in arb_input()) {
|
||||
let r1 = engine.eval(&policy, &input)?;
|
||||
let r2 = engine.eval(&policy, &input)?;
|
||||
prop_assert_eq!(r1, r2);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Properties worth testing in regorus:**
|
||||
- Value serialization round-trips
|
||||
- Evaluation determinism (same input → same output)
|
||||
- Interpreter/RVM equivalence (both paths produce same result)
|
||||
- Undefined propagation consistency
|
||||
- Resource limit enforcement (instruction budget halts execution)
|
||||
- RVM program serialization round-trips
|
||||
|
||||
### Tier 3: Z3 / SMT Solving (Planned)
|
||||
|
||||
For verifying policy properties symbolically:
|
||||
|
||||
- **Policy satisfiability**: is there any input that satisfies this policy?
|
||||
- **Policy equivalence**: do two policies produce the same result for all inputs?
|
||||
- **Policy subsumption**: does policy A imply policy B?
|
||||
- **Unreachable rules**: are there rules that can never fire?
|
||||
|
||||
This connects to the partial evaluation vision in
|
||||
`docs/knowledge/causality-and-partial-eval.md`.
|
||||
|
||||
### Tier 4: Verus (Planned)
|
||||
|
||||
Verus enables verified Rust — proving properties about Rust code at
|
||||
compile time. Potential targets in regorus:
|
||||
|
||||
- **Value type invariants**: prove that Value operations preserve type safety
|
||||
- **RVM instruction safety**: prove that well-formed programs cannot cause
|
||||
register overflow or invalid memory access
|
||||
- **Scheduler correctness**: prove that topological sort produces valid order
|
||||
- **Resource limit enforcement**: prove that instruction budget is checked
|
||||
|
||||
## Verification Strategies by Subsystem
|
||||
|
||||
### Value Type (`src/value.rs`)
|
||||
- Property test: all operations handle Undefined correctly
|
||||
- Property test: comparison is total ordering
|
||||
- Property test: serialization round-trips for all Value variants
|
||||
- Miri: Rc::make_mut patterns don't alias
|
||||
|
||||
### RVM (`src/rvm/`)
|
||||
- Property test: program serialization round-trips
|
||||
- Property test: instruction budget halts execution within bounds
|
||||
- Property test: register allocation stays within frame bounds
|
||||
- Miri: frame stack operations are memory-safe
|
||||
|
||||
### FFI (`bindings/ffi/`)
|
||||
- Miri: handle create/destroy cycles don't leak
|
||||
- Miri: panic containment doesn't cause UB
|
||||
- Property test: poisoned engine rejects all operations
|
||||
|
||||
### Builtins (`src/builtins/`)
|
||||
- Property test: builtins return Undefined (not error) for type mismatches
|
||||
- Property test: time parsing matches OPA reference for valid inputs
|
||||
- Property test: string operations handle UTF-8 edge cases
|
||||
|
||||
## Running Verification
|
||||
|
||||
```bash
|
||||
# Tier 1: Miri
|
||||
cargo +nightly miri test
|
||||
|
||||
# Tier 2: Property tests (if added)
|
||||
cargo test --test prop_tests
|
||||
|
||||
# Full verification suite
|
||||
cargo +nightly miri test && cargo test && cargo test --test opa --features opa-testutil
|
||||
```
|
||||
|
||||
## Reference
|
||||
|
||||
- `docs/knowledge/policy-evaluation-security.md` — Security properties to verify
|
||||
- `docs/knowledge/value-semantics.md` — Value invariants
|
||||
- `docs/knowledge/rvm-architecture.md` — RVM safety properties
|
||||
- `docs/knowledge/ffi-boundary.md` — FFI safety requirements
|
||||
- `docs/knowledge/causality-and-partial-eval.md` — Symbolic analysis vision
|
||||
Reference in New Issue
Block a user