Personas 0.2.0

Persona Agent

A local-first .NET 10 tool that builds an evidence-grounded reviewer persona of a specific engineer from their history (PRs, review comments, commits, code, docs) and uses it to answer questions and review pull requests the way that person would — in their voice, scoped to their expertise, and with every finding traceable to real evidence.

It is designed to run in addition to generic AI reviewers: it targets the small, repeated, domain-specific issues a particular engineer reliably flags, so the human expert can focus on the hard problems.

How it works

ingest                normalize PRs / comments / commits / code / docs from
                      Azure DevOps, GitHub, local Git, and folders
   -> index           evidence chunks -> SQLite FTS5 (BM25) + local ONNX embeddings, fused with RRF
   -> build           cluster the persona's recurring review comments, mine deterministic
                      "matcher" checks from them via Copilot, back-test each against history,
                      and write a versioned persona pack (memory, rubric, tone, mined checks)
   -> review          run the mined checks on a PR diff (deterministic, evidence-anchored),
                      merged ahead of the LLM's fuzzy residue, severity-ordered
   -> eval / feedback  measure precision/recall against history; record reviewer reactions
                      so dismissed checks stop firing

Key properties:

• Evidence-grounded. Every persona memory, rubric rule, and mined check cites real artifacts. A review finding without a citation (or pointing away from a changed line) is suppressed.
• Precision-first. Mined checks are kept only if they back-test well against the PRs whose comments produced them. Reviewer feedback suppresses checks the team keeps dismissing.
• Local & offline-capable. SQLite + local ONNX embeddings; no cloud vector DB. The only LLM dependency is the GitHub Copilot SDK, and every LLM path falls back to deterministic behavior on failure.
• Team mode. Run several personas over one PR and merge their findings, attributed to whoever raised each.

Requirements

• .NET SDK 10
• Optional: GitHub Copilot CLI / SDK auth for LLM-backed mining and review (llm.provider: github-copilot-sdk)
• Optional: local ONNX embedding model in models/ for semantic search (keyword search works without it)
• Optional: Git CLI for localGit ingestion
• Optional: Azure login compatible with DefaultAzureCredential (or a PAT) for live Azure DevOps ingestion
• Optional: a GitHub token for GitHub ingestion / PR review

Download the shared embedding model once:

./scripts/download-model.ps1     # writes models/all-MiniLM-L6-v2.onnx and models/vocab.txt

Examples below use persona <command>. From source, that is dotnet run --project "src/Persona.Cli/Persona.Cli.csproj" -- <command>. Build a single-file persona executable with ./scripts/publish.ps1 -Runtime win-x64.

Quick start

persona init --data persona-data --force
# edit persona-data/persona.yaml: set a real source + persona identity
persona validate    --data persona-data
persona ingest      --data persona-data
persona index       --data persona-data
persona build       --data persona-data --persona payments-expert
persona review      --data persona-data --persona payments-expert --diff change.diff
persona eval        --data persona-data --persona payments-expert

Every command prints a single JSON object on stdout (PascalCase properties) and returns a deterministic exit code, so it composes cleanly in scripts and CI.

End-to-end runbook

The examples use persona id payments-expert; substitute your own. --config <path> selects a manifest other than the data directory's persona.yaml. --data <path> is required whenever your dataDirectory is not the default ./persona-data.

1. Initialize and validate

persona init        --data persona-data --force
persona validate    --data persona-data
persona ado-validate --data persona-data    # live ADO probe (only if you configured ADO)

validate returns status: valid with errors: []. ado-validate probes each configured Azure DevOps source (repository listing, sample PR threads/commits/iterations/work items) with retries on 429/502/503/504. Resolve any errors before ingesting.

2. Ingest evidence

persona ingest --data persona-data

Writes provider-native JSON under raw/ and provider-neutral normalized/*.jsonl. Ingestion captures, where available: PRs, review comments (with thread resolution status and reactions/votes), commits, changed files, work items, local-git commits + C# symbols/chunks, and documents/transcripts. For Azure DevOps, real unified diffs are fetched only for files that received review comments (no unrelated files/PRs). maxPullRequests per source caps ingestion volume.

Targeted re-ingest of only ADO PR iteration changes:

persona ingest --data persona-data --code-changes-only

3. Index for search

persona index --data persona-data

Writes indexes/evidence-chunks.jsonl and indexes/evidence.db (SQLite + FTS5, plus ONNX embedding blobs). If the model files under models/ are missing, the output reports embeddings disabled and FTS5 keyword search still works.

4. Resolve identity candidates

build resolves persona aliases against PR/commit/comment authors. Anything it cannot confidently resolve is listed under unresolvedCandidates. Approve or reject manually (persisted in personas/<id>/identity-overrides.json, applied by build/ask/review/eval and stable across rebuilds):

persona identity list    --data persona-data --persona payments-expert
persona identity approve --data persona-data --persona payments-expert --candidate "azure-devops:alice@example.com" --reason "verified alias"
persona identity reject  --data persona-data --persona payments-expert --candidate "local-git:bot@example.com"      --reason "service account"

5. Build the persona pack

persona build --data persona-data --persona payments-expert

Writes personas/payments-expert/persona-pack.json plus the exploded companions (memory.json, review-rubric.json, tone-profile.json, evidence-map.json, matcher-specs.json, build-report.json) and per-stage diagnostics under build-stages/.

With llm.provider: github-copilot-sdk the build also mines matcher specs: it clusters the persona's recurring comments, asks Copilot (concurrently — one fresh CLI instance per cluster, default 10) to express each cluster as a deterministic check, back-tests each candidate against the historical diffs whose comments produced it, and keeps only those clearing the recall / false-positive thresholds. Deterministic builds mine nothing.

6. Ask and review

persona ask    --data persona-data --persona payments-expert --query "How should we review retry behavior?"
persona review --data persona-data --persona payments-expert --diff change.diff --persist-run-artifacts

review runs the persona's mined checks over the diff and merges those deterministic, evidence-anchored findings ahead of the LLM's residue, deduped by file/line and ordered by the severity ladder. --persist-run-artifacts writes context-pack.json / agent-input.json / agent-output.json / verified-output.json under runs/<runId>/ for debugging.

Review a PR by number instead of a diff file:

persona review --data persona-data --persona payments-expert --ado-pr 12345 --repo checkout-service
persona review --data persona-data --persona payments-expert --gh-pr  42    --repo checkout-service

Team / domain review — pass --persona more than once. Findings are merged and attributed to whichever personas raised them (mode: team, plus per-persona summaries):

persona review --data persona-data --persona payments-expert --persona platform-expert --diff change.diff

7. Evaluate and calibrate

persona eval --data persona-data --persona payments-expert

With no --eval-file, eval performs historical replay: it picks PRs the persona actually reviewed, hides their comments, re-reviews the diff, and scores recovery. Primary metrics are precision and category recall; misses are attributed to retrieval vs. reasoning vs. verification. The report also includes a threshold calibration suggestion — the lowest review.minConfidence whose false-positive rate stays within target (default 0.25), maximizing recall. It is a suggestion only; nothing is rewritten.

Diagnose retrieval bottlenecks for hidden comments:

persona eval-diagnose --data persona-data --persona payments-expert

Fixture-based smoke check instead of replay: persona eval --eval-file cases.json (schema under Eval).

8. Close the feedback loop

After deployment, record how reviewers reacted to the persona's findings. Once a mined spec accrues enough dismissals it is suppressed in later reviews (so the persona stops re-raising rejected checks):

persona feedback --data persona-data --persona payments-expert --spec "matcher:payments-expert:reliability:0" --outcome dismissed --reason "noisy"
persona feedback --data persona-data --persona payments-expert --spec "matcher:payments-expert:testing:1"     --outcome accepted

--outcome is accepted | fixed | dismissed | false-positive. The --spec id is the matcher-spec prefix of a review finding's id. Stored append-only under feedback/<persona>.jsonl.

Configuration

persona init writes a documented persona.yaml. Key sections:

Sources

sources:
  azureDevOps:
    - name: main-ado
      organizationUrl: https://dev.azure.com/my-org
      project: MyProject
      authentication: auto            # auto | default-azure-credential | az-cli | pat
      patEnvironmentVariable: AZDO_PAT
      maxPullRequests: 100
      repositories: [checkout-service, billing-api]
  github:
    - name: main-github
      apiBaseUrl: https://api.github.com
      owner: my-org
      authentication: auto            # auto | token | pat | none
      tokenEnvironmentVariable: GITHUB_TOKEN
      maxPullRequests: 100
      repositories: [checkout-service]
  localGit:
    - name: checkout-service
      path: C:/src/checkout-service    # reads git log + Roslyn C# symbols/chunks
  documents:
    - name: architecture-docs
      path: ./docs
      include: ["**/*.md"]
      exclude: []
  transcripts:
    - name: design-reviews
      enabled: true
      path: ./transcripts
      include: ["**/*.md", "**/*.txt"]

• ADO auto tries DefaultAzureCredential, then az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798, then the PAT env var. Sample values (e.g. my-org) are skipped with a warning.
• maxPullRequests is the primary guard on the build-time budget for prolific reviewers.

Personas

personas:
  payments-expert:
    displayName: Payments Expert
    toneMimicryPercent: 100
    identities:
      - { kind: email, value: alice@example.com }
      - { kind: azure-devops-user-id, value: aad-user-id-123 }
    scope:
      repositories: [checkout-service, billing-api]
      includeArtifacts: [authored_prs, reviewed_prs, review_comments, docs, transcripts]

Identity kinds include email, azure-devops-user-id, azure-devops-descriptor, github-login, github-user-id, and git-author. Define as many personas as you like; run them individually or together (team mode).

LLM

llm:
  provider: github-copilot-sdk        # github-copilot-sdk | deterministic | disabled
  defaultModel: auto                  # or a model id, e.g. claude-opus-4.8
  reasoning: { personaBuild: medium, ask: medium, review: high, verify: medium }

github-copilot-sdk uses the official GitHub.Copilot.SDK; any failure falls back to deterministic, citation-first output with a warning. deterministic/disabled are fully local (used by the test suite and offline runs); they also skip matcher mining.

Review (incl. feedback)

review:
  maxFindings: 10
  minConfidence: medium               # global floor (low|medium|high)
  categoryMinConfidence: {}           # per-category overrides, e.g. security: high
  postComments: false
  feedbackDismissalThreshold: 0.5     # suppress a spec dismissed at/above this rate ...
  minFeedbackSamples: 3               # ... once it has this many recorded reactions

Persona build (incl. matcher mining)

personaBuild:
  enableMatcherMining: true           # github-copilot-sdk only
  matcherMiningConcurrency: 10        # fresh Copilot CLI instance per cluster
  minMatcherClusterSupport: 3         # min distinct comments before mining a cluster
  minMatcherRecall: 0.5               # back-test gate (recall over source PRs)
  maxMatcherFalsePositiveRate: 0.25   # back-test gate (firing on non-commented files)
  # extraction floors:
  minDeterministicCategoryRescueCount: 3
  minDeterministicMemoryKindRescueCount: 3
  minMemoryStatementLength: 20
  minRubricTitleLength: 12
  minRubricDescriptionLength: 20

Retrieval

retrieval: { maxContextItems: 40, keywordTopK: 50, semanticTopK: 50, finalTopK: 40 }

Copilot environment variables

• PERSONA_COPILOT_GITHUB_TOKEN — explicit GitHub token for the SDK
• PERSONA_COPILOT_CLI_PATH / PERSONA_COPILOT_CLI_URL — custom CLI binary or server
• PERSONA_COPILOT_HOME — Copilot state directory
• PERSONA_COPILOT_LOG_LEVEL — CLI log level (default error)
• PERSONA_COPILOT_TIMEOUT_MINUTES — per-call timeout (default 5, clamped 1–60)
• PERSONA_COPILOT_SDK_DISABLED=true — force deterministic fallback (tests / offline)
• Embedding model paths come from embedding.modelPath / embedding.vocabPath.

Commands

Global flags: --data <path>, --config <path>, --top <n>, --output-file <path>, --persist-run-artifacts, --run-artifacts-dir <path>, --force.

Eval

Fixture file schema for persona eval --eval-file cases.json:

{
  "cases": [
    { "id": "ask-idempotency", "kind": "ask", "personaId": "payments-expert",
      "query": "How should idempotency behave during retry storms?",
      "expectedContains": ["Payments Expert"], "requireCitations": true },
    { "id": "review-idempotency", "kind": "review", "personaId": "payments-expert",
      "diffPath": "change.diff", "requireCitations": true, "minFindings": 1 }
  ]
}

Data layout

persona-data/
  persona.yaml
  raw/            provider-native JSON, kept before normalization
  normalized/     *.jsonl: pull-requests, review-comments, commits, code-changes,
                  work-items, documents, transcripts, code-symbols, code-chunks
  indexes/        evidence-chunks.jsonl, evidence.db (SQLite + FTS5)
  personas/<id>/  persona-pack.json + memory/review-rubric/tone-profile/evidence-map/
                  matcher-specs/build-report.json, build-stages/, identity-map.json,
                  identity-overrides.json
  feedback/       <persona>.jsonl  (reviewer reactions)
  runs/<runId>/   persisted ask/review/eval artifacts

Run as a published tool (dnx)

Personas is published to NuGet as Personas — one framework-dependent package per platform (win-x64/arm64, osx-arm64, linux-x64/arm64); the right one is selected automatically. The embedding model is baked in, so installed personas work with no extra downloads. Run it without installing using dnx (.NET 10's npx-equivalent), or install it globally:

# one-shot, no install (resolves + caches the tool on first use):
dnx Personas review --persona <name> --diff change.diff --output-file review.json

# or install once; the command is then `personas`:
dotnet tool install -g Personas
personas review --persona <name> --diff change.diff --output-file review.json

--output-file <path> writes the JSON result to a file (recommended for automation); stdout then prints only {"status":"ok","outputFile":"..."}. With no --data, the data directory defaults to PERSONAS_HOME (~/.personas).

MCP server (VS Code Copilot & GitHub Copilot CLI)

personas mcp starts a Model Context Protocol server over stdio (built on the official ModelContextProtocol .NET SDK) so coding agents can use an installed persona. It exposes read/compute-only tools:

Install the tool first so the server's stdout stays a clean JSON-RPC channel (prefer an installed personas mcp over dnx as the launch command), then register it with your client:

// VS Code — .vscode/mcp.json (the GitHub Copilot CLI takes an equivalent MCP config)
{
  "servers": {
    "personas": { "type": "stdio", "command": "personas", "args": ["mcp"] }
  }
}

Then, in agent mode, ask e.g. "Review this PR as matan." The review_pull_request / ask_persona tools drive the persona's Copilot engine, so they need GitHub Copilot auth (the same login you already use); the other tools are pure reads. The server resolves personas from --data (PERSONAS_HOME by default) and writes all logs to stderr.

Distribute personas

Ship a built persona as a portable .persona bundle, or as a NuGet package containing one or more personas:

# Export to a .persona file (full tier by default; --pack for the brain only):
personas export --persona <id> --out <id>.persona

# Package one or more personas into a NuGet, then push:
personas pack --package-id MyTeam.Personas --persona <id> [--persona <id2>] --package-version 1.0.0 --out ./out
dotnet nuget push ./out/MyTeam.Personas.1.0.0.nupkg --source <feed> --api-key <key>

# Install (from a file or a feed), then run by name with no persona.yaml:
personas install ./<id>.persona
personas install MyTeam.Personas --source <feed>
personas list

Agent skills

skills/ contains Agent Skills that teach AI agents to drive this CLI: review-with-persona, create-persona, and distribute-personas (each a SKILL.md plus PowerShell Core helper scripts under scripts/).

Troubleshooting

• Semantic search disabled — expected until models/all-MiniLM-L6-v2.onnx and models/vocab.txt exist; run ./scripts/download-model.ps1. Keyword search still works.
• Azure DevOps skips ingestion — check organizationUrl isn't the sample value, project is set, authentication is supported, and the shell can authenticate (or AZDO_PAT is set).
• Ask/review says persona pack missing — run persona build --persona <id> first.
• Copilot falls back to deterministic — auth/SDK startup failed; the run still completes with a warning. Force it locally with $env:PERSONA_COPILOT_SDK_DISABLED = "true".
• Invalid YAML — run persona validate; parser errors are returned as JSON validation errors.

Development

dotnet build "PersonaAgent.slnx"
dotnet test  "PersonaAgent.slnx"
./scripts/publish.ps1 -Runtime win-x64

Limitations

• The GitHub Copilot SDK integration uses the public preview SDK and may need updates if its API changes; it has not yet been validated against a live account end-to-end in this environment.
• Live Azure DevOps / GitHub API behavior is covered by test doubles but should be validated against a real org before relying on it.
• Code structural analysis is C# (Roslyn) only; other languages rely on the regex/manifest/path matcher primitives.
• GitHub inline-comment thread resolution requires GraphQL and is not yet captured (reactions are).