SystemHarness.Windows 0.28.0

system-harness

A computer-use primitives library for .NET — eyes, hands, and shell in one harness.

system-harness provides a unified interface for programmatic and interactive computer control. It wraps shell execution, process management, filesystem operations, screen capture, OCR, input simulation, UI automation, and document processing into a single coherent API.

No AI inside. No opinions about your agent framework. Just the primitives you need to use a computer — whether you're a bot or a human writing automation scripts.

Why

AI agents need to operate computers. The "brain" (LLM) decides what to do. But it still needs:

• Shell — run cmd, powershell, bash commands and get results
• Process — launch, kill, and list running programs
• FileSystem — read, write, move, delete files and directories
• Screen — capture what's on screen (for vision-capable models or logging)
• Mouse / Keyboard — click, type, drag when there's no API and GUI is the only way
• OCR — read text from screen regions without requiring vision models
• UI Automation — interact with UI elements by accessibility tree, not pixel coordinates
• Office Documents — read and write Word, Excel, PowerPoint, HWP without Office installed

Today, you stitch together 5+ libraries to get all of this. system-harness is one library, one interface, three layers:

Layer 1: Programmatic Control (fast, precise, preferred)
  Shell, Process, FileSystem, Window, Clipboard, Display, SystemInfo

Layer 2: Vision + Action (when GUI is the only way)
  Screen, Mouse, Keyboard, OCR, UIAutomation, TemplateMatcher, DialogHandler

Layer 3: App Automation (document processing)
  Office (Word, Excel, PowerPoint, HWP) — no installation required

Use Layer 1 whenever possible. Fall back to Layer 2 when you must.

Quick Start

dotnet add package SystemHarness.Core
dotnet add package SystemHarness.Windows  # Windows implementation

using SystemHarness;
using SystemHarness.Windows;

using var harness = new WindowsHarness();

// Layer 1 — Programmatic
var result = await harness.Shell.RunAsync("cmd", "/C echo Hello!");
Console.WriteLine(result.StdOut);   // "Hello!\r\n"

await harness.FileSystem.WriteAsync("hello.txt", "world");
var content = await harness.FileSystem.ReadAsync("hello.txt");

await harness.Process.StartAsync("notepad.exe");
await harness.Window.FocusAsync("Notepad");

// Layer 2 — Vision + Action
var screenshot = await harness.Screen.CaptureAsync();
// screenshot.Base64, screenshot.Width, screenshot.Height, screenshot.MimeType

await harness.Mouse.ClickAsync(350, 200);
await harness.Keyboard.TypeAsync("Hello World");
await harness.Keyboard.HotkeyAsync(default, Key.Ctrl, Key.S);

// OCR — read text from screen
var ocrResult = await harness.Ocr.RecognizeScreenAsync();
Console.WriteLine(ocrResult.Text);

// UI Automation — interact with elements by name
var tree = await harness.UIAutomation.GetTreeAsync("Notepad");
await harness.UIAutomation.TypeIntoAsync("Notepad", "Edit", "Hello from automation");

NuGet Packages

MCP Server (AI Tool Integration)

system-harness includes a built-in Model Context Protocol server with 174 commands across 25 categories, accessed through 3 MCP tools using a command dispatch pattern.

Installation

Download the latest release from GitHub Releases and extract it to a directory of your choice.

Configuration

Claude Desktop / Claude Code (claude_desktop_config.json or .mcp.json):

{
  "mcpServers": {
    "system-harness": {
      "command": "C:/path/to/SystemHarness.Mcp.exe"
    }
  }
}

From source (development):

{
  "mcpServers": {
    "system-harness": {
      "command": "dotnet",
      "args": ["run", "--project", "src/SystemHarness.Mcp"]
    }
  }
}

3 MCP Tools

Instead of 174 individual tool definitions (which consume ~12,000 tokens per API call), commands are accessed through 3 dispatch tools:

Command Categories

Compound Facades (reduce multiple tool calls to one)

Office Documents

Read and write Microsoft Office and Korean HWP documents without Office installed — uses OpenXML and OWPML directly.

using SystemHarness.Apps.Office;

// Register in DI
services.AddOfficeReaders();

// Read documents to markdown
var wordContent = await documentReader.ReadWordAsync("report.docx");
var excelContent = await documentReader.ReadExcelAsync("data.xlsx");
var pptxContent = await documentReader.ReadPowerPointAsync("slides.pptx");
var hwpContent = await hwpReader.ReadHwpxAsync("document.hwpx");

// Write documents from structured content
await documentReader.WriteWordAsync("output.docx", documentContent);
await documentReader.WriteExcelAsync("output.xlsx", spreadsheetContent);

// Find and replace
await documentReader.FindReplaceWordAsync("template.docx", new() { { "{{name}}", "John" } });

Safety Features

using var harness = new WindowsHarness(new HarnessOptions
{
    // Block dangerous commands (format, shutdown, rm -rf, etc.)
    CommandPolicy = CommandPolicy.CreateDefault(),

    // Record all actions for auditing
    AuditLog = new InMemoryAuditLog(),
});

// This throws CommandPolicyException:
await harness.Shell.RunAsync("format", "C: /FS:NTFS");

Emergency Stop

var stop = new EmergencyStop();
// Pass stop.Token to any CancellationToken parameter
// Call stop.Trigger() to cancel all operations at once

Safe Zones, Rate Limiting, and Confirmation

Available through MCP tools or programmatically:

• Safe zones — restrict mouse/keyboard to a window or screen region
• Rate limiting — cap actions per second to prevent runaway automation
• Confirmation gates — require user approval before destructive actions
• Action history — full audit trail of all tool invocations

Monitoring

Background monitors track system changes in real-time, writing events to JSONL files:

Dependency Injection

services.AddSystemHarness(); // default options
// or
services.AddSystemHarness(new HarnessOptions
{
    CommandPolicy = CommandPolicy.CreateDefault(),
});

// Inject IHarness or individual services:
public class MyService(IShell shell, IScreen screen, IOcr ocr) { }

Platform Factory

// Auto-detect platform at runtime
using var harness = HarnessFactory.Create();

Architecture

SystemHarness.Core              Interfaces + models (zero platform dependencies)
  |
  +-- SystemHarness.Windows     Win32/DXGI/SendInput/FlaUI (Windows implementation)
  +-- SystemHarness.Linux       X11/Wayland (planned)
  +-- SystemHarness.Mac         AppKit/AppleScript (planned)
  |
  +-- SystemHarness.Apps.Office OpenXML/OWPML document processing
  +-- SystemHarness.Apps.Email  IMAP/SMTP via MailKit
  +-- SystemHarness.Apps.Browser Playwright-based web automation
  |
  +-- SystemHarness.Mcp         MCP server (3 tools, 174 commands)

IHarness Services (15 interfaces)

Windows Implementation Details

Design Principles

• Zero AI dependency — no LLM SDKs, no model opinions, no agent loops
• AI-ready — Screenshot returns Base64 for vision APIs; ShellResult is structured
• Layer 1 first — programmatic control is always preferred; vision+action is the fallback
• One interface, multiple backends — same IHarness across all platforms
• Async-first — every operation is Task-based
• No magic — thin wrappers over OS primitives, not a framework
• Safety built-in — command policy, audit logging, emergency stop, safe zones

Roadmap

• Core interfaces and models (15 services)
• Windows Layer 1: Shell, Process, FileSystem, Window, Clipboard, Display, SystemInfo
• Windows Layer 2: Screen (DXGI+GDI), Mouse, Keyboard, OCR, UIAutomation
• Template matching (NCC-based image search)
• Compound facades: vision_click_text, app_open/close, report_get_screen
• Smart waiting: vision_wait_text, ui_wait_element, window_wait, vision_wait_change
• Action verification: vision_click_and_verify, vision_type_and_verify
• Background monitors: file, process, window, clipboard, screen, dialog
• Safety: EmergencyStop, safe zones, rate limiting, confirmation gates
• Session management: save, compare, bookmark
• MCP server with 174 commands (3-tool dispatch architecture)
• Office document processing (Word, Excel, PowerPoint, HWP)
• DPI-aware coordinates, Unicode support, cursor overlay
• NuGet packaging with SourceLink
• Linux implementation (X11/Wayland)
• macOS implementation (AppKit/AppleScript)

License

MIT