Datasynx Shadowing

The SOPs your team never has time to write — observed, generated, anonymized.

by Datasynx AI · ★ Star on GitHub

100% local MCP-native Claude-powered PII-anonymized

Shadowing observes daily workflows like a silent shadow — shell commands, active windows, git commits, file changes — and automatically generates structured Standard Operating Procedures (SOPs) via Claude. Everything stays on the employee's machine: a local SQLite database, no cloud sync, no daemon. Every export is automatically scrubbed of PII.

Quickstart

# 1. Setup (create DB + config)
shadowing init

# 2. Manual mode: start task → generate SOP
shadowing start

# 3. Automatic mode: observe workflow → auto-generate SOPs
shadowing observe --auto-sop

# 4. View SOPs
shadowing list
shadowing show <sop-id>

# 5. Start the web dashboard
shadowing ui

# 6. Export with anonymization
shadowing export --all

Installation

npm install -g @datasynx/agentic-ai-shadowing

Requirements

• Node.js ≥ 20 (Linux, macOS, or Windows)
• ANTHROPIC_API_KEY environment variable (for SOP generation)
• @datasynx/agentic-ai-cartography (optional, for infrastructure context)

export ANTHROPIC_API_KEY=sk-ant-...

Architecture

CLI (Commander.js — 27 commands)
  └─ shadowing init / start / observe / list / export / ui / ...
      ├─ TaskManager      Task lifecycle (start → pause → resume → complete)
      ├─ Observer         Heartbeat-based workflow capture
      │   ├─ WindowDetector   xdotool (Linux) / osascript (macOS) / P/Invoke (Win)
      │   ├─ ShellHistory     Zsh / Bash / Fish / PowerShell parser
      │   └─ Git + File       Commit tracking + file changes
      ├─ SessionAnalyzer  Silence clustering → task detection (LLM)
      ├─ SOPGenerator     Claude API → structured SOPs + tags
      ├─ Anonymizer       PII redaction (8+ patterns)
      ├─ Exporter         Markdown + manifest.json (atomic operations)
      ├─ Metrics          Consistency · Maturity · Freshness · Quality
      ├─ PrivacyManager   Consent + exclusion rules + degradation
      └─ ShadowingDB      SQLite WAL (11 tables, constraints, indices)

  Integrations:
      ├─ UIServer         REST API (17 endpoints) + HTML dashboard
      ├─ MCPServer        Model Context Protocol (18 tools, stdio)
      ├─ HookHandler      Claude Code event processing
      └─ Cartography      JGF graph import from agentic-ai-cartography

Task Management

SOP Management

Automatic Observation

Cross-platform capture

Metrics & Export

Export structure

exports/export_2026-02-24T14-30-00/
├── manifest.json          Metadata, tags, metrics summary
└── sops/
    ├── sop_001.md         Anonymized SOP (Markdown)
    └── sop_002.md

Integration

Quality Metrics

Consistency Score

max(0, 100 - CV * 2) — based on the coefficient of variation of execution durations.

Maturity Score (weighted)

Overall Quality Score

consistency * 0.35 + maturity * 0.35 + freshness * 0.30

Privacy & Anonymization

Redact-on-capture

PII and secrets are redacted before observation data is written to SQLite — window titles, shell commands, file paths, task titles/descriptions, and task notes never reach disk in raw form (anonymization.redact_on_capture, default true). Export-time anonymization runs as a second layer. Older databases can be cleaned retroactively with the idempotent shadowing scrub command.

Redaction patterns

Developer-secret detection is never configurable off. The entropy fallback skips git commit SHAs, UUIDs, and ordinary identifiers; the whole pipeline is idempotent. Custom replacements via config.anonymization.custom_replacements.

REST API

The web dashboard (shadowing ui) serves a dark-theme single-page app and 17 REST endpoints on port 3847. The API requires Bearer-token auth, is rate-limited, and rejects cross-origin requests by default (no CORS wildcard — allowlist via ui_allowed_origins).

Configuration

Config lives at ~/.datasynx/shadowing/config.json.

{
  "version": "1.0.0",
  "language": "en",
  "polling_interval_minutes": 15,
  "editor": "code",
  "ui_port": 3847,
  "ui_allowed_origins": [],
  "cartography_graph_path": null,
  "anonymization": {
    "redact_emails": true,
    "redact_ips": true,
    "redact_urls": true,
    "redact_phone_numbers": true,
    "redact_file_paths": true,
    "redact_high_entropy": true,
    "redact_on_capture": true,
    "custom_replacements": {}
  },
  "sop_generation": {
    "model": "claude-sonnet-4-20250514",
    "max_tokens": 4096,
    "temperature": 0.3,
    "include_cartography_context": true,
    "auto_generate_tags": true,
    "sop_language": "en",
    "base_url": null,
    "api_key_env": "ANTHROPIC_API_KEY",
    "use_structured_output": true
  }
}

Enterprise deployment

The only network egress is SOP generation — and it is fully configurable: base_url routes traffic through an internal LLM gateway or a local Anthropic-compatible model server (no-egress deployment), api_key_env lets gateways keep their own credential env-var naming, and use_structured_output: false falls back to plain-text parsing for gateways without tool support.

Datasynx Agentic Suite

Shadowing is one of three composable, local-first, MCP-native tools. They work standalone, but compound when combined: Cartography maps the systems, Shadowing documents the work, CRM runs the customer relationships.

FAQ