dsh-mac-vision
Personal / macOS AI Plugin
A native macOS vision plugin that gives text-only DeepSeek Harness models local OCR and visual evidence.
In brief
A DeepSeek Harness plugin for inspecting local images, clipboard content, screens, and application windows through on-device Apple Vision.
Built on Apple Vision so source images do not need to be uploaded to a vision-model service.
Text-only agent models can reason about local workflows but cannot directly inspect a screenshot, clipboard image, or application window. Sending that material to a separate vision-model service also introduces an avoidable privacy boundary.
dsh-mac-vision adds a native visual evidence layer to DeepSeek Harness. One plugin installs the capture and Apple Vision tools together with model-facing guidance for deciding when to use them and how to communicate uncertain results.
A useful local vision tool has to coordinate several concerns at once: macOS capture permissions, native OCR, small-text recovery, structured tool output, cancellation and timeout behavior, and careful separation between direct evidence and model interpretation.
What it includes
- 01On-device OCR for files, clipboard images, screens, and selected windows
- 02Structured evidence with confidence, coordinates, alternatives, and review state
- 03Automatic crop-and-upscale refinement for small text and notation
- 04Bundled Skill for separating observed, inferred, and uncertain results
How it works
- 01Select a local source
Accept an image file, clipboard image, full screen, frontmost application, or selected visible window.
- 02Capture on macOS
Resolve the requested source locally and respect the operating system's Screen Recording permission boundary where required.
- 03Run native analysis
Use Apple Vision for OCR and optional classification, barcode, saliency, people, or animal detection.
- 04Refine difficult regions
Crop and upscale small text or notation, then run a second OCR pass while retaining conflicting candidates.
- 05Return evidence to the model
Provide structured observations and guide the agent to distinguish direct evidence, interpretation, and uncertainty.
Build notes
Keep image analysis on device
local by default- Context
- Screenshots and local files may contain private workspace, account, or application data.
- Approach
- Use the macOS Vision framework for OCR and supported detections instead of uploading source images to a vision-model service.
- What it enables
- The visual source remains on the Mac while the agent receives structured evidence it can reason over.
Bundle execution tools and usage policy
agent integration- Context
- A native tool can return evidence, but a model still needs guidance about when to call it and how to report ambiguous results.
- Approach
- Ship the vision tools and their model-facing Skill as one installable Harness plugin.
- What it enables
- Users install one package, while the runtime loads both execution capability and consistent evidence-reporting behavior.
Preserve uncertainty in structured output
evidence design- Context
- Small text, formulas, and visually ambiguous regions can produce plausible but conflicting OCR candidates.
- Approach
- Return confidence, coordinates, alternatives, refinement state, and explicit distinctions between completed and unrun detectors.
- What it enables
- The agent can identify what was directly observed, what was inferred, and what still requires review.
Compatibility
- The plugin requires macOS, Node.js 22 or later, DeepSeek Harness, and a local Swift compiler.
- Screen and application-window sources may require Screen Recording permission; local image files do not.
- Small text, formulas, and ambiguous regions can still require a targeted second inspection and human review.