dsh-video-lens

项目详情摘要

dsh-video-lens Video understanding for DeepSeek Harness — give text-only agents eyes and ears on video.** A DeepSeek Harness (DSH) plugin that lets text-only LLM agents understand local video files. It provides two tools: Tool | What it does | `video_probe` | Cheap, instant metadata via `ffprobe`: container, duration, resolution, fps, codecs, audio tracks, subtitles. | `video_analyze` | Content understanding: **scene-change-aware frame sampling** (`ffmpeg scdet`), **optional ASR transcript** (speech with timestamps), fused with any OpenAI-compatible vision model into structured evidence JSON. | `video_ask` | **Time-anchored Q&A**: parses explicit time references ("at 3:20", "第2分钟") or locates relevant speech via transcript keyword matching, re-samples frames from the matched windows, and answers with grounded evidence (answer + confidence + supporting timestamps). | v0.3.1. The plugin never locks you into a provider: vision and ASR are both OpenAI-compatible endpoints configured via `baseUrl` + `model` + key env var. How it works Scene changes are detected with ffmpeg's `scdet` filter (ffmpeg ≥ 6.0). Videos without detectable cuts fall back to uniform midpoint sampling. ASR is **strictly additive**: if `asrApiKeyEnv` is unset or the provider fails, the visual analysis still completes and `transcript` is `null`. All media work is delegated to `ffmpeg`/`ffprobe` on `PATH` — no native decoding in the agent. Install Prerequisites: Node.js ≥ 20, `ffmpeg` ≥ 6.0 (recommended) with `ffprobe` on PATH (`brew install ffmpeg` / `apt install ffmpeg`). Option A — npm (recommended) Option B — from source (development) Clone the repo, then mount it into your DSH profile via a local link: Either way, register the bundle in your profile's `package.json` — **this exact block is the full p