Getting Started

Requirements

• Rust and Cargo
• ffmpeg and ffprobe on PATH
• Optional: LM Studio or another OpenAI-compatible local vision endpoint at http://localhost:1234/v1
• Optional: Magenta RealTime setup for real model-backed music generation
• Node.js and pnpm for the desktop app and docs site

The core pipeline runs locally. With local media tools, a downloaded Magenta model, and a local vision endpoint or fallback path, you do not need paid music generation API calls.

Build The CLI

cargo build

Run the development binary directly:

target/debug/vidbgm --help

Or install it as a local vidbgm command on your PATH:

cargo install --path .

The CLI is currently installed from source. It is not published to crates.io or Homebrew yet.

Check Magenta Readiness

target/debug/vidbgm magenta-status --magenta-model mrt2_small

If the model and resources are missing, run setup explicitly:

target/debug/vidbgm magenta-setup --magenta-model mrt2_small --yes

Analyze A Video

The examples below use the CLI directly. For the full command matrix, option groups, smoke checks, and evaluation workflow, see the CLI Guide.

target/debug/vidbgm analyze \
  --video sample-video.mov \
  --prompt "energetic cinematic electronic music for a first-person cycling video, steady rhythm, uplifting but not distracting" \
  --frame-interval-seconds 30 \
  --vision-timeout-seconds 120 \
  --out output/full-analysis.json \
  --workdir output/full-sample

Generate Music

target/debug/vidbgm generate \
  --video sample-video.mov \
  --prompt "energetic cinematic electronic music for a first-person cycling video, steady rhythm, uplifting but not distracting" \
  --frame-interval-seconds 30 \
  --vision-timeout-seconds 120 \
  --out-audio output/full-music.wav \
  --workdir output/full-generate \
  --prompt-update-mode continuous

Render A Mixed Video

target/debug/vidbgm render \
  --video sample-video.mov \
  --prompt "energetic cinematic electronic music for a first-person cycling video, steady rhythm, uplifting but not distracting" \
  --duration 60 \
  --frame-interval-seconds 15 \
  --vision-timeout-seconds 120 \
  --audio-mode mix \
  --music-volume-db -3 \
  --original-volume-db -18 \
  --out-video output/vertical-slice-mix.mov \
  --workdir output/render-mix