dji360 MCP: Command-Driven, Face-Free 360 Camera Edits
An MCP server that turns raw Osmo 360 files into gyro-stabilised, level, 4K reframed edits with no people's faces, entirely by command.
Reverse-engineered the OSV container and IMU layout, calibrated IMU against vision, face scanning with manual review, planned shots with re-aiming, GPU encodes and detached jobs; first deliverable a 2.5-minute stabilised ride edit.
- OSV container and IMU format reverse-engineered
- IMU-to-vision calibration per frame step
- Face-free output with manual review of flagged shots
- Windows stdio-MCP deadlock root-caused and fixed
I wanted edits from a 360 camera without touching the vendor's desktop app, and with no bystanders' faces in the result. The vendor offers no command-line export, so the project reverse-engineered the file format: two 3840-square 10-bit fisheye streams, 200-degree lenses 180 degrees apart, and per-frame quaternions and accelerometer samples inside a protobuf metadata track, with the body-to-lens rotation and sign conventions worked out by calibrating IMU steps against visual motion.
The MCP server exposes the workflow as tools: list clips, contact sheets, previews, face scanning, shot planning, per-shot edits (yaw, pitch, field of view), rendering, face re-checks, shot sheets, assembly and video checks, with long jobs detached and tracked in status files. Face detection flags leaves and truck grilles as well as real bystanders, so crop sheets are always eyeballed and shots dominated by the rider's helmet are dropped.
Performance work mattered: in-process threaded decode is five times faster than piping 44 MB frames, and the encoder needs constant-QP mode to avoid a bitrate cap at 4K. A Windows stdio-MCP deadlock (lazy heavy imports and inherited stdin pipes) was diagnosed and fixed. The first deliverables are a stabilised ride edit of 29 shots and an MCP-only rebuild with 32 shots.
How it works
- list_clips and contact_sheet survey the footage
- scan_faces flags people per clip
- plan proposes shots; edit_shot re-aims yaw, pitch and FOV
- render stabilises and reframes each shot on the GPU
- check_faces and assemble produce the final 4K file
Screens
