Agent API And MCP Setup

Fraime.it can expose open projects to local agents through a local bridge and MCP helper. The app stays in control of project files, preview, undo, save, and export while agents propose structured edits.

When to Use Agents

Use the Agent API when you want help with structured timeline work:

• Inspect open projects and timeline state
• Create a working copy before broad changes
• Add or update annotations, subtitles, overlays, and visual assets
• Suggest trims, filler-word cuts, or speed-ups
• Preview an edit plan before applying it
• Export a project after review

Do not use agents as the final privacy reviewer. Always manually check redactions and exported videos.

Setup From the App

Open Fraime.it and use the Agent API controls in the app settings or menu.

• Install for Claude Code and Install for Codex safely merge the bundled helper into each client's documented user configuration after one-time Home folder access. Install for Cursor opens Cursor's official one-click MCP deeplink for confirmation.
• Copy MCP Config gives the configuration snippet for other compatible local agent clients.
• Copy Helper Path gives the local helper executable path.
• Keep the Fraime.it app open while agents inspect or edit projects.
• Open the project you want the agent to work with before asking for project-specific actions.

If an agent cannot find the project, confirm the project window is open and the local bridge is running.

Working Copies

Use working copies for experiments, aggressive trims, launch clips, and alternate exports.

Agent-created working copies use a visible .agent naming convention so the original recording stays separate. Keep the original project as source material and apply broad agent edits to a copy.

Review Before Apply

Agents can dry-run edits and describe what would change before applying the plan. Review the summary carefully, especially changes that affect:

• Project timing and clip ranges
• Redactions and text replacements
• Subtitle timing
• Export settings
• Large sections of the timeline

Apply only the edits you understand. If the plan is too broad, ask for a smaller one, such as only title cards, only filler cuts, or only overlay suggestions.

Edit Plan Operations

An edit plan is a list of typed operations, applied in order, where each one sees the result of the previous ones:

{ "version": 2, "operations": [{ "type": "addChapter", "time": 12.5, "title": "Setup" }] }

Version 2 is strict: each operation accepts only its documented keys, IDs are required for updates and deletes, numeric bounds and enum values are enforced, and unknown or irrelevant keys fail instead of being ignored.

Rather than guessing operation names, agents call fraimeit_describe_operations to read the catalog. With no arguments it returns the index of every operation grouped into categories such as clips, annotations, zoom, privacy, and audio. Passing category, operations, or search narrows it and expands each entry to its required and optional parameters, the accepted values for constrained fields, and notes about how the operation behaves.

The catalog is generated from the same types the bridge validates against, so it cannot drift from what the app actually accepts.

Two behaviors are worth knowing before an agent edits timing on your behalf. Trimming is not rippling: updateClip moves one clip's edge and leaves a gap, so the project keeps its length and downstream chapters, zooms, and annotations stay where they were. deleteClip with mode: "ripple" is the operation that closes a gap, and it retimes chapters, zooms, annotations, cursor data, and agent beats along with the footage. Comparing the before and after durations in a preview shows which one you are about to get.

Available Operation Types

The bridge is designed around structured project operations, including:

• Listing open projects
• Opening projects
• Reading project summaries and timeline data
• Creating working copies or duplicates
• Describing the edit-plan operation catalog
• Previewing edit plans
• Applying edit plans
• Undoing and redoing edits
• Saving projects
• Capturing frame snapshots
• Exporting projects
• Running transcription, OCR redaction analysis, smart trim, and filler-word analysis
• Importing image assets into the project and placing them on the canvas

fraimeit_get_frame_snapshot renders one fully composited frame from the current edited timeline directly to PNG, including zooms, annotations, spotlights, overlays, backgrounds, cursor styling, redactions, and subtitles. Agents can therefore preview and refine visual edits without repeatedly exporting the video. The MCP helper stages the render inside Fraime.it's sandbox and copies it to the requested outputPath, so agent workspaces do not need to be granted directly to the app.

• Renaming or deleting project packages
• Starting, pausing, resuming, and stopping a recording from Claude Code
• Optionally installing Browser Automation, then launching an isolated Chrome session through Puppeteer
• Staging a browser window and binding its viewport for agent-driven demos
• Declaring live demo beats that become synthetic cursor movement, chapters, and zoom framing

The recording window remains responsible for macOS permission flows, but it does not need to stay in front. When an elicitation-capable MCP client such as Claude Code calls fraimeit_start_recording, the helper lists current sources and displays a compact sequence for capture quality, audio/camera toggles, conditional device selection, and automatic effects. The helper applies the accepted values and starts in paused agent-paced mode so planning latency is never captured.

The typed fraimeit_list_capture_sources and fraimeit_configure_recording tools remain available to clients that need lower-level automation. The configuration result echoes the applied state and canStartRecording. Clients without MCP elicitation support must explicitly configure and confirm settings before fraimeit_start_recording.

Display IDs are reusable. Window IDs are ephemeral, so list sources again before configuring a window in a later session. Use the literal device ID default to select the system-default microphone or camera. Supported typed settings are 30/60 FPS, Retina capture, countdown, system audio, microphone/device, webcam/device, Auto Zoom, and Auto Spotlight. Rectangular area capture can be selected interactively or configured with a display ID and display-local region.

During any active capture, fraimeit_pause_recording and fraimeit_resume_recording control the same continuous recording. While paused, Fraime.it omits screen frames, system audio, microphone audio, webcam frames, and telemetry rather than storing a gap. The elapsed recording timer also stops. Stopping while paused finalizes normally.

For owned-browser demos, beats may be marked while paused. Each visible fraimeit_browser_* action atomically resumes capture immediately before dispatch and pauses it again after the action settles.

Agent-Driven Browser Demos

Browser control is optional. In Recording Settings → Advanced → Browser Automation, choose Download & Install, then enable it. Fraime.it downloads a pinned, checksum-verified Node runtime for the current Mac only after that explicit action; ordinary recording, editing, and non-browser Agent API tools do not require it.

Once enabled, FraimeMCP launches installed Google Chrome with an isolated temporary profile, then attaches its Puppeteer backend to that owned process. Puppeteer never launches the browser or adds Chrome's --enable-automation flag. This avoids extension and automation banners while keeping the user’s normal Chrome profile, cookies, tabs, and extensions out of the recording. Before each browser action, call fraimeit_mark_beat with the target point, element rectangle, script beat ID, and target duration. Fraime.it keeps the cursor out of captured pixels and reconstructs a smooth synthetic cursor performance from those declarations during finalization. Post-recording pacing restores beats shorter than their target with a frame hold and reports visible-action overruns instead of silently missing the estimate.

Fraime.it bundles a record-fraimeit-demo Agent Skill that orchestrates this workflow from a plain-language idea and URL. Install it for Cursor, Claude Code, or Codex from Recording Settings → Advanced → Agent Demo Skill. The skill creates semantic instructions itself and resolves live elements at runtime; users never need to provide selectors or coordinates.

For Codex, copy the Codex MCP config from Fraime.it into ~/.codex/config.toml; the skill installs to ~/.codex/skills/record-fraimeit-demo/SKILL.md. Restart Codex after installation so it discovers the skill. Older Codex builds may also require skills = true under [features] in the same config file.

Multi-window Demos

Fraime.it can stage exactly two windows with fixed geometry. Use swap when the windows should take turns occupying the same frame, splitHorizontal for a left-right layout, or splitVertical for a top-bottom layout. The size can use the same hd720p, fullHD1080p, portrait720p, maxPortrait9x16, and square1080p presets as single-window staging, or explicit width and height values.

Swap recording captures each real window directly and retargets the live stream between takes while paused, so other windows and notifications cannot cover the recorded source. Split recording still uses an area crop; anything visible inside that region is recorded. Clear split layouts and disable interruptions before starting. Privacy OCR remains a separate opt-in review.

The tool sequence is:

1. Launch or identify both windows, then call fraimeit_stage_layout before recording.
2. For swap, configure fraimeit_configure_recording with sourceKind: "window" and the intended first slot's source ID. For a split, use sourceKind: "area", the returned display ID, and returned display-local region.
3. Start recording in agent-paced mode.
4. For a browser in swap, bind its viewport without windowFrame. For a split, pass that slot's returned region-relative frame as windowFrame.
5. Mark browser beats in viewport space. Mark terminal or IDE beats in region space, relative to the active window for a swap or the full crop for a split.
6. In a swap, activate the first window while paused, then activate again only when changing to the other window's take. Activation verifies the global frame and retargets capture. Splits keep both windows visible, so activation is optional.
7. Wrap each visible terminal or IDE operation with fraimeit_begin_window_action and fraimeit_end_window_action; the pair resumes capture for the work, records its timing span, settles, and pauses again.
8. Stop immediately if either window moves or resizes, or if a split's crop changes. Layout and viewport geometry stay fixed for the whole recording.

The browser-only sequence is:

1. Call fraimeit_browser_status and require installed: true and enabled: true
2. fraimeit_browser_launch
3. Dry-run with fresh fraimeit_browser_snapshot refs and bounded browser actions
4. Match the capture source by owningApplicationPID, then call fraimeit_stage_window with the canonical preset field
5. Configure and start recording
6. fraimeit_bind_viewport
7. While paused, inspect, plan, and call fraimeit_mark_beat; the corresponding fraimeit_browser_* action atomically resumes and pauses capture
8. After the expected state settles, pause again before planning the next beat
9. Stop recording and preserve the source intent with fraimeit_save_demo_script
10. Inspect the timeline, export, and call fraimeit_browser_close

Viewport points use CSS pixels with a top-left origin. Rebind after resizing or changing browser zoom. Snapshot refs are ephemeral and must not be persisted in demo.script.json. FraimeMCP rejects unknown, missing, and incorrectly typed top-level tool arguments before dispatch rather than silently ignoring them.

Visible browser interactions are timestamped directly on the recording clock. After processing, fraimeit_get_timeline exposes those spans as agentEvents. Agent pacing preserves the interactions and generated zoom holds, removes the exact idle intervals between them, and cuts the unused tail after the final hold. Frame snapshots remain useful for checking composition, but agents should never use them to infer edit timing.

The session follows the demo rather than the browser at large. Cross-origin iframe content such as a cookie banner appears in snapshots with page-space coordinates, a tab the demo page opens becomes the active page, an unrelated tab never takes over, and a blocking alert, confirm, or beforeunload dialog is accepted automatically. Locator actions report performed, failed, or ambiguous; ambiguous actions must be verified from a fresh snapshot rather than retried blindly. Click, type, and text selection never auto-scroll an off-screen or scroll-container-clipped target: requiresScroll: true means reveal it explicitly, take a fresh snapshot, and act on the new ref. fraimeit_browser_select_text applies a visible native selection to all text inside a ref or an exact occurrence. Nested-container scrolling can target a snapshot ref and reports verified before/after offsets. durationSeconds makes one scroll call a continuous eased gesture for the requested duration; agents should not approximate a long glide with repeated short calls. The owned Chrome process is reclaimed on close, on termination signals, and by the next helper launch if the previous one was killed.

Typing supports visible text inputs, textareas, editable controls, native select options, and checkbox/radio/switch state. The current browser tool set does not provide hover, drag-and-drop, precise slider movement, arbitrary keyboard shortcuts, file upload, canvas/WebGL coordinate interaction, arbitrary page-copy mutation, or control of Chrome's own UI.

When Claude Code calls fraimeit_stop_recording, Fraime.it waits only for capture finalization, creates durable post-recording.json job metadata, and opens the project. Transcript selection, event import, chapter generation, pacing suggestions, and the timeline save continue asynchronously with progress in the Timeline Assistant. The assistant opens automatically for agent recordings and shows failures instead of leaving an indefinite spinner. Pacing is never applied automatically. Privacy OCR remains a separate, opt-in analysis tool and is not part of this pipeline.

Exact tool names depend on the MCP client, but the workflow should stay review-first: inspect, preview, apply, verify, save or export.

Safe Prompting

Good prompts are specific:

• "Create a 30 second launch clip from this project, but preview the plan first."
• "Add title cards at the major section changes."
• "Suggest filler-word cuts from the transcript without applying them."
• "Find places where a zoom or callout would help explain the UI."
• "Prepare a vertical Short as a working copy."

Avoid vague prompts like "make it better" when you care about timing, privacy, brand tone, or export format.

Troubleshooting

If the MCP client cannot connect, copy the config again from the current app build and verify the helper path. If browser tools are unavailable, call fraimeit_browser_status and follow its reason; install or enable Browser Automation in Recording Settings rather than using shell-based browser control. If project actions fail, make sure the project is open in Fraime.it. If edits apply but the result is not useful, undo in the app or return to the original project/working copy.