Compare commits

...
Author SHA1 Message Date
Tanish Manku e9ea2a938c Scene Builder: T-pose in clip gaps + drag-preview card for animations
- Gaps show the default pose: a disabled three.js action leaves the skeleton
  frozen on its last frame, so evaluateAt now resets any character with no clip
  under the playhead to its bind (T) pose via Skeleton.pose() (skeletons cached
  per character). Fixes blank space after a clip holding the clip's last pose.
- Animation drags now show the shared DragGhost tilt card with the clip
  thumbnail: the drawer's native-DnD handlers drive dragItem/dragPosition/
  assetDraggingUnder and suppress the browser's default drag image, matching
  object/character pickup motion.
2026-07-09 00:59:24 -07:00
Tanish Manku dc5608613d webapp: stop dev API host override pointing at localhost:12345
The DEV block set the API host to window.location.origin (so /v1 calls hit the
Vite proxy → api.storyteller.ai) and then immediately called setDevelopment(),
which overrode it to http://localhost:12345 — a local backend that isn't
running. Every media-token resolution failed, so getMediaUrlByToken returned ''
and GLB fetches fell back to localhost, serving nothing. The inline comment
already described this line as disabled; it just wasn't commented out. Now it
is, so local dev resolves media through the proxy as intended.
2026-07-09 00:59:24 -07:00
Tanish Manku 1a90bd54c2 Scene Builder: one clip row per character + overlap guard
Collapse the per-character clips onto a single timeline row (playback is
sequential — one clip at a time), replacing stacked lanes. Stacking only earned
its keep for additive layering, which is out of scope.

- TimelineController: add/move route through resolveFreeStart (snap to nearest
  non-overlapping slot); resizeClipLane and resolveClipDuration clamp against
  nextStartAfter so a strip never overruns the following clip. A character's
  strips stay a non-overlapping sequence, so evaluateAt has at most one active
  clip at any playhead.
- TimelineClipRow: renders all of a character's strips on one shared row
  (shown even when empty, as a drop hint); per-strip move/trim/loop and an
  inline x to remove.
- TimelineEditor: one TimelineClipRow per character instead of one per lane.
2026-07-08 00:04:13 -07:00
Tanish Manku 2920a189f0 Scene Builder: Mixamo clip-lane timeline UI + real clip length (phase C)
Stacked clip-strip rendering on the timeline with move/trim/loop/delete, drag-
and-drop from the Animations drawer onto a character's lane, and backfill of the
clip's real length into the strip once its GLB loads.

- TimelineClipRow: one stacked strip per clip lane under its character's
  keyframe row; body drags to move, right edge trims length, loop chip toggles
  repeat, trash removes. Clip edits ride the timeline Save/Cancel session (no
  per-op undo, matching keyframes).
- TimelineEditor: each character row is a drop target for clip drags; drop time
  is read from the ruler rect. Non-character rows ignore clip drags.
- AnimationsDrawer clips are now draggable (ANIMATION_CLIP_MIME) as well as
  click-to-add.
- ClipStrip.autoDuration: a fresh drop seeds a placeholder width, then
  CharacterAnimationManager.resolveClipDuration adopts the loaded clip's true
  length (re-clamping start); a user trim clears the flag so reload never
  clobbers a hand-set duration.
- clipBindsToCharacter warns when a clip resolves 0 tracks on the character —
  the diagnostic for whether retargeting is needed.
2026-07-07 23:51:37 -07:00
Tanish Manku 7f5e82cbd9 Scene Builder: Mixamo clip playback engine + drawer (phase B, spike)
Deterministic, playhead-driven skeletal-animation playback for character clip
lanes, plus a minimal click-to-add drawer so retargeting/binding can be verified
in the viewport before the full drag-drop / stacked-strip UI is built.

- CharacterAnimationManager: one AnimationMixer per character; each clip lane is
  an AnimationAction posed at an absolute time from the playhead + mixer.update(0)
  (scrub / play / record all pose identically). syncClipLanes() reconciles the
  runtime on every lane change; a load token guards superseded async clip loads.
- Scene.loadRawGlb(media_id): raw GLTF (scene + animations) with no scene-add /
  placeholder — sources clips for binding. Clips bind by node name (shared
  mixamorig:* naming); SkeletonUtils.retargetClip is the documented fallback.
- TimelineController: getPlayhead(); evaluate() poses characters; sync on
  add/move/remove clip lane + load/cancel/clear.
- actions: addClipToCharacter / removeClipLane.
- comps/AnimationsDrawer: right-docked, gated on a selected character; click a
  curated clip to drop it as a lane at the playhead.

Compiles clean (tsc exit 0); playback is NOT yet runtime-verified.
2026-07-07 15:42:24 -07:00
Tanish Manku 5c2fbbc9f3 Scene Builder: animation clip-lane data model (Mixamo drawer, phase A)
Foundation for placing Mixamo skeletal-animation clips on the timeline as
stacked lanes under character objects, alongside the existing keyframe tracks.

- ClipStrip / ClipLane types; TimelineData gains a `clipLanes` array (only the
  clip reference + placement is serialized — the AnimationClip is resolved and
  retargeted from sourceMediaId at runtime).
- TimelineController: addClipLane / moveClipLane / removeClipLane / clipLanesFor,
  create()/loadTimeline() init + backfill clipLanes, emitChanged carries them.
- TimelineChangedEvent → store mirror (timelineClipLanes) via EngineStoreBridge.
- Persistence rides the existing timeline save/load (getTimeline/loadTimeline).
2026-07-07 15:28:50 -07:00
Tanish Manku 11c7aa979d Scene Builder fixes: record-mode camera lock, Blender default framing, auto-keyframe
- Record mode is now fully immutable: gate MouseControls mouse-look + focus()
  on the viewport lock (new MouseControlsDeps.isViewportLocked → cameraController
  .locked), matching the existing FreeCam/tick gates.
- Default main (viewport) camera pulled back + elevated to a Blender-style 3/4
  view so a blank scene shows the full render-camera frustum.
- Auto-keyframe: mutating an already-keyframed object (gizmo drag-end or panel
  commit) replaces/creates a keyframe at the playhead via
  TimelineController.autoKeyIfTracked.
- loadTimeline backfills fps/duration/tracks defensively (legacy save safety).
2026-07-07 15:23:50 -07:00
bflatastic 26aa3d3839 3D scene editor: exact camera framing, timeline UX, outliner + chrome redesign
Camera framing
- Replace the full-canvas Letterbox with CameraFrame: an inset, aspect-correct
  frame (dimmed surround, rule-of-thirds, focal label) shown only in camera
  view and record mode — the free viewport no longer shows mattes
- The frame shows the EXACT render framing: CameraController.setCameraFrameRect
  offsets the viewport projection via camera.setViewOffset; snapshots and the
  timeline recorder clear/reapply the offset so captures stay full-bleed
- recordTimeline now encodes at the render camera's resolution/aspect with the
  grid hidden (was grabbing the raw viewport canvas)
- Frame avoids the bottom UI; expanded-timeline height is measured live

Aspect ratio
- Default is now 16:9 via DEFAULT_CAMERA_ASPECT_RATIO (store, controller,
  scene reset, newScene); removed the model cold-sync that forced Square
- New top-right aspect popover (AspectRatioMenu rewritten on PopoverMenu with
  proportional AspectRatioIcon, now exported from ui-promptbox)

Timeline
- Expanded editor shows every scene object as a track row (no selection
  needed), capped at ~3.5 rows then scrollable (scrollbar hidden so lanes
  stay width-aligned with the ruler)
- Restyled per design: dark lanes, keyframe diamonds + connecting line,
  playhead diamond handle, select-none everywhere
- Scrubbing: drag the ruler to scrub smoothly, following the pointer 1:1;
  times quantize to the timeline's frame grid (30fps) — transport readout
  shows frame timecode (m:ss:ff); keyframe drags quantize the same way
- Fixed playhead/keyframe misalignment (scrollbar shrank lanes vs ruler;
  label indent must be padding, not margin, to keep the w-32 column)
- Delete/Backspace removes the selected keyframe (capture-phase, so the
  engine's delete-scene-object keybind doesn't fire); clicking away
  deselects it
- Motion (easing) popover opens from the curve chip BETWEEN keyframes (new
  timelineEasingKeyframeId store field), centered above the chip in lane
  coordinates, backdrop-blurred, dismissed on outside click; selecting a
  keyframe no longer opens it
- Camera-track playback/scrubbing is now visible in camera view: the
  timeline mirrors the animated ::CAM:: pose onto the viewport camera

Record mode
- Viewport camera fully locked: mouse-look blocked via isMovable(), right-drag
  pan and wheel zoom gated in useFreeCam (build mode unaffected)

Chrome
- Outliner restyled as the "Scene" panel (+ Add via shared openAssetModal,
  two-line rows with friendly kind labels, blue selected card), open by
  default, vertically centered, fixed 45vh height independent of aspect ratio
- Build/Record pill fully rounded (new TabSelector listClassName prop)
- CameraStatusPill moved to the top-right camera cluster, height-matched
  with the aspect popover
- ExitCameraView is a regular labeled button docked right of the timeline bar
2026-07-07 23:15:54 +07:00
Tanish Manku 06315490b7 Scene Builder launch re-scope: drop promptbox, lightbox handoff, editable timeline duration
- Remove SceneBuilderPromptBox (MCP not ready). The animation timeline is the
  sole build-mode bottom UI: collapsed bar by default, expanded editor via
  chevron; an empty timeline is auto-created on init; auto-expands on load when
  the scene has keyframes or an imported THREE animation.
- Editable max timeline duration: TimelineController.setDuration (clamp 1-60s,
  covers 5-30) + click-to-edit DurationLabel in the bar/ruler.
- Capture/Record output → review modal (CompletionModal) that appears FIRST
  with a local preview (offline-safe), then uploads (images auto, videos manual)
  with status; on success opens the app Lightbox on the token (all destinations
  for the kind), on failure shows Retry.
- Adapter: add openMediaLightbox(token, kind); keep uploadMedia; remove
  openImageInEditor/openVideoInEditor (Lightbox owns destination routing). Host
  renders <Lightbox> and resolves cdnUrl via MediaFilesApi.

Compile-verified (tsc -b clean for pagescene + changed webapp files).
2026-07-06 23:54:47 -07:00
Tanish Manku c15d8bd1ec Scene Builder checkpoint: camera view fixes, record/capture + completion modal, save/load + undo persistence
Builds on the prior Scene Builder commit. Compile-verified (tsc -b clean for
pagescene + changed webapp files); see frontend/libs/components/pagescene/CLAUDE.md.

- Camera view: fixed record→build engine-teardown wipe (PreviewEngineCamera /
  cam-view canvas must stay mounted — hide via CSS, never unmount); Esc +
  viewport double-click exit; double-click frustum to enter; robust enter/exit
  pose; frustum pick-proxy hitbox; CameraStatusPill (Viewport vs render cam).
- Build promptbox: dedicated SceneBuilderPromptBox (MCP stub) replaced
  PromptBox3D; buildMode/toggle removed.
- Record mode: Capture (still) + Record (timeline→mp4 via mediabunny),
  RenderOverlay progress, producedArtifact cache, CompletionModal
  (preview/playback + Delete/Upload/Edit) + adapter handoff to 2D/create-video/
  video-editor (lazy, data-URL image handoff; best-effort uploads).
- Persistence: timeline + render-camera transform in scene JSON (save/load +
  unmount cache); ::CAM:: recreated on load after cameras restore.
- Undo: timeline transactional (SaveTimelineAction); camera moves via TransformAction.
2026-07-06 23:26:18 -07:00
Tanish Manku 40ec291395 Scene Builder: mode pill, prompted mode, timeline, camera split, record mode
Adds the Scene Builder feature set to the pagescene 3D editor (+ the shared
ui-promptbox). All UI-level; compile-verified only (tsc -b), not yet runtime
tested — see frontend/libs/components/pagescene/CLAUDE.md for the full spec and
per-slice "unverified" list.

- Build/Record mode pill (SceneModePill, TabSelector) + store `sceneMode`.
- Build sub-modes: Manual (current) vs Prompted. Prompted prompt bar shows a
  textbox + add-animation-timeline + prompt history + Update (Update is a stub
  pending the MCP scene-descriptor backend). Toggle via ButtonIconSelect.
- Animation timeline (in-memory): TimelineController + keyframe model +
  cubic-bezier interpolation ticked in the render loop; collapsed bar +
  expanded multi-track editor + Motion easing popover (draggable bezier);
  Save is an undoable action, Cancel reverts.
- Camera split (single camera): render camera is a selectable wireframe-frustum
  placeholder ("::CAM::") in the scene; look-through camera view via
  double-click / outliner view-from-camera icon; ExitCameraView button.
- Record mode: read-only playback through the locked render camera; build
  chrome hidden; Capture (still) works, Record (video) is a stub.
2026-07-03 05:24:48 -07:00
Brandon Thomas 3a2e3b30ec debug logs #2 (#1723) 2026-07-02 19:36:48 -04:00
Brandon Thomas 82c0a7a4d9 debug logs: new field, new queries, new APIs, new dashbord UX (#1722) 2026-07-02 16:43:00 -04:00
120 changed files with 7050 additions and 1138 deletions
@@ -0,0 +1,12 @@
{
"db_name": "MySQL",
"query": "\nINSERT INTO debug_logs\nSET\n event_token = ?,\n debug_log_type = ?,\n maybe_log_level = ?,\n maybe_creator_user_token = ?,\n maybe_ip_address = ?,\n maybe_url = ?,\n message = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 7
},
"nullable": []
},
"hash": "32a8d1cfec89ba2056463a33ddc501a8a99283c7c00982a7fa79f7ff382c9b22"
}
@@ -1,80 +0,0 @@
{
"db_name": "MySQL",
"query": "\nSELECT\n id as `id: u64`,\n event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,\n debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,\n maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,\n message,\n created_at\nFROM debug_logs\nWHERE event_token = ?\nORDER BY id ASC\nLIMIT ?\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id: u64",
"type_info": {
"type": "LongLong",
"flags": "NOT_NULL | PRIMARY_KEY | UNSIGNED | AUTO_INCREMENT",
"char_set": 63,
"max_size": 20
}
},
{
"ordinal": 1,
"name": "event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 2,
"name": "debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 96
}
},
{
"ordinal": 3,
"name": "maybe_creator_user_token: tokens::tokens::users::UserToken",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 4,
"name": "message",
"type_info": {
"type": "Blob",
"flags": "NOT_NULL | BLOB | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 67108860
}
},
{
"ordinal": 5,
"name": "created_at",
"type_info": {
"type": "Datetime",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY",
"char_set": 63,
"max_size": 19
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false,
false,
true,
false,
false
]
},
"hash": "a62b88fec40a11952cc199383090ea338f97091e8bfc6b68aad67416685889af"
}
@@ -1,12 +0,0 @@
{
"db_name": "MySQL",
"query": "\nINSERT INTO debug_logs\nSET\n event_token = ?,\n debug_log_type = ?,\n maybe_creator_user_token = ?,\n message = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "a9745ecf60b564ab096ee1dbe124221b5f288a4ae38e788e19bbd359db0b580f"
}
@@ -0,0 +1,146 @@
{
"db_name": "MySQL",
"query": "\nSELECT\n d.id as `id: u64`,\n d.event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,\n d.debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,\n d.maybe_log_level as `maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel`,\n d.maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,\n d.maybe_ip_address,\n d.maybe_url,\n d.message,\n d.created_at,\n u.display_name as `maybe_user_display_name?`,\n u.username as `maybe_user_username?`,\n u.email_gravatar_hash as `maybe_user_gravatar_hash?`\nFROM debug_logs d\nLEFT OUTER JOIN users u\n ON u.token = d.maybe_creator_user_token\nWHERE d.maybe_creator_user_token = ?\n AND d.id < ?\nORDER BY d.id DESC\nLIMIT ?\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id: u64",
"type_info": {
"type": "LongLong",
"flags": "NOT_NULL | PRIMARY_KEY | UNSIGNED | AUTO_INCREMENT",
"char_set": 63,
"max_size": 20
}
},
{
"ordinal": 1,
"name": "event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 2,
"name": "debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 96
}
},
{
"ordinal": 3,
"name": "maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 64
}
},
{
"ordinal": 4,
"name": "maybe_creator_user_token: tokens::tokens::users::UserToken",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 5,
"name": "maybe_ip_address",
"type_info": {
"type": "VarString",
"flags": "BINARY",
"char_set": 224,
"max_size": 160
}
},
{
"ordinal": 6,
"name": "maybe_url",
"type_info": {
"type": "VarString",
"flags": "BINARY",
"char_set": 224,
"max_size": 1020
}
},
{
"ordinal": 7,
"name": "message",
"type_info": {
"type": "Blob",
"flags": "NOT_NULL | BLOB | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 67108860
}
},
{
"ordinal": 8,
"name": "created_at",
"type_info": {
"type": "Datetime",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY",
"char_set": 63,
"max_size": 19
}
},
{
"ordinal": 9,
"name": "maybe_user_display_name?",
"type_info": {
"type": "VarString",
"flags": "BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 80
}
},
{
"ordinal": 10,
"name": "maybe_user_username?",
"type_info": {
"type": "VarString",
"flags": "UNIQUE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 80
}
},
{
"ordinal": 11,
"name": "maybe_user_gravatar_hash?",
"type_info": {
"type": "String",
"flags": "BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
}
],
"parameters": {
"Right": 3
},
"nullable": [
false,
false,
false,
true,
true,
true,
true,
false,
false,
true,
true,
true
]
},
"hash": "d1ade7fcfd3efac1cd3163c392110327bb73edb74fa0d9693fee5feddb8a3223"
}
@@ -0,0 +1,146 @@
{
"db_name": "MySQL",
"query": "\nSELECT\n d.id as `id: u64`,\n d.event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,\n d.debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,\n d.maybe_log_level as `maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel`,\n d.maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,\n d.maybe_ip_address,\n d.maybe_url,\n d.message,\n d.created_at,\n u.display_name as `maybe_user_display_name?`,\n u.username as `maybe_user_username?`,\n u.email_gravatar_hash as `maybe_user_gravatar_hash?`\nFROM debug_logs d\nLEFT OUTER JOIN users u\n ON u.token = d.maybe_creator_user_token\nWHERE d.event_token = ?\nORDER BY d.id ASC\nLIMIT ?\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "id: u64",
"type_info": {
"type": "LongLong",
"flags": "NOT_NULL | PRIMARY_KEY | UNSIGNED | AUTO_INCREMENT",
"char_set": 63,
"max_size": 20
}
},
{
"ordinal": 1,
"name": "event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 2,
"name": "debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 96
}
},
{
"ordinal": 3,
"name": "maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 64
}
},
{
"ordinal": 4,
"name": "maybe_creator_user_token: tokens::tokens::users::UserToken",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 5,
"name": "maybe_ip_address",
"type_info": {
"type": "VarString",
"flags": "BINARY",
"char_set": 224,
"max_size": 160
}
},
{
"ordinal": 6,
"name": "maybe_url",
"type_info": {
"type": "VarString",
"flags": "BINARY",
"char_set": 224,
"max_size": 1020
}
},
{
"ordinal": 7,
"name": "message",
"type_info": {
"type": "Blob",
"flags": "NOT_NULL | BLOB | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 67108860
}
},
{
"ordinal": 8,
"name": "created_at",
"type_info": {
"type": "Datetime",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY",
"char_set": 63,
"max_size": 19
}
},
{
"ordinal": 9,
"name": "maybe_user_display_name?",
"type_info": {
"type": "VarString",
"flags": "BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 80
}
},
{
"ordinal": 10,
"name": "maybe_user_username?",
"type_info": {
"type": "VarString",
"flags": "UNIQUE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 80
}
},
{
"ordinal": 11,
"name": "maybe_user_gravatar_hash?",
"type_info": {
"type": "String",
"flags": "BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false,
false,
true,
true,
true,
true,
false,
false,
true,
true,
true
]
},
"hash": "e9a149ffc963a59019fdd69bd53128fad946856d432108bf9b923dfeaf2442c5"
}
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
DROP COLUMN maybe_log_level;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
ADD COLUMN maybe_log_level VARCHAR(16) DEFAULT NULL;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
DROP INDEX index_maybe_log_level;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
ADD INDEX index_maybe_log_level (maybe_log_level);
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
DROP COLUMN maybe_ip_address;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
ADD COLUMN maybe_ip_address VARCHAR(40) DEFAULT NULL;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
DROP COLUMN maybe_url;
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE debug_logs
ADD COLUMN maybe_url VARCHAR(255) DEFAULT NULL;
@@ -13,9 +13,18 @@ CREATE TABLE debug_logs (
-- The type of debug log entry.
debug_log_type VARCHAR(24) NOT NULL,
-- The severity level of the log entry (info, warn, error, debug, trace).
maybe_log_level VARCHAR(16) DEFAULT NULL,
-- The user who created the log entry (if any).
maybe_creator_user_token VARCHAR(32) DEFAULT NULL,
-- The client IP address of the request (if any). Wide enough for IPv6.
maybe_ip_address VARCHAR(40) DEFAULT NULL,
-- The request URL (if any). Truncated to 255 characters on insert.
maybe_url VARCHAR(255) DEFAULT NULL,
-- The log message body.
message MEDIUMTEXT NOT NULL,
@@ -25,6 +34,7 @@ CREATE TABLE debug_logs (
PRIMARY KEY (id),
KEY index_event_token (event_token),
KEY index_debug_log_type (debug_log_type),
KEY index_maybe_log_level (maybe_log_level),
KEY index_maybe_creator_user_token (maybe_creator_user_token),
KEY index_created_at (created_at)
@@ -0,0 +1,36 @@
use chrono::{DateTime, Utc};
use serde_derive::Serialize;
use utoipa::ToSchema;
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
/// A single debug log row.
/// Shared by the moderation debug-log list endpoints.
#[derive(Serialize, ToSchema)]
pub struct ModerationDebugLogEntry {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub maybe_creator_user_token: Option<UserToken>,
/// The client IP address of the request (if recorded).
pub maybe_ip_address: Option<String>,
/// The request URL (if recorded; truncated to 255 chars).
pub maybe_url: Option<String>,
pub message: String,
pub created_at: DateTime<Utc>,
/// The creator user, joined in when the row has a user token.
pub maybe_user: Option<ModerationDebugLogUser>,
}
/// The creator user of a debug log row.
#[derive(Serialize, ToSchema)]
pub struct ModerationDebugLogUser {
pub user_token: UserToken,
pub display_name: String,
pub username: String,
pub gravatar_hash: String,
}
@@ -0,0 +1,4 @@
pub mod debug_log_entry;
pub mod moderation_list_all_debug_logs;
pub mod moderation_list_debug_logs_for_token;
pub mod moderation_list_debug_logs_for_user;
@@ -0,0 +1,76 @@
use chrono::{DateTime, Utc};
use serde_derive::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use enums::error::enum_error::EnumError;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
use crate::moderation::debug_logs::debug_log_entry::ModerationDebugLogUser;
pub const MODERATION_LIST_ALL_DEBUG_LOGS_PATH: &str = "/v1/moderation/debug_logs/list_all";
// ── Query params ──
#[derive(Deserialize, IntoParams, ToSchema)]
pub struct ModerationListAllDebugLogsQueryParams {
/// Optional comma-separated list of severity levels to include,
/// eg. `severities=error,warn`. Omit to include all rows.
pub severities: Option<String>,
/// Cursor for pagination. Pass the `next_cursor` from a previous response.
pub cursor: Option<u64>,
/// Max rows per page (default 50, max 200).
pub limit: Option<u32>,
}
impl ModerationListAllDebugLogsQueryParams {
/// Parse the comma-separated `severities` parameter.
/// Returns `None` when the parameter is absent or empty.
pub fn parsed_severities(&self) -> Result<Option<Vec<DebugLogLevel>>, EnumError> {
let raw = match self.severities.as_deref() {
Some(value) if !value.trim().is_empty() => value,
_ => return Ok(None),
};
let levels = raw
.split(',')
.map(str::trim)
.filter(|part| !part.is_empty())
.map(DebugLogLevel::from_str)
.collect::<Result<Vec<DebugLogLevel>, EnumError>>()?;
Ok(if levels.is_empty() { None } else { Some(levels) })
}
}
// ── Response ──
#[derive(Serialize, ToSchema)]
pub struct ModerationListAllDebugLogsSuccessResponse {
pub success: bool,
/// Most recent first.
pub debug_logs: Vec<ModerationListAllDebugLogsEntry>,
/// Cursor for the next page. `None` if there are no more results.
pub next_cursor: Option<u64>,
}
/// A debug log row with the creator user joined in (when one exists).
#[derive(Serialize, ToSchema)]
pub struct ModerationListAllDebugLogsEntry {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub maybe_creator_user_token: Option<UserToken>,
/// The client IP address of the request (if recorded).
pub maybe_ip_address: Option<String>,
/// The request URL (if recorded; truncated to 255 chars).
pub maybe_url: Option<String>,
pub message: String,
pub created_at: DateTime<Utc>,
pub maybe_user: Option<ModerationDebugLogUser>,
}
@@ -0,0 +1,30 @@
use serde_derive::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use crate::moderation::debug_logs::debug_log_entry::ModerationDebugLogEntry;
pub const MODERATION_LIST_DEBUG_LOGS_FOR_TOKEN_PATH: &str = "/v1/moderation/debug_logs/list/{token}";
// ── Path params ──
#[derive(Deserialize, ToSchema)]
pub struct ModerationListDebugLogsForTokenPathInfo {
pub token: DebugLogEventToken,
}
// ── Query params ──
#[derive(Deserialize, IntoParams, ToSchema)]
pub struct ModerationListDebugLogsForTokenQueryParams {
pub limit: Option<u32>,
}
// ── Response ──
#[derive(Serialize, ToSchema)]
pub struct ModerationListDebugLogsForTokenSuccessResponse {
pub success: bool,
pub debug_logs: Vec<ModerationDebugLogEntry>,
}
@@ -0,0 +1,37 @@
use serde_derive::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use tokens::tokens::users::UserToken;
use crate::moderation::debug_logs::debug_log_entry::ModerationDebugLogEntry;
pub const MODERATION_LIST_DEBUG_LOGS_FOR_USER_PATH: &str = "/v1/moderation/debug_logs/user_list/{user_token}";
// ── Path params ──
#[derive(Deserialize, ToSchema)]
pub struct ModerationListDebugLogsForUserPathInfo {
pub user_token: UserToken,
}
// ── Query params ──
#[derive(Deserialize, IntoParams, ToSchema)]
pub struct ModerationListDebugLogsForUserQueryParams {
/// Cursor for pagination. Pass the `next_cursor` from a previous response.
pub cursor: Option<u64>,
/// Max rows per page (default 50, max 200).
pub limit: Option<u32>,
}
// ── Response ──
#[derive(Serialize, ToSchema)]
pub struct ModerationListDebugLogsForUserSuccessResponse {
pub success: bool,
/// Most recent first.
pub debug_logs: Vec<ModerationDebugLogEntry>,
/// Cursor for the next page. `None` if there are no more results.
pub next_cursor: Option<u64>,
}
@@ -1,4 +1,5 @@
pub mod alerts;
pub mod debug_logs;
pub mod jobs;
pub mod user;
pub mod user_referrals;
@@ -3,6 +3,7 @@ use std::marker::PhantomData;
use log::warn;
use sqlx::{Executor, MySql, QueryBuilder};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
@@ -18,6 +19,12 @@ where
/// The user associated with these debug log events (if any).
pub maybe_creator_user_token: Option<&'e UserToken>,
/// The client IP address of the request (if any). Truncated on insert.
pub maybe_ip_address: Option<&'e str>,
/// The request URL (if any). Truncated to 255 characters on insert.
pub maybe_url: Option<&'e str>,
/// The events to insert.
pub events: Vec<DebugLogEvent>,
@@ -31,6 +38,7 @@ where
/// Callers should use `serde_json::to_string(...)` before constructing this.
pub struct DebugLogEvent {
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub message: String,
}
@@ -54,9 +62,11 @@ where
}
let maybe_user_token_str = args.maybe_creator_user_token.map(|t| t.as_str().to_string());
let maybe_ip_address = args.maybe_ip_address.map(|ip| truncate_chars(ip, MAX_IP_ADDRESS_CHARS));
let maybe_url = args.maybe_url.map(|url| truncate_chars(url, MAX_URL_CHARS));
let mut query_builder = QueryBuilder::new(
"INSERT INTO debug_logs (event_token, debug_log_type, maybe_creator_user_token, message) VALUES "
"INSERT INTO debug_logs (event_token, debug_log_type, maybe_log_level, maybe_creator_user_token, maybe_ip_address, maybe_url, message) VALUES "
);
for (i, event) in args.events.iter().enumerate() {
@@ -65,8 +75,14 @@ where
query_builder.push(", ");
query_builder.push_bind(event.debug_log_type.to_str());
query_builder.push(", ");
query_builder.push_bind(event.maybe_log_level.map(|l| l.to_str()));
query_builder.push(", ");
query_builder.push_bind(&maybe_user_token_str);
query_builder.push(", ");
query_builder.push_bind(maybe_ip_address);
query_builder.push(", ");
query_builder.push_bind(maybe_url);
query_builder.push(", ");
query_builder.push_bind(&event.message);
query_builder.push(")");
@@ -81,3 +97,17 @@ where
Ok(event_token)
}
/// Maximum stored length for `maybe_ip_address` (VARCHAR(40)).
const MAX_IP_ADDRESS_CHARS: usize = 40;
/// Maximum stored length for `maybe_url` (VARCHAR(255)).
const MAX_URL_CHARS: usize = 255;
/// Truncate to at most `max_chars` characters (UTF-8 safe).
fn truncate_chars(value: &str, max_chars: usize) -> &str {
match value.char_indices().nth(max_chars) {
Some((idx, _)) => &value[..idx],
None => value,
}
}
@@ -2,6 +2,7 @@ use std::marker::PhantomData;
use sqlx::{Executor, MySql};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
@@ -20,9 +21,18 @@ where
/// The type of debug log event.
pub debug_log_type: DebugLogType,
/// The severity level of the log event (if any).
pub maybe_log_level: Option<DebugLogLevel>,
/// The log message body (pre-serialized JSON or any text).
pub message: &'e str,
/// The client IP address of the request (if any). Truncated on insert.
pub maybe_ip_address: Option<&'e str>,
/// The request URL (if any). Truncated to 255 characters on insert.
pub maybe_url: Option<&'e str>,
pub mysql_executor: E,
pub phantom: PhantomData<&'c E>,
}
@@ -47,12 +57,18 @@ INSERT INTO debug_logs
SET
event_token = ?,
debug_log_type = ?,
maybe_log_level = ?,
maybe_creator_user_token = ?,
maybe_ip_address = ?,
maybe_url = ?,
message = ?
"#,
event_token.as_str(),
args.debug_log_type.to_str(),
args.maybe_log_level.map(|l| l.to_str()),
args.maybe_creator_user_token.map(|t| t.as_str()),
args.maybe_ip_address.map(|ip| truncate_chars(ip, MAX_IP_ADDRESS_CHARS)),
args.maybe_url.map(|url| truncate_chars(url, MAX_URL_CHARS)),
args.message,
)
.execute(args.mysql_executor)
@@ -60,3 +76,17 @@ SET
Ok(event_token)
}
/// Maximum stored length for `maybe_ip_address` (VARCHAR(40)).
const MAX_IP_ADDRESS_CHARS: usize = 40;
/// Maximum stored length for `maybe_url` (VARCHAR(255)).
const MAX_URL_CHARS: usize = 255;
/// Truncate to at most `max_chars` characters (UTF-8 safe).
fn truncate_chars(value: &str, max_chars: usize) -> &str {
match value.char_indices().nth(max_chars) {
Some((idx, _)) => &value[..idx],
None => value,
}
}
@@ -0,0 +1,148 @@
use std::marker::PhantomData;
use chrono::{DateTime, Utc};
use sqlx::mysql::MySqlRow;
use sqlx::{Executor, MySql, QueryBuilder, Row};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
const DEFAULT_LIMIT: u32 = 50;
const MAX_LIMIT: u32 = 200;
pub struct ListAllDebugLogsArgs<'c, E>
where
E: Executor<'c, Database = MySql>,
{
/// If provided, only rows with one of these log levels are returned.
/// (Rows with a NULL log level are excluded when filtering.)
pub maybe_log_levels: Option<Vec<DebugLogLevel>>,
/// Cursor for pagination: only rows with `id` strictly below this are
/// returned. Pass the `next_cursor` from a previous page.
pub maybe_id_cursor: Option<u64>,
pub limit: Option<u32>,
pub mysql_executor: E,
pub phantom: PhantomData<&'c E>,
}
pub struct ListAllDebugLogsResult {
pub debug_logs: Vec<AllDebugLogRow>,
/// Cursor for the next page. `None` if there are no more results.
pub next_cursor: Option<u64>,
}
/// A debug log row left-outer-joined against the creator user (if any).
#[derive(Debug)]
pub struct AllDebugLogRow {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub maybe_creator_user_token: Option<UserToken>,
pub maybe_ip_address: Option<String>,
pub maybe_url: Option<String>,
pub message: String,
pub created_at: DateTime<Utc>,
// Joined user fields (present when the creator user exists).
pub maybe_user_display_name: Option<String>,
pub maybe_user_username: Option<String>,
pub maybe_user_gravatar_hash: Option<String>,
}
/// List all debug logs, most recent first, cursor-paginated by `id`,
/// optionally scoped to a set of log levels.
///
/// NB: Uses `QueryBuilder` (not the compile-time macros) because of the
/// dynamic `WHERE maybe_log_level IN (...)` clause.
pub async fn list_all_debug_logs<'c, E>(
args: ListAllDebugLogsArgs<'c, E>,
) -> Result<ListAllDebugLogsResult, sqlx::Error>
where
E: Executor<'c, Database = MySql>,
{
let limit = args.limit.unwrap_or(DEFAULT_LIMIT).min(MAX_LIMIT);
// Fetch limit + 1 so we can detect whether there's a next page.
let fetch_limit = (limit + 1) as i64;
let id_cursor = args.maybe_id_cursor.unwrap_or(u64::MAX);
let mut query_builder: QueryBuilder<MySql> = QueryBuilder::new(
r#"
SELECT
d.id,
d.event_token,
d.debug_log_type,
d.maybe_log_level,
d.maybe_creator_user_token,
d.maybe_ip_address,
d.maybe_url,
d.message,
d.created_at,
u.display_name as maybe_user_display_name,
u.username as maybe_user_username,
u.email_gravatar_hash as maybe_user_gravatar_hash
FROM debug_logs d
LEFT OUTER JOIN users u
ON u.token = d.maybe_creator_user_token
WHERE d.id < "#,
);
query_builder.push_bind(id_cursor);
if let Some(levels) = args.maybe_log_levels.as_ref().filter(|l| !l.is_empty()) {
query_builder.push(" AND d.maybe_log_level IN (");
let mut separated = query_builder.separated(", ");
for level in levels {
separated.push_bind(level.to_str());
}
separated.push_unseparated(")");
}
query_builder.push(" ORDER BY d.id DESC LIMIT ");
query_builder.push_bind(fetch_limit);
let raw_rows: Vec<MySqlRow> = query_builder
.build()
.fetch_all(args.mysql_executor)
.await?;
let mut debug_logs = raw_rows
.into_iter()
.map(row_from_mysql_row)
.collect::<Result<Vec<AllDebugLogRow>, sqlx::Error>>()?;
let next_cursor = if debug_logs.len() > limit as usize {
debug_logs.truncate(limit as usize);
debug_logs.last().map(|row| row.id)
} else {
None
};
Ok(ListAllDebugLogsResult {
debug_logs,
next_cursor,
})
}
fn row_from_mysql_row(row: MySqlRow) -> Result<AllDebugLogRow, sqlx::Error> {
let created_at: DateTime<Utc> = row.try_get("created_at")?;
Ok(AllDebugLogRow {
id: row.try_get("id")?,
event_token: row.try_get("event_token")?,
debug_log_type: row.try_get("debug_log_type")?,
maybe_log_level: row.try_get("maybe_log_level")?,
maybe_creator_user_token: row.try_get("maybe_creator_user_token")?,
maybe_ip_address: row.try_get("maybe_ip_address")?,
maybe_url: row.try_get("maybe_url")?,
message: row.try_get("message")?,
created_at,
maybe_user_display_name: row.try_get("maybe_user_display_name")?,
maybe_user_username: row.try_get("maybe_user_username")?,
maybe_user_gravatar_hash: row.try_get("maybe_user_gravatar_hash")?,
})
}
@@ -3,6 +3,7 @@ use std::marker::PhantomData;
use chrono::{DateTime, NaiveDateTime, Utc};
use sqlx::{Executor, MySql};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
@@ -24,9 +25,17 @@ pub struct DebugLogRow {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub maybe_creator_user_token: Option<UserToken>,
pub maybe_ip_address: Option<String>,
pub maybe_url: Option<String>,
pub message: String,
pub created_at: DateTime<Utc>,
// Joined user fields (present when the creator user exists).
pub maybe_user_display_name: Option<String>,
pub maybe_user_username: Option<String>,
pub maybe_user_gravatar_hash: Option<String>,
}
#[derive(Debug)]
@@ -34,9 +43,15 @@ struct RawDebugLogRow {
id: u64,
event_token: DebugLogEventToken,
debug_log_type: DebugLogType,
maybe_log_level: Option<DebugLogLevel>,
maybe_creator_user_token: Option<UserToken>,
maybe_ip_address: Option<String>,
maybe_url: Option<String>,
message: String,
created_at: NaiveDateTime,
maybe_user_display_name: Option<String>,
maybe_user_username: Option<String>,
maybe_user_gravatar_hash: Option<String>,
}
pub async fn list_debug_logs_for_token<'e, 'c: 'e, E>(
@@ -51,15 +66,23 @@ where
RawDebugLogRow,
r#"
SELECT
id as `id: u64`,
event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,
debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,
maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,
message,
created_at
FROM debug_logs
WHERE event_token = ?
ORDER BY id ASC
d.id as `id: u64`,
d.event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,
d.debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,
d.maybe_log_level as `maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel`,
d.maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,
d.maybe_ip_address,
d.maybe_url,
d.message,
d.created_at,
u.display_name as `maybe_user_display_name?`,
u.username as `maybe_user_username?`,
u.email_gravatar_hash as `maybe_user_gravatar_hash?`
FROM debug_logs d
LEFT OUTER JOIN users u
ON u.token = d.maybe_creator_user_token
WHERE d.event_token = ?
ORDER BY d.id ASC
LIMIT ?
"#,
args.event_token.as_str(),
@@ -73,9 +96,15 @@ LIMIT ?
id: row.id,
event_token: row.event_token,
debug_log_type: row.debug_log_type,
maybe_log_level: row.maybe_log_level,
maybe_creator_user_token: row.maybe_creator_user_token,
maybe_ip_address: row.maybe_ip_address,
maybe_url: row.maybe_url,
message: row.message,
created_at: row.created_at.and_utc(),
maybe_user_display_name: row.maybe_user_display_name,
maybe_user_username: row.maybe_user_username,
maybe_user_gravatar_hash: row.maybe_user_gravatar_hash,
}
}).collect();
@@ -0,0 +1,142 @@
use std::marker::PhantomData;
use chrono::{DateTime, NaiveDateTime, Utc};
use sqlx::{Executor, MySql};
use enums::by_table::debug_logs::debug_log_level::DebugLogLevel;
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
const DEFAULT_LIMIT: u32 = 50;
const MAX_LIMIT: u32 = 200;
pub struct ListDebugLogsForUserArgs<'e, 'c, E>
where
E: 'e + Executor<'c, Database = MySql>,
{
pub user_token: &'e UserToken,
/// Cursor for pagination: only rows with `id` strictly below this are
/// returned. Pass the `next_cursor` from a previous page.
pub maybe_id_cursor: Option<u64>,
pub limit: Option<u32>,
pub mysql_executor: E,
pub phantom: PhantomData<&'c E>,
}
pub struct ListDebugLogsForUserResult {
pub debug_logs: Vec<UserDebugLogRow>,
/// Cursor for the next page. `None` if there are no more results.
pub next_cursor: Option<u64>,
}
#[derive(Debug)]
pub struct UserDebugLogRow {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_log_level: Option<DebugLogLevel>,
pub maybe_creator_user_token: Option<UserToken>,
pub maybe_ip_address: Option<String>,
pub maybe_url: Option<String>,
pub message: String,
pub created_at: DateTime<Utc>,
// Joined user fields (present when the creator user exists).
pub maybe_user_display_name: Option<String>,
pub maybe_user_username: Option<String>,
pub maybe_user_gravatar_hash: Option<String>,
}
#[derive(Debug)]
struct RawUserDebugLogRow {
id: u64,
event_token: DebugLogEventToken,
debug_log_type: DebugLogType,
maybe_log_level: Option<DebugLogLevel>,
maybe_creator_user_token: Option<UserToken>,
maybe_ip_address: Option<String>,
maybe_url: Option<String>,
message: String,
created_at: NaiveDateTime,
maybe_user_display_name: Option<String>,
maybe_user_username: Option<String>,
maybe_user_gravatar_hash: Option<String>,
}
/// List debug logs for a user, most recent first, cursor-paginated by `id`.
pub async fn list_debug_logs_for_user<'e, 'c: 'e, E>(
args: ListDebugLogsForUserArgs<'e, 'c, E>,
) -> Result<ListDebugLogsForUserResult, sqlx::Error>
where
E: 'e + Executor<'c, Database = MySql>,
{
let limit = args.limit.unwrap_or(DEFAULT_LIMIT).min(MAX_LIMIT);
// Fetch limit + 1 so we can detect whether there's a next page.
let fetch_limit = (limit + 1) as i64;
// Use u64::MAX when no cursor so all rows are included.
let id_cursor = args.maybe_id_cursor.unwrap_or(u64::MAX);
let rows = sqlx::query_as!(
RawUserDebugLogRow,
r#"
SELECT
d.id as `id: u64`,
d.event_token as `event_token: tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken`,
d.debug_log_type as `debug_log_type: enums::by_table::debug_logs::debug_log_type::DebugLogType`,
d.maybe_log_level as `maybe_log_level: enums::by_table::debug_logs::debug_log_level::DebugLogLevel`,
d.maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,
d.maybe_ip_address,
d.maybe_url,
d.message,
d.created_at,
u.display_name as `maybe_user_display_name?`,
u.username as `maybe_user_username?`,
u.email_gravatar_hash as `maybe_user_gravatar_hash?`
FROM debug_logs d
LEFT OUTER JOIN users u
ON u.token = d.maybe_creator_user_token
WHERE d.maybe_creator_user_token = ?
AND d.id < ?
ORDER BY d.id DESC
LIMIT ?
"#,
args.user_token.as_str(),
id_cursor,
fetch_limit,
)
.fetch_all(args.mysql_executor)
.await?;
let mut debug_logs: Vec<UserDebugLogRow> = rows.into_iter().map(|row| {
UserDebugLogRow {
id: row.id,
event_token: row.event_token,
debug_log_type: row.debug_log_type,
maybe_log_level: row.maybe_log_level,
maybe_creator_user_token: row.maybe_creator_user_token,
maybe_ip_address: row.maybe_ip_address,
maybe_url: row.maybe_url,
message: row.message,
created_at: row.created_at.and_utc(),
maybe_user_display_name: row.maybe_user_display_name,
maybe_user_username: row.maybe_user_username,
maybe_user_gravatar_hash: row.maybe_user_gravatar_hash,
}
}).collect();
let next_cursor = if debug_logs.len() > limit as usize {
debug_logs.truncate(limit as usize);
debug_logs.last().map(|row| row.id)
} else {
None
};
Ok(ListDebugLogsForUserResult {
debug_logs,
next_cursor,
})
}
@@ -1,3 +1,5 @@
pub mod bulk_insert_debug_logs;
pub mod insert_debug_log;
pub mod list_all_debug_logs;
pub mod list_debug_logs_for_token;
pub mod list_debug_logs_for_user;
@@ -0,0 +1,151 @@
use std::collections::BTreeSet;
#[cfg(test)]
use strum::EnumCount;
#[cfg(test)]
use strum::EnumIter;
use utoipa::ToSchema;
use crate::error::enum_error::EnumError;
/// Maximum serialized string length for database storage.
pub const MAX_LENGTH: usize = 16;
/// Used in the `debug_logs` table in a `VARCHAR(16)` field (`maybe_log_level`).
///
/// DO NOT CHANGE VALUES WITHOUT A MIGRATION STRATEGY.
#[cfg_attr(test, derive(EnumIter, EnumCount))]
#[derive(Clone, Copy, Eq, PartialEq, Ord, PartialOrd, Hash, Serialize, Deserialize, ToSchema)]
#[serde(rename_all = "snake_case")]
pub enum DebugLogLevel {
Info,
Warn,
Error,
Debug,
Trace,
}
impl_enum_display_and_debug_using_to_str!(DebugLogLevel);
impl_mysql_enum_coders!(DebugLogLevel);
impl_mysql_from_row!(DebugLogLevel);
impl DebugLogLevel {
pub fn to_str(&self) -> &'static str {
match self {
Self::Info => "info",
Self::Warn => "warn",
Self::Error => "error",
Self::Debug => "debug",
Self::Trace => "trace",
}
}
pub fn from_str(value: &str) -> Result<Self, EnumError> {
match value {
"info" => Ok(Self::Info),
"warn" => Ok(Self::Warn),
"error" => Ok(Self::Error),
"debug" => Ok(Self::Debug),
"trace" => Ok(Self::Trace),
_ => Err(EnumError::CouldNotConvertFromString(value.to_string())),
}
}
pub fn all_variants() -> BTreeSet<Self> {
BTreeSet::from([
Self::Info,
Self::Warn,
Self::Error,
Self::Debug,
Self::Trace,
])
}
}
#[cfg(test)]
mod tests {
use crate::by_table::debug_logs::debug_log_level::DebugLogLevel;
use crate::by_table::debug_logs::debug_log_level::MAX_LENGTH;
use crate::test_helpers::assert_serialization;
mod explicit_checks {
use super::*;
use crate::error::enum_error::EnumError;
#[test]
fn test_serialization() {
assert_serialization(DebugLogLevel::Info, "info");
assert_serialization(DebugLogLevel::Warn, "warn");
assert_serialization(DebugLogLevel::Error, "error");
assert_serialization(DebugLogLevel::Debug, "debug");
assert_serialization(DebugLogLevel::Trace, "trace");
}
#[test]
fn to_str() {
assert_eq!(DebugLogLevel::Info.to_str(), "info");
assert_eq!(DebugLogLevel::Warn.to_str(), "warn");
assert_eq!(DebugLogLevel::Error.to_str(), "error");
assert_eq!(DebugLogLevel::Debug.to_str(), "debug");
assert_eq!(DebugLogLevel::Trace.to_str(), "trace");
}
#[test]
fn from_str() {
assert_eq!(DebugLogLevel::from_str("info").unwrap(), DebugLogLevel::Info);
assert_eq!(DebugLogLevel::from_str("warn").unwrap(), DebugLogLevel::Warn);
assert_eq!(DebugLogLevel::from_str("error").unwrap(), DebugLogLevel::Error);
assert_eq!(DebugLogLevel::from_str("debug").unwrap(), DebugLogLevel::Debug);
assert_eq!(DebugLogLevel::from_str("trace").unwrap(), DebugLogLevel::Trace);
}
#[test]
fn from_str_err() {
let result = DebugLogLevel::from_str("invalid");
assert!(result.is_err());
if let Err(EnumError::CouldNotConvertFromString(value)) = result {
assert_eq!(value, "invalid");
} else {
panic!("Expected EnumError::CouldNotConvertFromString");
}
}
#[test]
fn all_variants() {
let mut variants = DebugLogLevel::all_variants();
assert_eq!(variants.len(), 5);
assert_eq!(variants.pop_first(), Some(DebugLogLevel::Info));
assert_eq!(variants.pop_first(), Some(DebugLogLevel::Warn));
assert_eq!(variants.pop_first(), Some(DebugLogLevel::Error));
assert_eq!(variants.pop_first(), Some(DebugLogLevel::Debug));
assert_eq!(variants.pop_first(), Some(DebugLogLevel::Trace));
assert_eq!(variants.pop_first(), None);
}
}
mod mechanical_checks {
use super::*;
#[test]
fn variant_length() {
use strum::IntoEnumIterator;
assert_eq!(DebugLogLevel::all_variants().len(), DebugLogLevel::iter().len());
}
#[test]
fn round_trip() {
for variant in DebugLogLevel::all_variants() {
assert_eq!(variant, DebugLogLevel::from_str(variant.to_str()).unwrap());
assert_eq!(variant, DebugLogLevel::from_str(&format!("{}", variant)).unwrap());
assert_eq!(variant, DebugLogLevel::from_str(&format!("{:?}", variant)).unwrap());
}
}
#[test]
fn max_length() {
for variant in DebugLogLevel::all_variants() {
assert!(variant.to_str().len() <= MAX_LENGTH);
}
}
}
}
@@ -1 +1,2 @@
pub mod debug_log_level;
pub mod debug_log_type;
@@ -88,7 +88,10 @@ use crate::http_server::endpoints::media_files::upsert_upload::write_error::Medi
use crate::http_server::endpoints::media_files::upsert_upload::write_scene_file::write_scene_file_media_file_handler::*;
use crate::http_server::endpoints::model_download::enqueue_gptsovits_model_download_handler::*;
use artcraft_api_defs::generate::video::edit::beeble_switchx_edit_video::*;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_token_handler::*;
use artcraft_api_defs::moderation::debug_logs::debug_log_entry::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_all_debug_logs::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_token::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_user::*;
use crate::http_server::endpoints::moderation::jobs::moderation_get_job_by_token_handler::*;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_edit_user_feature_flags_handler::*;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_list_all_available_user_feature_flags_handler::*;
@@ -602,7 +605,9 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
crate::http_server::endpoints::moderation::wallets::moderator_add_banked_balance_to_wallet_handler::moderator_add_banked_balance_to_wallet_handler,
crate::http_server::endpoints::moderation::wallets::moderator_create_wallet_for_user_handler::moderator_create_wallet_for_user_handler,
crate::http_server::endpoints::moderation::wallets::moderator_get_wallet_handler::moderator_get_wallet_handler,
crate::http_server::endpoints::moderation::debug_logs::moderation_list_all_debug_logs_handler::moderation_list_all_debug_logs_handler,
crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_token_handler::moderation_list_debug_logs_for_token_handler,
crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_user_handler::moderation_list_debug_logs_for_user_handler,
crate::http_server::endpoints::moderation::jobs::moderation_get_job_by_token_handler::moderation_get_job_by_token_handler,
crate::http_server::endpoints::moderation::staff_audit_logs::moderator_list_staff_audit_logs_handler::moderator_list_staff_audit_logs_handler,
crate::http_server::endpoints::moderation::user_referrals::moderator_list_global_user_referrals_handler::moderator_list_global_user_referrals_handler,
@@ -1030,10 +1035,17 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
GetJobByTokenPathInfo,
GetJobByTokenSuccessResponse,
ModerationJobResponse,
ListDebugLogsPathInfo,
ListDebugLogsQueryParams,
ListDebugLogsSuccessResponse,
DebugLogEntry,
ModerationDebugLogEntry,
ModerationDebugLogUser,
ModerationListAllDebugLogsEntry,
ModerationListAllDebugLogsQueryParams,
ModerationListAllDebugLogsSuccessResponse,
ModerationListDebugLogsForTokenPathInfo,
ModerationListDebugLogsForTokenQueryParams,
ModerationListDebugLogsForTokenSuccessResponse,
ModerationListDebugLogsForUserPathInfo,
ModerationListDebugLogsForUserQueryParams,
ModerationListDebugLogsForUserSuccessResponse,
ListStaffAuditLogsQueryParams,
ListStaffAuditLogsSuccessResponse,
ListImpersonationRequestsQueryParams,
@@ -1 +1,3 @@
pub mod moderation_list_all_debug_logs_handler;
pub mod moderation_list_debug_logs_for_token_handler;
pub mod moderation_list_debug_logs_for_user_handler;
@@ -0,0 +1,91 @@
use std::sync::Arc;
use actix_web::web::Json;
use actix_web::{web, HttpRequest};
use log::warn;
use artcraft_api_defs::moderation::debug_logs::debug_log_entry::ModerationDebugLogUser;
use artcraft_api_defs::moderation::debug_logs::moderation_list_all_debug_logs::{
ModerationListAllDebugLogsEntry, ModerationListAllDebugLogsQueryParams,
ModerationListAllDebugLogsSuccessResponse,
};
use mysql_queries::queries::debug_logs::list_all_debug_logs::{
list_all_debug_logs, ListAllDebugLogsArgs,
};
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::user_lookup::user_session::require_moderator::require_moderator;
use crate::state::server_state::ServerState;
#[utoipa::path(
get,
tag = "Moderation",
path = "/v1/moderation/debug_logs/list_all",
params(
ModerationListAllDebugLogsQueryParams,
),
responses(
(status = 200, description = "Success", body = ModerationListAllDebugLogsSuccessResponse),
(status = 400, description = "Bad request (invalid severity)"),
(status = 401, description = "Unauthorized"),
(status = 500, description = "Server error"),
),
)]
pub async fn moderation_list_all_debug_logs_handler(
http_request: HttpRequest,
query: web::Query<ModerationListAllDebugLogsQueryParams>,
server_state: web::Data<Arc<ServerState>>,
) -> Result<Json<ModerationListAllDebugLogsSuccessResponse>, CommonWebError> {
let maybe_log_levels = query.parsed_severities().map_err(|err| {
CommonWebError::BadInputWithSimpleMessage(format!("invalid severities parameter: {}", err))
})?;
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
let _user_session = require_moderator(&http_request, &server_state.session_checker, &mut *mysql_connection).await?;
let result = list_all_debug_logs(ListAllDebugLogsArgs {
maybe_log_levels,
maybe_id_cursor: query.cursor,
limit: query.limit,
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
}).await.map_err(|err| {
warn!("Error listing all debug logs: {:?}", err);
CommonWebError::from_error(err)
})?;
let debug_logs: Vec<ModerationListAllDebugLogsEntry> = result.debug_logs.into_iter().map(|row| {
// The join only yields user fields when the creator user exists.
let maybe_user = match (row.maybe_creator_user_token.clone(), row.maybe_user_display_name, row.maybe_user_username, row.maybe_user_gravatar_hash) {
(Some(user_token), Some(display_name), Some(username), Some(gravatar_hash)) => {
Some(ModerationDebugLogUser {
user_token,
display_name,
username,
gravatar_hash,
})
}
_ => None,
};
ModerationListAllDebugLogsEntry {
id: row.id,
event_token: row.event_token,
debug_log_type: row.debug_log_type,
maybe_log_level: row.maybe_log_level,
maybe_creator_user_token: row.maybe_creator_user_token,
maybe_ip_address: row.maybe_ip_address,
maybe_url: row.maybe_url,
message: row.message,
created_at: row.created_at,
maybe_user,
}
}).collect();
Ok(Json(ModerationListAllDebugLogsSuccessResponse {
success: true,
debug_logs,
next_cursor: result.next_cursor,
}))
}
@@ -2,80 +2,45 @@ use std::sync::Arc;
use actix_web::web::Json;
use actix_web::{web, HttpRequest};
use chrono::{DateTime, Utc};
use log::warn;
use serde_derive::{Deserialize, Serialize};
use utoipa::{IntoParams, ToSchema};
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use artcraft_api_defs::moderation::debug_logs::debug_log_entry::{ModerationDebugLogEntry, ModerationDebugLogUser};
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_token::{
ModerationListDebugLogsForTokenPathInfo, ModerationListDebugLogsForTokenQueryParams,
ModerationListDebugLogsForTokenSuccessResponse,
};
use mysql_queries::queries::debug_logs::list_debug_logs_for_token::{
list_debug_logs_for_token, ListDebugLogsForTokenArgs,
};
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
use tokens::tokens::users::UserToken;
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::user_lookup::user_session::require_moderator::require_moderator;
use crate::state::server_state::ServerState;
// ── Path params ──
#[derive(Deserialize, ToSchema)]
pub struct ListDebugLogsPathInfo {
pub token: DebugLogEventToken,
}
// ── Query params ──
#[derive(Deserialize, IntoParams, ToSchema)]
pub struct ListDebugLogsQueryParams {
pub limit: Option<u32>,
}
// ── Response ──
#[derive(Serialize, ToSchema)]
pub struct ListDebugLogsSuccessResponse {
pub success: bool,
pub debug_logs: Vec<DebugLogEntry>,
}
#[derive(Serialize, ToSchema)]
pub struct DebugLogEntry {
pub id: u64,
pub event_token: DebugLogEventToken,
pub debug_log_type: DebugLogType,
pub maybe_creator_user_token: Option<UserToken>,
pub message: String,
pub created_at: DateTime<Utc>,
}
// ── Handler ──
#[utoipa::path(
get,
tag = "Moderation",
path = "/v1/moderation/debug_logs/list/{token}",
params(
("token" = String, Path, description = "Debug log event token"),
ListDebugLogsQueryParams,
ModerationListDebugLogsForTokenQueryParams,
),
responses(
(status = 200, description = "Success", body = ListDebugLogsSuccessResponse),
(status = 200, description = "Success", body = ModerationListDebugLogsForTokenSuccessResponse),
(status = 401, description = "Unauthorized"),
(status = 500, description = "Server error"),
),
)]
pub async fn moderation_list_debug_logs_for_token_handler(
http_request: HttpRequest,
path: web::Path<ListDebugLogsPathInfo>,
query: web::Query<ListDebugLogsQueryParams>,
path: web::Path<ModerationListDebugLogsForTokenPathInfo>,
query: web::Query<ModerationListDebugLogsForTokenQueryParams>,
server_state: web::Data<Arc<ServerState>>,
) -> Result<Json<ListDebugLogsSuccessResponse>, CommonWebError> {
let _user_session = require_moderator(&http_request, &server_state.session_checker, &server_state.mysql_pool).await?;
) -> Result<Json<ModerationListDebugLogsForTokenSuccessResponse>, CommonWebError> {
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
let _user_session = require_moderator(&http_request, &server_state.session_checker, &mut *mysql_connection).await?;
let rows = list_debug_logs_for_token(ListDebugLogsForTokenArgs {
event_token: &path.token,
limit: query.limit,
@@ -86,18 +51,35 @@ pub async fn moderation_list_debug_logs_for_token_handler(
CommonWebError::from_error(err)
})?;
let debug_logs: Vec<DebugLogEntry> = rows.into_iter().map(|row| {
DebugLogEntry {
let debug_logs: Vec<ModerationDebugLogEntry> = rows.into_iter().map(|row| {
// The join only yields user fields when the creator user exists.
let maybe_user = match (row.maybe_creator_user_token.clone(), row.maybe_user_display_name, row.maybe_user_username, row.maybe_user_gravatar_hash) {
(Some(user_token), Some(display_name), Some(username), Some(gravatar_hash)) => {
Some(ModerationDebugLogUser {
user_token,
display_name,
username,
gravatar_hash,
})
}
_ => None,
};
ModerationDebugLogEntry {
id: row.id,
event_token: row.event_token,
debug_log_type: row.debug_log_type,
maybe_log_level: row.maybe_log_level,
maybe_creator_user_token: row.maybe_creator_user_token,
maybe_ip_address: row.maybe_ip_address,
maybe_url: row.maybe_url,
message: row.message,
created_at: row.created_at,
maybe_user,
}
}).collect();
Ok(Json(ListDebugLogsSuccessResponse {
Ok(Json(ModerationListDebugLogsForTokenSuccessResponse {
success: true,
debug_logs,
}))
@@ -0,0 +1,88 @@
use std::sync::Arc;
use actix_web::web::Json;
use actix_web::{web, HttpRequest};
use log::warn;
use artcraft_api_defs::moderation::debug_logs::debug_log_entry::{ModerationDebugLogEntry, ModerationDebugLogUser};
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_user::{
ModerationListDebugLogsForUserPathInfo, ModerationListDebugLogsForUserQueryParams,
ModerationListDebugLogsForUserSuccessResponse,
};
use mysql_queries::queries::debug_logs::list_debug_logs_for_user::{
list_debug_logs_for_user, ListDebugLogsForUserArgs,
};
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::user_lookup::user_session::require_moderator::require_moderator;
use crate::state::server_state::ServerState;
#[utoipa::path(
get,
tag = "Moderation",
path = "/v1/moderation/debug_logs/user_list/{user_token}",
params(
("user_token" = String, Path, description = "User token"),
ModerationListDebugLogsForUserQueryParams,
),
responses(
(status = 200, description = "Success", body = ModerationListDebugLogsForUserSuccessResponse),
(status = 401, description = "Unauthorized"),
(status = 500, description = "Server error"),
),
)]
pub async fn moderation_list_debug_logs_for_user_handler(
http_request: HttpRequest,
path: web::Path<ModerationListDebugLogsForUserPathInfo>,
query: web::Query<ModerationListDebugLogsForUserQueryParams>,
server_state: web::Data<Arc<ServerState>>,
) -> Result<Json<ModerationListDebugLogsForUserSuccessResponse>, CommonWebError> {
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
let _user_session = require_moderator(&http_request, &server_state.session_checker, &mut *mysql_connection).await?;
let result = list_debug_logs_for_user(ListDebugLogsForUserArgs {
user_token: &path.user_token,
maybe_id_cursor: query.cursor,
limit: query.limit,
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
}).await.map_err(|err| {
warn!("Error listing debug logs for user {}: {:?}", path.user_token, err);
CommonWebError::from_error(err)
})?;
let debug_logs: Vec<ModerationDebugLogEntry> = result.debug_logs.into_iter().map(|row| {
// The join only yields user fields when the creator user exists.
let maybe_user = match (row.maybe_creator_user_token.clone(), row.maybe_user_display_name, row.maybe_user_username, row.maybe_user_gravatar_hash) {
(Some(user_token), Some(display_name), Some(username), Some(gravatar_hash)) => {
Some(ModerationDebugLogUser {
user_token,
display_name,
username,
gravatar_hash,
})
}
_ => None,
};
ModerationDebugLogEntry {
id: row.id,
event_token: row.event_token,
debug_log_type: row.debug_log_type,
maybe_log_level: row.maybe_log_level,
maybe_creator_user_token: row.maybe_creator_user_token,
maybe_ip_address: row.maybe_ip_address,
maybe_url: row.maybe_url,
message: row.message,
created_at: row.created_at,
maybe_user,
}
}).collect();
Ok(Json(ModerationListDebugLogsForUserSuccessResponse {
success: true,
debug_logs,
next_cursor: result.next_cursor,
}))
}
@@ -106,6 +106,7 @@ pub async fn omni_api_image_generate_handler(
})?;
let ip_address = get_request_ip(&http_request);
let request_url = http_request.uri().to_string();
// ==================== INGEST URL INPUTS ==================== //
@@ -138,6 +139,9 @@ pub async fn omni_api_image_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::HttpRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &serde_json::to_string(&*request).unwrap_or_default(),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -169,6 +173,9 @@ pub async fn omni_api_image_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::FalRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -119,6 +119,7 @@ pub async fn omni_api_video_generate_handler(
})?;
let ip_address = get_request_ip(&http_request);
let request_url = http_request.uri().to_string();
// ==================== INGEST URL INPUTS ==================== //
@@ -205,6 +206,9 @@ pub async fn omni_api_video_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::HttpRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &serde_json::to_string(&*request).unwrap_or_default(),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -236,6 +240,9 @@ pub async fn omni_api_video_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::FalRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -253,6 +260,9 @@ pub async fn omni_api_video_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::GrokApiRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -130,10 +130,16 @@ pub async fn omni_gen_image_generate_handler(
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
let ip_address = get_request_ip(&http_request);
let request_url = http_request.uri().to_string();
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::HttpRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &serde_json::to_string(&*request).unwrap_or_default(),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -165,6 +171,9 @@ pub async fn omni_gen_image_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::FalRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -176,7 +185,6 @@ pub async fn omni_gen_image_generate_handler(
// ==================== WRITE RESULT ==================== //
let ip_address = get_request_ip(&http_request);
let maybe_platform_type = get_request_platform_type(&http_request);
let mut transaction = mysql_connection
@@ -191,10 +191,16 @@ pub async fn omni_gen_video_generate_handler(
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
let ip_address = get_request_ip(&http_request);
let request_url = http_request.uri().to_string();
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::HttpRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &serde_json::to_string(&*request).unwrap_or_default(),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -226,6 +232,9 @@ pub async fn omni_gen_video_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::FalRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -243,6 +252,9 @@ pub async fn omni_gen_video_generate_handler(
apriori_debug_log_event_token: Some(&debug_log_event_token),
maybe_creator_user_token: Some(user_token),
debug_log_type: DebugLogType::GrokApiRequest,
maybe_log_level: None,
maybe_ip_address: Some(&ip_address),
maybe_url: Some(&request_url),
message: &format!("{:#?}", outbound_request),
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -254,7 +266,6 @@ pub async fn omni_gen_video_generate_handler(
// ==================== WRITE RESULT ==================== //
let ip_address = get_request_ip(&http_request);
let maybe_platform_type = get_request_platform_type(&http_request);
let mut transaction = mysql_connection.begin().await.map_err(|err| {
@@ -101,6 +101,9 @@ pub async fn beeble_webhook_handler(
apriori_debug_log_event_token: Some(debug_log_event_token),
maybe_creator_user_token: job.maybe_creator_user_token.as_ref(),
debug_log_type: DebugLogType::BeebleWebhook,
maybe_log_level: None,
maybe_ip_address: None,
maybe_url: None,
message: &raw_body,
mysql_executor: &mut *mysql_connection,
phantom: Default::default(),
@@ -52,6 +52,9 @@ pub async fn handle_failed_fal_webhook(
apriori_debug_log_event_token: Some(debug_log_event_token),
maybe_creator_user_token: job.maybe_creator_user_token.as_ref(),
debug_log_type: DebugLogType::FalWebhook,
maybe_log_level: None,
maybe_ip_address: None,
maybe_url: None,
message: raw_body,
mysql_executor: &mut **mysql_connection,
phantom: Default::default(),
@@ -52,6 +52,9 @@ pub async fn handle_successful_fal_webhook(
apriori_debug_log_event_token: Some(debug_log_event_token),
maybe_creator_user_token: job.maybe_creator_user_token.as_ref(),
debug_log_type: DebugLogType::FalWebhook,
maybe_log_level: None,
maybe_ip_address: None,
maybe_url: None,
message: raw_body,
mysql_executor: &mut **mysql_connection,
phantom: Default::default(),
@@ -36,7 +36,9 @@ use crate::http_server::endpoints::moderation::user::moderator_user_lookup_by_st
use crate::http_server::endpoints::moderation::user::moderator_user_lookup_handler::moderator_user_lookup_handler;
use crate::http_server::endpoints::moderation::alerts::moderation_send_alert_handler::moderation_send_alert_handler;
use crate::http_server::endpoints::moderation::staff_audit_logs::moderator_list_staff_audit_logs_handler::moderator_list_staff_audit_logs_handler;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_all_debug_logs_handler::moderation_list_all_debug_logs_handler;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_token_handler::moderation_list_debug_logs_for_token_handler;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_user_handler::moderation_list_debug_logs_for_user_handler;
use crate::http_server::endpoints::moderation::jobs::moderation_get_job_by_token_handler::moderation_get_job_by_token_handler;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_edit_user_feature_flags_handler::moderator_edit_user_feature_flags_handler;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_list_all_available_user_feature_flags_handler::moderator_list_all_available_user_feature_flags_handler;
@@ -102,6 +104,14 @@ pub fn add_moderator_routes<T, B> (app: App<T>) -> App<T>
.route(web::get().to(moderation_list_debug_logs_for_token_handler))
.route(web::head().to(|| HttpResponse::Ok()))
)
.service(web::resource("/list_all")
.route(web::get().to(moderation_list_all_debug_logs_handler))
.route(web::head().to(|| HttpResponse::Ok()))
)
.service(web::resource("/user_list/{user_token}")
.route(web::get().to(moderation_list_debug_logs_for_user_handler))
.route(web::head().to(|| HttpResponse::Ok()))
)
)
.service(
web::scope("/user_emails")
+15 -2
View File
@@ -1,4 +1,4 @@
import { BrowserRouter, Routes, Route, Navigate, useLocation, useSearchParams } from "react-router-dom";
import { BrowserRouter, Routes, Route, Navigate, useLocation, useSearchParams, useParams } from "react-router-dom";
import { AuthProvider, useAuth } from "@/context/AuthContext";
import { DashboardLayout } from "@/layouts/DashboardLayout";
import { LoginPage } from "@/pages/Login";
@@ -22,6 +22,8 @@ import { UserReferrals } from "@/pages/UserReferrals";
import { JobInfo } from "@/pages/JobInfo";
import { JobTokenSearch } from "@/pages/JobTokenSearch";
import { DebugLogs } from "@/pages/DebugLogs";
import { DebugLogsAll } from "@/pages/DebugLogsAll";
import { UserDebugLogs } from "@/pages/UserDebugLogs";
import { DebugLogsSearch } from "@/pages/DebugLogsSearch";
import { SpendEvents } from "@/pages/SpendEvents";
import { ReengagementList } from "@/pages/ReengagementList";
@@ -84,9 +86,13 @@ function AppRoutes() {
path="moderation/debug-logs-search"
element={<DebugLogsSearch />}
/>
<Route path="debug_logs/list" element={<DebugLogsAll />} />
<Route path="debug_logs/user/:userToken" element={<UserDebugLogs />} />
<Route path="debug_logs/:eventToken" element={<DebugLogs />} />
{/* Legacy URL — redirect old links/bookmarks to the new path. */}
<Route
path="moderation/debug-logs/:eventToken"
element={<DebugLogs />}
element={<LegacyDebugLogsRedirect />}
/>
</Route>
@@ -109,3 +115,10 @@ function App() {
}
export default App;
function LegacyDebugLogsRedirect() {
const { eventToken } = useParams<{ eventToken: string }>();
return (
<Navigate to={`/debug_logs/${encodeURIComponent(eventToken || "")}`} replace />
);
}
@@ -1,5 +1,6 @@
import { ApiManager, type ApiResponse } from "./ApiManager";
import type {
AllDebugLog,
Wallet,
WalletWithOwner,
WalletLedgerEntry,
@@ -739,6 +740,74 @@ export class ModerationApi extends ApiManager {
}));
}
public async ListAllDebugLogs(params: {
severities?: string[];
cursor?: number;
limit?: number;
}): Promise<
ApiResponse<{ debug_logs: AllDebugLog[]; next_cursor: number | null }>
> {
const endpoint = `${this.getApiSchemeAndHost()}/v1/moderation/debug_logs/list_all`;
return await this.get<{
success: boolean;
debug_logs: AllDebugLog[];
next_cursor: number | null;
error_message?: string;
}>({
endpoint,
query: {
severities: params.severities?.length
? params.severities.join(",")
: undefined,
cursor: params.cursor,
limit: params.limit,
},
})
.then((response) => ({
success: response.success,
data: {
debug_logs: response.debug_logs || [],
next_cursor: response.next_cursor ?? null,
},
errorMessage: response.error_message,
}))
.catch((err) => ({
success: false,
errorMessage: err.message,
}));
}
public async ListUserDebugLogs(
user_token: string,
cursor?: number,
limit?: number,
): Promise<
ApiResponse<{ debug_logs: DebugLog[]; next_cursor: number | null }>
> {
const endpoint = `${this.getApiSchemeAndHost()}/v1/moderation/debug_logs/user_list/${encodeURIComponent(user_token)}`;
return await this.get<{
success: boolean;
debug_logs: DebugLog[];
next_cursor: number | null;
error_message?: string;
}>({
endpoint,
query: { cursor, limit },
})
.then((response) => ({
success: response.success,
data: {
debug_logs: response.debug_logs || [],
next_cursor: response.next_cursor ?? null,
},
errorMessage: response.error_message,
}))
.catch((err) => ({
success: false,
errorMessage: err.message,
}));
}
public async ListUserJobs(
user_token: string,
): Promise<ApiResponse<{ jobs: UserJob[] }>> {
@@ -0,0 +1,444 @@
import { useEffect, useMemo, useState } from "react";
import { Link } from "react-router-dom";
import { Badge } from "@/components/ui/badge";
import { Button } from "@/components/ui/button";
import { humanize } from "@/lib/utils";
import type { DebugLog } from "@/types";
import {
IconCheck,
IconChevronDown,
IconChevronRight,
IconCopy,
IconListSearch,
IconUser,
} from "@tabler/icons-react";
/** Compact form for the header row, eg. "Jul 2, 18:02:47". */
export function formatDebugLogDateTimeCompact(iso: string): string {
return new Date(iso).toLocaleString("en-US", {
month: "short",
day: "numeric",
hour: "2-digit",
minute: "2-digit",
second: "2-digit",
hour12: false,
});
}
export function formatDebugLogDateTime(iso: string): string {
return new Date(iso).toLocaleString("en-US", {
month: "short",
day: "numeric",
year: "numeric",
hour: "numeric",
minute: "2-digit",
second: "2-digit",
});
}
function getLogTypeBadgeProps(logType: string): {
variant: "default" | "secondary" | "destructive" | "outline";
className: string;
} {
switch (logType) {
case "http_request":
return {
variant: "secondary",
className: "bg-blue-500/10 text-blue-400 border-transparent",
};
case "fal_request":
return {
variant: "secondary",
className: "bg-purple-500/10 text-purple-400 border-transparent",
};
case "fal_queue":
return {
variant: "secondary",
className: "bg-fuchsia-500/10 text-fuchsia-400 border-transparent",
};
case "fal_webhook":
return {
variant: "secondary",
className: "bg-amber-500/10 text-amber-400 border-transparent",
};
case "kinovi_request":
return {
variant: "secondary",
className: "bg-cyan-500/10 text-cyan-400 border-transparent",
};
default:
return { variant: "outline", className: "text-muted-foreground" };
}
}
export function getLogLevelBadgeClassName(level: string): string {
switch (level) {
case "error":
return "bg-red-500/10 text-red-400 border-transparent";
case "warn":
return "bg-amber-500/10 text-amber-400 border-transparent";
case "info":
return "bg-sky-500/10 text-sky-400 border-transparent";
case "debug":
return "bg-slate-500/10 text-slate-400 border-transparent";
case "trace":
return "bg-zinc-500/10 text-zinc-400 border-transparent";
default:
return "text-muted-foreground";
}
}
/** Broadcast expand/collapse-all. Bump `generation` to re-apply. */
export interface ExpandAllState {
mode: "expand" | "collapse";
generation: number;
}
export interface DebugLogCardProps {
log: DebugLog;
copiedId: string | null;
onCopy: (value: string, id: string) => void;
/** Start expanded (defaults to true). */
defaultExpanded?: boolean;
/** Expand/collapse-all broadcast from the page. */
expandAll?: ExpandAllState;
}
export function DebugLogCard({
log,
copiedId,
onCopy,
defaultExpanded = true,
expandAll,
}: DebugLogCardProps) {
const maybeUser = log.maybe_user;
const [expanded, setExpanded] = useState(defaultExpanded);
// Apply page-level expand/collapse-all; individual toggles keep working after.
useEffect(() => {
if (expandAll) setExpanded(expandAll.mode === "expand");
}, [expandAll?.generation]);
const badge = getLogTypeBadgeProps(log.debug_log_type);
const copyId = `log_${log.id}`;
return (
<div className="rounded-xl border bg-card text-card-foreground shadow-sm overflow-hidden">
<button
type="button"
onClick={() => setExpanded((v) => !v)}
className="w-full flex items-center gap-3 p-4 hover:bg-muted/30 transition-colors text-left"
>
{expanded ? (
<IconChevronDown className="size-4 shrink-0 text-muted-foreground" />
) : (
<IconChevronRight className="size-4 shrink-0 text-muted-foreground" />
)}
<Badge variant={badge.variant} className={badge.className}>
{humanize(log.debug_log_type)}
</Badge>
{log.maybe_log_level && (
<Badge
variant="secondary"
className={getLogLevelBadgeClassName(log.maybe_log_level)}
>
{log.maybe_log_level.toUpperCase()}
</Badge>
)}
<span className="text-xs text-muted-foreground tabular-nums whitespace-nowrap">
{formatDebugLogDateTimeCompact(log.created_at)}
</span>
{log.maybe_url && (
<span
className="font-mono text-xs text-muted-foreground truncate max-w-[18rem]"
title={log.maybe_url}
>
{log.maybe_url}
</span>
)}
{maybeUser && (
<span
className="inline-flex items-center min-w-0"
onClick={(e) => e.stopPropagation()}
>
<Link
to={`/user/profile/${encodeURIComponent(maybeUser.username)}`}
className="inline-flex items-center gap-1 hover:underline text-foreground/80 text-xs min-w-0"
title={maybeUser.username}
>
<IconUser className="size-3.5 shrink-0" />
<span className="truncate">{maybeUser.display_name}</span>
</Link>
</span>
)}
{log.maybe_ip_address && (
<span
className="font-mono text-xs text-muted-foreground truncate min-w-0 max-w-[10rem]"
title={log.maybe_ip_address}
>
{log.maybe_ip_address}
</span>
)}
<span
className="ml-auto inline-flex items-center gap-1.5 min-w-0"
onClick={(e) => e.stopPropagation()}
>
<Link
to={`/debug_logs/${encodeURIComponent(log.event_token)}`}
className="font-mono text-xs text-muted-foreground hover:text-foreground hover:underline truncate"
title={`View event ${log.event_token}`}
>
{log.event_token}
</Link>
<span className="text-xs text-muted-foreground/60 font-mono whitespace-nowrap">
(#{log.id})
</span>
</span>
<span
className="ml-2 inline-flex items-center"
onClick={(e) => {
e.stopPropagation();
onCopy(log.message, copyId);
}}
>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 text-muted-foreground hover:text-foreground"
title="Copy payload"
>
{copiedId === copyId ? (
<IconCheck className="size-3.5 text-emerald-400" />
) : (
<IconCopy className="size-3.5" />
)}
</Button>
</span>
</button>
{expanded && (
<div className="border-t border-border/50 p-4 flex flex-col gap-3">
{/* Line 1: timestamp, path, ip, event token — all click-to-copy. */}
<div className="flex flex-wrap items-center gap-x-6 gap-y-1 text-xs text-muted-foreground">
<span className="tabular-nums whitespace-nowrap">
{formatDebugLogDateTime(log.created_at)}
</span>
{log.maybe_url && (
<CopyText
label="Path:"
value={log.maybe_url}
display={log.maybe_url}
copyKey={`url_${log.id}`}
copiedId={copiedId}
onCopy={onCopy}
title="Copy URL"
className="font-mono truncate max-w-[32rem]"
/>
)}
{log.maybe_ip_address && (
<CopyText
label="IP:"
value={log.maybe_ip_address}
display={log.maybe_ip_address}
copyKey={`ip_${log.id}`}
copiedId={copiedId}
onCopy={onCopy}
title="Copy IP address"
className="font-mono truncate max-w-[16rem]"
/>
)}
<CopyText
label="Event Token:"
value={log.event_token}
display={log.event_token}
copyKey={`event_${log.id}`}
copiedId={copiedId}
onCopy={onCopy}
title="Copy event token"
className="font-mono truncate"
/>
</div>
{/* Line 2: the user and their event logs. */}
{log.maybe_creator_user_token && (
<div className="flex flex-wrap items-center gap-x-6 gap-y-1 text-xs text-muted-foreground">
<span className="inline-flex items-center gap-1.5 min-w-0">
<span className="whitespace-nowrap">User:</span>
{maybeUser ? (
<Link
to={`/user/profile/${encodeURIComponent(maybeUser.username)}`}
className="inline-flex items-center gap-1.5 text-foreground/80 hover:text-foreground hover:underline min-w-0"
title="View profile"
>
<IconUser className="size-3.5 shrink-0" />
<span className="truncate">{maybeUser.display_name}</span>
</Link>
) : (
<CopyText
value={log.maybe_creator_user_token}
display={log.maybe_creator_user_token}
copyKey={`user_token_${log.id}`}
copiedId={copiedId}
onCopy={onCopy}
title="Copy user token"
className="font-mono truncate"
icon={<IconUser className="size-3.5 shrink-0" />}
/>
)}
</span>
<Link
to={`/debug_logs/user/${encodeURIComponent(log.maybe_creator_user_token)}`}
className="inline-flex items-center gap-1 hover:text-foreground hover:underline shrink-0"
title="View this user's debug logs"
>
<IconListSearch className="size-3.5" />
user event logs
</Link>
</div>
)}
<PrettyPayload raw={log.message} />
</div>
)}
</div>
);
}
interface CopyTextProps {
/** Optional label rendered before the value, eg. "Path:". */
label?: string;
/** The raw value copied to the clipboard. */
value: string;
/** The rendered text (may differ from the copied value). */
display: string;
copyKey: string;
copiedId: string | null;
onCopy: (value: string, id: string) => void;
title: string;
className?: string;
icon?: React.ReactNode;
}
function CopyText({
label,
value,
display,
copyKey,
copiedId,
onCopy,
title,
className,
icon,
}: CopyTextProps) {
return (
<button
type="button"
onClick={() => onCopy(value, copyKey)}
className="inline-flex items-center gap-1 hover:text-foreground cursor-pointer min-w-0"
title={title}
>
{label && <span className="whitespace-nowrap">{label}</span>}
{icon}
<span className={className}>{display}</span>
{copiedId === copyKey ? (
<IconCheck className="size-3 shrink-0 text-emerald-400" />
) : (
<IconCopy className="size-3 shrink-0 opacity-60" />
)}
</button>
);
}
function tryParseJson(raw: string): unknown | undefined {
const trimmed = raw.trim();
if (
!(
(trimmed.startsWith("{") && trimmed.endsWith("}")) ||
(trimmed.startsWith("[") && trimmed.endsWith("]"))
)
) {
return undefined;
}
try {
return JSON.parse(trimmed);
} catch {
return undefined;
}
}
export function PrettyPayload({ raw }: { raw: string }) {
const parsed = useMemo(() => tryParseJson(raw), [raw]);
if (parsed !== undefined) {
return (
<pre className="text-xs font-mono whitespace-pre-wrap wrap-break-word bg-muted/40 p-4 rounded-lg max-h-[600px] overflow-auto text-foreground/90">
{JSON.stringify(parsed, null, 2)}
</pre>
);
}
// Likely a Rust struct debug print or plain text. Re-indent loosely so it's readable.
const formatted = formatRustyDebug(raw);
return (
<pre className="text-xs font-mono whitespace-pre-wrap wrap-break-word bg-muted/40 p-4 rounded-lg max-h-[600px] overflow-auto text-foreground/80">
{formatted}
</pre>
);
}
// Lightweight reformatter for Rust's `Debug` output (e.g. `Foo { a: 1, b: Bar { c: 2 } }`).
// Adds line breaks and indentation around `{`, `}`, `[`, `]`, and top-level commas.
// Falls back gracefully for anything it doesn't understand.
function formatRustyDebug(raw: string): string {
const looksRusty = /[{[].+[}\]]/s.test(raw);
if (!looksRusty) return raw;
let depth = 0;
let out = "";
let inString = false;
let stringChar: '"' | "'" | null = null;
const indent = () => " ".repeat(depth);
for (let i = 0; i < raw.length; i++) {
const ch = raw[i];
const prev = raw[i - 1];
// String tracking (handle escapes)
if (inString) {
out += ch;
if (ch === stringChar && prev !== "\\") {
inString = false;
stringChar = null;
}
continue;
}
if (ch === '"' || ch === "'") {
inString = true;
stringChar = ch;
out += ch;
continue;
}
if (ch === "{" || ch === "[") {
depth++;
out += ch + "\n" + indent();
continue;
}
if (ch === "}" || ch === "]") {
depth = Math.max(0, depth - 1);
out = out.replace(/[ \t]+$/, "");
out += "\n" + indent() + ch;
continue;
}
if (ch === "," && depth > 0) {
out += ",\n" + indent();
// Skip following whitespace
while (i + 1 < raw.length && raw[i + 1] === " ") i++;
continue;
}
out += ch;
}
return out.trim();
}
@@ -71,8 +71,8 @@ const data = {
label: "Oncall",
items: [
{ title: "Page Oncall", url: "/send-pager", icon: IconBellRinging },
{ title: "Debug Logs", url: "/debug_logs/list", icon: IconBug },
{ title: "Search Job by Token", url: "/moderation/job-search", icon: IconBriefcase },
{ title: "Search Debug Logs", url: "/moderation/debug-logs-search", icon: IconBug },
{ title: "Impersonation Logs", url: "/impersonation", icon: IconKey },
{ title: "Staff Audit Logs", url: "/staff-audit-logs", icon: IconClipboardList },
{ title: "Feature Flags", url: "/feature-flags", icon: IconFlag },
@@ -3,71 +3,21 @@ import { useParams, useNavigate } from "react-router-dom";
import { ModerationApi } from "@/api/ModerationApi";
import type { DebugLog } from "@/types";
import { Button } from "@/components/ui/button";
import { Badge } from "@/components/ui/badge";
import { Skeleton } from "@/components/ui/skeleton";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { Input } from "@/components/ui/input";
import { usePageTitle } from "@/hooks/usePageTitle";
import { humanize } from "@/lib/utils";
import { RawTag } from "@/components/RawTag";
import { DebugLogCard } from "@/components/DebugLogCard";
import {
IconAlertCircle,
IconArrowLeft,
IconBug,
IconCheck,
IconCopy,
IconChevronDown,
IconChevronRight,
IconRefresh,
IconSearch,
} from "@tabler/icons-react";
function formatDateTime(iso: string): string {
return new Date(iso).toLocaleString("en-US", {
month: "short",
day: "numeric",
year: "numeric",
hour: "numeric",
minute: "2-digit",
second: "2-digit",
});
}
function getLogTypeBadgeProps(logType: string): {
variant: "default" | "secondary" | "destructive" | "outline";
className: string;
} {
switch (logType) {
case "http_request":
return {
variant: "secondary",
className: "bg-blue-500/10 text-blue-400 border-transparent",
};
case "fal_request":
return {
variant: "secondary",
className: "bg-purple-500/10 text-purple-400 border-transparent",
};
case "fal_queue":
return {
variant: "secondary",
className: "bg-fuchsia-500/10 text-fuchsia-400 border-transparent",
};
case "fal_webhook":
return {
variant: "secondary",
className: "bg-amber-500/10 text-amber-400 border-transparent",
};
case "kinovi_request":
return {
variant: "secondary",
className: "bg-cyan-500/10 text-cyan-400 border-transparent",
};
default:
return { variant: "outline", className: "text-muted-foreground" };
}
}
const DEFAULT_LIMIT = 100;
export function DebugLogs() {
@@ -244,172 +194,3 @@ export function DebugLogs() {
</div>
);
}
interface DebugLogCardProps {
log: DebugLog;
copiedId: string | null;
onCopy: (value: string, id: string) => void;
}
function DebugLogCard({ log, copiedId, onCopy }: DebugLogCardProps) {
const [expanded, setExpanded] = useState(true);
const badge = getLogTypeBadgeProps(log.debug_log_type);
const copyId = `log_${log.id}`;
return (
<div className="rounded-xl border bg-card text-card-foreground shadow-sm overflow-hidden">
<button
type="button"
onClick={() => setExpanded((v) => !v)}
className="w-full flex items-center gap-3 p-4 hover:bg-muted/30 transition-colors text-left"
>
{expanded ? (
<IconChevronDown className="size-4 shrink-0 text-muted-foreground" />
) : (
<IconChevronRight className="size-4 shrink-0 text-muted-foreground" />
)}
<Badge variant={badge.variant} className={badge.className}>
{humanize(log.debug_log_type)}
</Badge>
<RawTag value={log.debug_log_type} />
<span className="text-xs text-muted-foreground tabular-nums whitespace-nowrap">
{formatDateTime(log.created_at)}
</span>
<span className="text-xs text-muted-foreground/60 font-mono ml-auto">
#{log.id}
</span>
<span
className="ml-2 inline-flex items-center"
onClick={(e) => {
e.stopPropagation();
onCopy(log.message, copyId);
}}
>
<Button
variant="ghost"
size="icon"
className="h-7 w-7 text-muted-foreground hover:text-foreground"
title="Copy payload"
>
{copiedId === copyId ? (
<IconCheck className="size-3.5 text-emerald-400" />
) : (
<IconCopy className="size-3.5" />
)}
</Button>
</span>
</button>
{expanded && (
<div className="border-t border-border/50 p-4 flex flex-col gap-3">
{log.maybe_creator_user_token && (
<div className="text-xs text-muted-foreground">
<span className="uppercase tracking-wider font-medium">
User Token:
</span>{" "}
<span className="font-mono text-foreground/80">
{log.maybe_creator_user_token}
</span>
</div>
)}
<PrettyPayload raw={log.message} />
</div>
)}
</div>
);
}
function tryParseJson(raw: string): unknown | undefined {
const trimmed = raw.trim();
if (
!(
(trimmed.startsWith("{") && trimmed.endsWith("}")) ||
(trimmed.startsWith("[") && trimmed.endsWith("]"))
)
) {
return undefined;
}
try {
return JSON.parse(trimmed);
} catch {
return undefined;
}
}
function PrettyPayload({ raw }: { raw: string }) {
const parsed = useMemo(() => tryParseJson(raw), [raw]);
if (parsed !== undefined) {
return (
<pre className="text-xs font-mono whitespace-pre-wrap wrap-break-word bg-muted/40 p-4 rounded-lg max-h-[600px] overflow-auto text-foreground/90">
{JSON.stringify(parsed, null, 2)}
</pre>
);
}
// Likely a Rust struct debug print or plain text. Re-indent loosely so it's readable.
const formatted = formatRustyDebug(raw);
return (
<pre className="text-xs font-mono whitespace-pre-wrap wrap-break-word bg-muted/40 p-4 rounded-lg max-h-[600px] overflow-auto text-foreground/80">
{formatted}
</pre>
);
}
// Lightweight reformatter for Rust's `Debug` output (e.g. `Foo { a: 1, b: Bar { c: 2 } }`).
// Adds line breaks and indentation around `{`, `}`, `[`, `]`, and top-level commas.
// Falls back gracefully for anything it doesn't understand.
function formatRustyDebug(raw: string): string {
const looksRusty = /[{[].+[}\]]/s.test(raw);
if (!looksRusty) return raw;
let depth = 0;
let out = "";
let inString = false;
let stringChar: '"' | "'" | null = null;
const indent = () => " ".repeat(depth);
for (let i = 0; i < raw.length; i++) {
const ch = raw[i];
const prev = raw[i - 1];
// String tracking (handle escapes)
if (inString) {
out += ch;
if (ch === stringChar && prev !== "\\") {
inString = false;
stringChar = null;
}
continue;
}
if (ch === '"' || ch === "'") {
inString = true;
stringChar = ch;
out += ch;
continue;
}
if (ch === "{" || ch === "[") {
depth++;
out += ch + "\n" + indent();
continue;
}
if (ch === "}" || ch === "]") {
depth = Math.max(0, depth - 1);
out = out.replace(/[ \t]+$/, "");
out += "\n" + indent() + ch;
continue;
}
if (ch === "," && depth > 0) {
out += ",\n" + indent();
// Skip following whitespace
while (i + 1 < raw.length && raw[i + 1] === " ") i++;
continue;
}
out += ch;
}
return out.trim();
}
@@ -0,0 +1,289 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { Link } from "react-router-dom";
import { ModerationApi } from "@/api/ModerationApi";
import type { AllDebugLog, DebugLogLevel } from "@/types";
import { Button } from "@/components/ui/button";
import { Badge } from "@/components/ui/badge";
import { Skeleton } from "@/components/ui/skeleton";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { usePageTitle } from "@/hooks/usePageTitle";
import {
DebugLogCard,
getLogLevelBadgeClassName,
type ExpandAllState,
} from "@/components/DebugLogCard";
import {
IconAlertCircle,
IconBug,
IconChevronsDown,
IconChevronsUp,
IconLoader2,
IconRefresh,
IconSearch,
} from "@tabler/icons-react";
const PAGE_SIZE = 50;
const ALL_SEVERITIES: DebugLogLevel[] = [
"error",
"warn",
"info",
"debug",
"trace",
];
export function DebugLogsAll() {
usePageTitle("Debug Logs");
const [logs, setLogs] = useState<AllDebugLog[]>([]);
const [nextCursor, setNextCursor] = useState<number | null>(null);
const [isLoading, setIsLoading] = useState(true);
const [isLoadingMore, setIsLoadingMore] = useState(false);
const [error, setError] = useState<string | null>(null);
const [copiedId, setCopiedId] = useState<string | null>(null);
const [expandAll, setExpandAll] = useState<ExpandAllState | undefined>();
const broadcastExpandAll = (mode: ExpandAllState["mode"]) => {
setExpandAll((prev) => ({ mode, generation: (prev?.generation ?? 0) + 1 }));
};
const [severities, setSeverities] = useState<DebugLogLevel[]>([]);
const observerRef = useRef<IntersectionObserver | null>(null);
const sentinelRef = useRef<HTMLDivElement | null>(null);
const loadingRef = useRef(false);
const copy = (value: string, id: string) => {
navigator.clipboard.writeText(value).then(() => {
setCopiedId(id);
setTimeout(() => setCopiedId((c) => (c === id ? null : c)), 2000);
});
};
const loadData = useCallback(
async (cursor: number | null, append: boolean, levels: DebugLogLevel[]) => {
if (loadingRef.current) return;
loadingRef.current = true;
if (append) {
setIsLoadingMore(true);
} else {
setIsLoading(true);
setError(null);
}
try {
const api = new ModerationApi();
const resp = await api.ListAllDebugLogs({
severities: levels,
cursor: cursor ?? undefined,
limit: PAGE_SIZE,
});
if (resp.success && resp.data) {
const page = resp.data.debug_logs;
setLogs((prev) => (append ? [...prev, ...page] : page));
setNextCursor(resp.data.next_cursor);
} else {
setError(resp.errorMessage || "Failed to load debug logs");
}
} catch (err: any) {
setError(err.message || "Failed to load debug logs");
} finally {
setIsLoading(false);
setIsLoadingMore(false);
loadingRef.current = false;
}
},
[],
);
// Initial load + reload on severity filter change.
useEffect(() => {
setLogs([]);
setNextCursor(null);
loadData(null, false, severities);
}, [severities, loadData]);
// Infinite scroll sentinel.
useEffect(() => {
observerRef.current?.disconnect();
if (nextCursor === null) return;
observerRef.current = new IntersectionObserver(
(entries) => {
if (entries[0].isIntersecting) {
loadData(nextCursor, true, severities);
}
},
{ threshold: 0.1 },
);
if (sentinelRef.current) observerRef.current.observe(sentinelRef.current);
return () => observerRef.current?.disconnect();
}, [nextCursor, severities, loadData]);
const toggleSeverity = (level: DebugLogLevel) => {
setSeverities((prev) =>
prev.includes(level)
? prev.filter((l) => l !== level)
: [...prev, level],
);
};
return (
<div className="flex flex-col gap-8 pb-10">
{/* Header */}
<div className="flex items-center gap-4">
<div className="flex-1 min-w-0">
<h1 className="text-3xl font-bold tracking-tight flex items-center gap-3">
<IconBug className="size-6 text-muted-foreground" />
Debug Logs
</h1>
<p className="text-muted-foreground text-sm mt-1">
All debug logs across the system, most recent first.
</p>
</div>
<div className="flex gap-2">
<Button variant="outline" size="sm" asChild>
<Link
to="/moderation/debug-logs-search"
title="Search debug logs by event token"
>
<IconSearch className="size-4" />
Search
</Link>
</Button>
<Button
variant="outline"
size="sm"
onClick={() => loadData(null, false, severities)}
disabled={isLoading}
>
<IconRefresh className="size-4" />
Refresh
</Button>
</div>
</div>
{/* Severity filter */}
<div className="flex flex-wrap items-center gap-2">
<span className="text-xs uppercase tracking-wider text-muted-foreground font-medium">
Severity:
</span>
{ALL_SEVERITIES.map((level) => {
const active = severities.includes(level);
return (
<button
key={level}
type="button"
onClick={() => toggleSeverity(level)}
className="cursor-pointer"
>
<Badge
variant={active ? "secondary" : "outline"}
className={
active
? getLogLevelBadgeClassName(level) + " ring-1 ring-current"
: "text-muted-foreground/60 hover:text-muted-foreground"
}
>
{level.toUpperCase()}
</Badge>
</button>
);
})}
{severities.length > 0 && (
<Button
variant="ghost"
size="sm"
className="h-6 px-2 text-xs text-muted-foreground"
onClick={() => setSeverities([])}
>
Clear
</Button>
)}
</div>
{error && !isLoading && (
<Alert variant="destructive" className="max-w-xl">
<IconAlertCircle className="size-4" />
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
{isLoading && (
<div className="flex flex-col gap-3">
{Array.from({ length: 5 }).map((_, i) => (
<div
key={i}
className="rounded-xl border bg-card shadow-sm flex flex-col p-4 gap-3"
>
<div className="flex items-center gap-3">
<Skeleton className="h-5 w-24 rounded-full" />
<Skeleton className="h-4 w-40" />
</div>
<Skeleton className="h-14 w-full" />
</div>
))}
</div>
)}
{!isLoading && logs.length === 0 && !error && (
<div className="flex flex-col items-center justify-center py-20 bg-muted/20 border border-dashed rounded-xl">
<IconBug className="size-10 text-muted-foreground mb-4 opacity-20" />
<p className="text-muted-foreground">
{severities.length > 0
? "No debug logs match the selected severities."
: "No debug logs found."}
</p>
</div>
)}
{/* Sticky expand/collapse toolbar */}
<div className="sticky top-0 z-20 -mx-4 lg:-mx-6 px-4 lg:px-6 py-2 -my-4 bg-background/80 backdrop-blur-md border-b flex items-center gap-2">
<span className="text-xs text-muted-foreground mr-1">
{logs.length} log{logs.length === 1 ? "" : "s"}
</span>
<Button
variant="outline"
size="sm"
className="h-7"
onClick={() => broadcastExpandAll("expand")}
>
<IconChevronsDown className="size-4" />
Expand all
</Button>
<Button
variant="outline"
size="sm"
className="h-7"
onClick={() => broadcastExpandAll("collapse")}
>
<IconChevronsUp className="size-4" />
Collapse all
</Button>
</div>
{!isLoading && logs.length > 0 && (
<div className="flex flex-col gap-3">
{logs.map((log) => (
<DebugLogCard
key={log.id}
log={log}
copiedId={copiedId}
onCopy={copy}
defaultExpanded={false}
expandAll={expandAll}
/>
))}
</div>
)}
{/* Infinite scroll sentinel */}
{nextCursor !== null && !isLoading && (
<div ref={sentinelRef} className="flex justify-center py-6">
{isLoadingMore && (
<IconLoader2 className="size-5 animate-spin text-muted-foreground" />
)}
</div>
)}
</div>
);
}
@@ -13,7 +13,7 @@ export function DebugLogsSearch() {
buttonLabel="Open"
icon={IconBug}
buildHref={(token) =>
`/moderation/debug-logs/${encodeURIComponent(token)}`
`/debug_logs/${encodeURIComponent(token)}`
}
/>
);
@@ -314,7 +314,7 @@ export function JobInfo() {
{job.maybe_debug_log_event_token && (
<Button variant="default" size="sm" asChild>
<Link
to={`/moderation/debug-logs/${encodeURIComponent(
to={`/debug_logs/${encodeURIComponent(
job.maybe_debug_log_event_token,
)}`}
>
@@ -0,0 +1,274 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { Link, useNavigate, useParams } from "react-router-dom";
import { ModerationApi } from "@/api/ModerationApi";
import type { DebugLog } from "@/types";
import { Button } from "@/components/ui/button";
import { Skeleton } from "@/components/ui/skeleton";
import { Alert, AlertDescription } from "@/components/ui/alert";
import { usePageTitle } from "@/hooks/usePageTitle";
import { DebugLogCard, type ExpandAllState } from "@/components/DebugLogCard";
import {
IconAlertCircle,
IconArrowLeft,
IconBug,
IconChevronsDown,
IconChevronsUp,
IconLoader2,
IconRefresh,
IconUser,
} from "@tabler/icons-react";
const PAGE_SIZE = 50;
interface UserHeaderInfo {
display_name: string;
username: string;
}
export function UserDebugLogs() {
const { userToken } = useParams<{ userToken: string }>();
const navigate = useNavigate();
const [user, setUser] = useState<UserHeaderInfo | null>(null);
const [logs, setLogs] = useState<DebugLog[]>([]);
const [nextCursor, setNextCursor] = useState<number | null>(null);
const [isLoading, setIsLoading] = useState(true);
const [isLoadingMore, setIsLoadingMore] = useState(false);
const [error, setError] = useState<string | null>(null);
const [copiedId, setCopiedId] = useState<string | null>(null);
const [expandAll, setExpandAll] = useState<ExpandAllState | undefined>();
const broadcastExpandAll = (mode: ExpandAllState["mode"]) => {
setExpandAll((prev) => ({ mode, generation: (prev?.generation ?? 0) + 1 }));
};
usePageTitle(
user ? `Debug Logs · ${user.display_name}` : "User Debug Logs",
);
const observerRef = useRef<IntersectionObserver | null>(null);
const sentinelRef = useRef<HTMLDivElement | null>(null);
const loadingRef = useRef(false);
const copy = (value: string, id: string) => {
navigator.clipboard.writeText(value).then(() => {
setCopiedId(id);
setTimeout(() => setCopiedId((c) => (c === id ? null : c)), 2000);
});
};
const loadLogs = useCallback(
async (cursor: number | null, append: boolean) => {
if (!userToken || loadingRef.current) return;
loadingRef.current = true;
if (append) {
setIsLoadingMore(true);
} else {
setIsLoading(true);
setError(null);
}
try {
const api = new ModerationApi();
const resp = await api.ListUserDebugLogs(
userToken,
cursor ?? undefined,
PAGE_SIZE,
);
if (resp.success && resp.data) {
const page = resp.data.debug_logs;
setLogs((prev) => (append ? [...prev, ...page] : page));
setNextCursor(resp.data.next_cursor);
} else {
setError(resp.errorMessage || "Failed to load debug logs");
}
} catch (err: any) {
setError(err.message || "Failed to load debug logs");
} finally {
setIsLoading(false);
setIsLoadingMore(false);
loadingRef.current = false;
}
},
[userToken],
);
// Load logs + basic user info (for the header) on mount.
useEffect(() => {
setLogs([]);
setNextCursor(null);
setUser(null);
loadLogs(null, false);
if (userToken) {
const api = new ModerationApi();
api.UserLookup(userToken).then((resp) => {
if (resp.success && resp.data?.maybe_user) {
setUser({
display_name: resp.data.maybe_user.display_name,
username: resp.data.maybe_user.username,
});
}
});
}
}, [userToken, loadLogs]);
// Infinite scroll sentinel.
useEffect(() => {
observerRef.current?.disconnect();
if (nextCursor === null) return;
observerRef.current = new IntersectionObserver(
(entries) => {
if (entries[0].isIntersecting) {
loadLogs(nextCursor, true);
}
},
{ threshold: 0.1 },
);
if (sentinelRef.current) observerRef.current.observe(sentinelRef.current);
return () => observerRef.current?.disconnect();
}, [nextCursor, loadLogs]);
return (
<div className="flex flex-col gap-8 pb-10">
{/* Header */}
<div className="flex items-center gap-4">
<Button
variant="ghost"
size="icon"
onClick={() => navigate(-1)}
className="p-0! h-auto w-auto hover:bg-transparent! text-foreground/70 hover:text-foreground/50"
>
<IconArrowLeft className="size-6" />
</Button>
<div className="flex-1 min-w-0">
<h1 className="text-3xl font-bold tracking-tight flex items-center gap-3">
<IconBug className="size-6 text-muted-foreground" />
User Debug Logs
</h1>
<p className="text-muted-foreground text-sm mt-1 flex items-center gap-2 min-w-0">
{user ? (
<>
<Link
to={`/user/profile/${encodeURIComponent(user.username)}`}
className="inline-flex items-center gap-1.5 hover:underline text-foreground/80 min-w-0"
title="View profile"
>
<IconUser className="size-4 shrink-0" />
<span className="truncate">{user.display_name}</span>
</Link>
<span className="font-mono text-muted-foreground/70 truncate">
{userToken}
</span>
</>
) : (
<span className="font-mono truncate">{userToken}</span>
)}
</p>
</div>
<div className="flex gap-2">
{user && (
<Button variant="outline" size="sm" asChild>
<Link to={`/user/profile/${encodeURIComponent(user.username)}`}>
<IconUser className="size-4" />
Profile
</Link>
</Button>
)}
<Button
variant="outline"
size="sm"
onClick={() => loadLogs(null, false)}
disabled={isLoading}
>
<IconRefresh className="size-4" />
Refresh
</Button>
</div>
</div>
{error && !isLoading && (
<Alert variant="destructive" className="max-w-xl">
<IconAlertCircle className="size-4" />
<AlertDescription>{error}</AlertDescription>
</Alert>
)}
{isLoading && (
<div className="flex flex-col gap-3">
{Array.from({ length: 5 }).map((_, i) => (
<div
key={i}
className="rounded-xl border bg-card shadow-sm flex flex-col p-4 gap-3"
>
<div className="flex items-center gap-3">
<Skeleton className="h-5 w-24 rounded-full" />
<Skeleton className="h-4 w-40" />
</div>
<Skeleton className="h-14 w-full" />
</div>
))}
</div>
)}
{!isLoading && logs.length === 0 && !error && (
<div className="flex flex-col items-center justify-center py-20 bg-muted/20 border border-dashed rounded-xl">
<IconBug className="size-10 text-muted-foreground mb-4 opacity-20" />
<p className="text-muted-foreground">
No debug logs found for this user.
</p>
</div>
)}
{/* Sticky expand/collapse toolbar */}
<div className="sticky top-0 z-20 -mx-4 lg:-mx-6 px-4 lg:px-6 py-2 -my-4 bg-background/80 backdrop-blur-md border-b flex items-center gap-2">
<span className="text-xs text-muted-foreground mr-1">
{logs.length} log{logs.length === 1 ? "" : "s"}
</span>
<Button
variant="outline"
size="sm"
className="h-7"
onClick={() => broadcastExpandAll("expand")}
>
<IconChevronsDown className="size-4" />
Expand all
</Button>
<Button
variant="outline"
size="sm"
className="h-7"
onClick={() => broadcastExpandAll("collapse")}
>
<IconChevronsUp className="size-4" />
Collapse all
</Button>
</div>
{!isLoading && logs.length > 0 && (
<div className="flex flex-col gap-3">
{logs.map((log) => (
<DebugLogCard
key={log.id}
log={log}
copiedId={copiedId}
onCopy={copy}
defaultExpanded={false}
expandAll={expandAll}
/>
))}
</div>
)}
{/* Infinite scroll sentinel */}
{nextCursor !== null && !isLoading && (
<div ref={sentinelRef} className="flex justify-center py-6">
{isLoadingMore && (
<IconLoader2 className="size-5 animate-spin text-muted-foreground" />
)}
</div>
)}
</div>
);
}
@@ -25,6 +25,7 @@ import {
IconMail,
IconUser,
IconShieldLock,
IconBug,
IconChartLine,
IconCoin,
IconReceipt2,
@@ -568,6 +569,12 @@ export function UserProfile() {
Spend History
</Link>
</Button>
<Button variant="outline" size="sm" asChild>
<Link to={`/debug_logs/user/${user.token}`}>
<IconBug className="size-4" />
User Debug Logs
</Link>
</Button>
<Button
variant="outline"
size="sm"
@@ -111,15 +111,37 @@ export type DebugLogType =
| "kinovi_request"
| (string & {});
export type DebugLogLevel =
| "info"
| "warn"
| "error"
| "debug"
| "trace"
| (string & {});
export interface DebugLog {
created_at: string;
debug_log_type: DebugLogType;
event_token: string;
id: number;
maybe_log_level: DebugLogLevel | null;
maybe_creator_user_token: string | null;
maybe_ip_address: string | null;
maybe_url: string | null;
message: string;
maybe_user: DebugLogUser | null;
}
/** Creator user info joined onto a debug log row. */
export interface DebugLogUser {
user_token: string;
display_name: string;
username: string;
gravatar_hash: string;
}
export type AllDebugLog = DebugLog;
export interface SignupUser {
created_at: string;
display_name: string;
+1 -1
View File
@@ -24,7 +24,7 @@ if (import.meta.env.DEV) {
// (disabled — it overrides the origin above and points at localhost:12345,
// which breaks dev unless a local backend is running. Re-enable only when
// testing against a local storyteller-web.)
//StorytellerApiHostStore.getInstance().setDevelopment();
// StorytellerApiHostStore.getInstance().setDevelopment();
} catch (e) {
console.warn("Failed to set dev API host override", e);
}
@@ -1,4 +1,4 @@
import { useCallback, useEffect, useRef } from "react";
import { useCallback, useEffect, useRef, useState } from "react";
import {
Link,
useNavigate,
@@ -10,6 +10,7 @@ import { faDesktop, faHouse } from "@fortawesome/pro-solid-svg-icons";
import { Button } from "@storyteller/ui-button";
import { MediaFilesApi } from "@storyteller/api";
import { Stage3D, usePageSceneStore } from "@storyteller/ui-pagescene";
import { Lightbox } from "../../components/lightbox/lightbox";
import {
GalleryModal,
GalleryDragComponent,
@@ -239,6 +240,41 @@ function PageSceneEditor() {
navigate("/create-image");
}, [navigate]);
// Record-mode handoff: open the app Lightbox on an uploaded media token. The
// Lightbox offers every destination for the media kind (Edit-on-Canvas,
// Make-Video, Recreate, Share, Download). Resolve the CDN URL from the token
// so the preview + actions have a real URL to work with.
const [lightbox, setLightbox] = useState<{
token: string;
cdnUrl: string;
mediaClass: string;
} | null>(null);
const openMediaLightbox = useCallback(
(token: string, kind: "image" | "video") => {
void (async () => {
let cdnUrl = "";
try {
const resp = await new MediaFilesApi().GetMediaFileByToken({
mediaFileToken: token,
});
const links = (
resp?.data as { media_links?: { cdn_url?: string } } | undefined
)?.media_links;
cdnUrl = links?.cdn_url ?? "";
} catch {
// ignore — open anyway; the sidebar metadata still loads by token
}
setLightbox({
token,
cdnUrl,
mediaClass: kind === "video" ? "video" : "image",
});
})();
},
[],
);
const wrapperRef = useRef<HTMLDivElement | null>(null);
useEffect(() => {
@@ -272,6 +308,7 @@ function PageSceneEditor() {
userToken: user?.user_token,
initialSceneToken: sceneToken,
navigateToImageTo3D,
openMediaLightbox,
getViewportSize,
promptSignup: openSignupCta,
onRequestNewSceneSelector: openSceneSplash,
@@ -328,6 +365,16 @@ function PageSceneEditor() {
adds. Both components portal themselves out of this wrapper. */}
<GalleryModal mode="view" />
<GalleryDragComponent />
{/* Capture/Record handoff: the app Lightbox opens on the uploaded token
with all destinations for the media kind. */}
<Lightbox
isOpen={!!lightbox}
onClose={() => setLightbox(null)}
mediaToken={lightbox?.token}
cdnUrl={lightbox?.cdnUrl}
mediaClass={lightbox?.mediaClass}
showBatchCarousel={false}
/>
{demoToken && (
<DemoOutputOverlay imageToken={imageToken} videoToken={videoToken} />
)}
@@ -19,6 +19,7 @@ import {
UsersApi,
} from "@storyteller/api";
import type { PageSceneAdapter } from "@storyteller/ui-pagescene";
import { uploadByKind } from "../video-editor/adapters/upload-by-kind";
import {
getActiveEditor,
ToastTypes,
@@ -155,6 +156,10 @@ export interface WebAppPageSceneAdapterOptions {
userToken: string | undefined;
initialSceneToken: string | undefined;
navigateToImageTo3D: () => void;
// Record-mode handoff: open the app Lightbox on an uploaded media token
// (all destinations for the kind). Built page-side (needs local state +
// navigate), passed through here like navigateToImageTo3D.
openMediaLightbox: (token: string, kind: "image" | "video") => void;
// Wrapper size — kept in a ref so the closure sees live values without
// rebuilding the adapter on every resize.
getViewportSize: () => { width: number; height: number };
@@ -173,6 +178,7 @@ export const useWebAppPageSceneAdapter = (
userToken,
initialSceneToken,
navigateToImageTo3D,
openMediaLightbox,
getViewportSize,
promptSignup,
onRequestNewSceneSelector,
@@ -386,6 +392,13 @@ export const useWebAppPageSceneAdapter = (
navigateToImageTo3D,
openMediaLightbox,
// Persist a produced still/video to the media library. Images
// auto-upload after Capture; videos upload on demand. Returns the token.
uploadMedia: ({ kind, blob, fileName, title }) =>
uploadByKind({ kind, blob, fileName, title }),
promptSignup,
onRequestNewSceneSelector,
@@ -425,6 +438,7 @@ export const useWebAppPageSceneAdapter = (
userToken,
initialSceneToken,
navigateToImageTo3D,
openMediaLightbox,
getViewportSize,
promptSignup,
onRequestNewSceneSelector,
@@ -0,0 +1,368 @@
# CLAUDE.md — pagescene (3D Scene Editor)
Guidance for working in the `pagescene` library — the 3D editor (a.k.a. "pagescene" / "Stage3D").
## ⚠️ Engine-canvas mounting invariant (footgun)
`EngineProvider`'s editor-construction effect depends on three DOM nodes: `sceneContainer`,
`editorCanvas`, and **`camViewCanvas`** (owned by `PreviewEngineCamera` → `CameraViewCanvas`).
If ANY of these unmounts, the effect tears down + recreates the whole `Editor` (serializing to
cache, then reloading) — which wipes a fresh/unsaved scene. **Never conditionally unmount the
components that host these canvases** (`SceneContainer`, `EditorCanvas`, `PreviewEngineCamera`)
for mode/chrome gating — hide them with CSS (`hidden`) instead. (Regression fixed: record mode
had gated `PreviewEngineCamera` behind `!isRecord`, wiping the scene on record→build toggle.)
## Behavior notes (fixes)
- **Record mode is fully immutable.** The viewport lock (`CameraController.locked`, set by the
record effect) gates every camera-move path: FreeCam drag/wheel/keyboard (`useFreeCam`),
FreeCam integration (`tickPerFrame`), AND `MouseControls` mouse-look + `focus()` (gated via
`MouseControlsDeps.isViewportLocked`). If you add a new camera-move path, gate it on `locked`.
- **Default camera framing** (`DEFAULT_CAMERAS.main`): a Blender-style pulled-back/elevated 3/4
view (`pos (-4.5,4,6)`, `lookAt (0,0.5,0.6)`) so the whole render-camera (`cam2`) frustum is
visible on a blank scene.
- **Auto-keyframe**: mutating an **already-keyframed** object (gizmo drag-end or inspector commit)
calls `TimelineController.autoKeyIfTracked(uuid)` → `addKeyframe` at the playhead, which
replace-or-creates a keyframe at the current scrub point. No-op for never-keyframed objects
(first keyframe is always explicit).
## Architecture (existing, do not rebuild)
Strict one-way data flow. Do not violate it.
```
Three.js engine → typed event bus → EngineStoreBridge → Zustand store → React → actions → engine methods
```
- **`Stage3D.tsx`** — public entry component; takes an `adapter` + optional slots/callbacks.
- **`Stage3DBody.tsx`** — layout composition (canvas, panels, toolbars, modals). Mounts the external `PromptBox3D`.
- **`PageSceneStore.ts`** — the single Zustand store (~60 fields). All UI state. The engine never imports this store.
- **`adapter.ts`** — platform-abstraction interface. All host-specific logic (HTTP, upload, auth, navigation, modals) flows through `PageSceneAdapter`. Hosts (Tauri app, web app) inject it at mount.
- **`comps/Controls3D/Controls3D.tsx`** — top-center toolbar: `+` add, Move/Rotate/Scale (`transformMode`), World/Local (`transformSpace`).
- **`comps/Outliner/Outliner.tsx`** — left "Scene" panel (code name: Outliner). Driven by `outlinerItems`; each `OutlinerRow` has lock + visibility toggles.
- **`engine/editor.ts`** — main `Editor` class; owns subsystems (GizmoController, CameraController, HistoryManager, SelectionBridge) and the prompt text (`editor.positive_prompt`).
- **Undo/redo** — `HistoryManager` + command-pattern `UndoableAction` classes. Every mutation that should be undoable records an action.
Key boundary: the **bottom prompt bar is NOT in this library**. It is `PromptBox3D` from the external `@storyteller/ui-promptbox` package, mounted by `Stage3DBody`. Anything that changes the prompt bar crosses the pagescene ↔ ui-promptbox boundary.
---
## Feature: Scene Builder (IN PROGRESS — branch `feature/scene-builder`)
Status: **spec / in development.** This section describes intended behavior; not all of it is built yet. Images referenced are from Figma design frames.
### Top-level mode pill (net-new)
A segmented toggle at top-center (above the `Controls3D` toolbar) with two modes:
- **Build** — default. Scene composition/layout. Used unless the user switches.
- **Record** — read-only playback/output mode (see below).
Rule: a scene with **no animations/keyframes (static)** defaults to **Build**.
This mode does not exist in code today. It needs a new store field (e.g. `sceneMode: "build" | "record"`) and a new component rendered in `Stage3DBody`. Distinct from the existing `editorState` enum (`EDIT` | `CAMERA_VIEW`), which is a different axis — don't overload it.
### Launch re-scope — drop promptbox, lightbox handoff, editable duration
- **Promptbox removed**: `SceneBuilderPromptBox` deleted (MCP not ready). The **animation
timeline** is the sole build-mode bottom UI: collapsed `TimelineBar` by default, expanded
`TimelineEditor` via chevron. An empty timeline is auto-created on editor init
(`editor.timelineController.create()`), so the bar is always functional (no "Add timeline"
button). Auto-expands once on load when `timelineTracks.length > 0` or
`editor.sceneHasAnimation()` (imported THREE clips).
- **Editable max duration**: `TimelineController.setDuration(s)` (clamped `[1,60]`, covers 5–30s) +
`comps/Timeline/DurationLabel.tsx` (click the total-time → number input). Persists via `getTimeline`.
- **Capture/Record → review modal (appears first) → app Lightbox**:
- `CompletionModal` opens **immediately** for both kinds (`producedArtifact`) with a **local
preview** (object URL, works offline) + upload status. Images **auto-upload** on open; videos
show a manual **Upload** button (large). On success → `adapter.openMediaLightbox(token, kind)`
(destinations) + close; on failure → **Retry** (so the modal is always visible even when the
upload API is unreachable in dev). The modal — not `RenderOverlay` — owns the upload.
- `RecordControls`: Capture → produce image `producedArtifact`; Record → encode (overlay) →
produce video `producedArtifact`.
- Adapter: `openMediaLightbox(token, kind)` (host resolves cdnUrl via `MediaFilesApi` and renders
the app `<Lightbox>`); `uploadMedia` kept; `openImageInEditor`/`openVideoInEditor` **removed**
(the Lightbox owns destination routing — Edit-on-Canvas/Make-Video/Recreate/Share/Download).
- Uploading first means the Lightbox gets a real token + cdnUrl → its actions route correctly
(retires the earlier blob/data-URL handoff fragility).
### Spec revision — 3D delegates generation to 2D/Video (branch `feature/scene-builder`)
The 3D editor no longer generates. It composes; 2D (pagedraw) and the video experiences generate.
- **Build promptbox is now the scene-builder/MCP tool only** — `comps/SceneBuilderPromptBox/`
replaced `PromptBox3D` in `Stage3DBody` (the Manual/Prompted toggle + `buildMode` store field are
gone; `PromptBox3D` is no longer mounted by 3D). Update is still an MCP stub (records prompt +
history on the engine). Hosts the collapsed `TimelineBar` + Add-animation-timeline affordance.
- **Record mode = Capture (still) / Record (timeline → video)**:
- `engine/recording/TimelineRecorder.ts` — deterministic `seekTo`→`renderScene`→`CanvasSource.add`
encode via **mediabunny** (already a dep). Capture reuses `snapShotOfCurrentFrame(false)`.
- `comps/RenderOverlay/` (opaque `LoadingDots` + progress) covers the viewport while encoding.
- Output cached locally as `producedArtifact` in the store (`{kind, blob, objectUrl, ...}`) — no
upload from 3D.
- **`comps/CompletionModal/`** — preview hub: image full-preview / video playback + **Delete**
(revokes URL), **Upload** (`adapter.uploadMedia` → library token), **Edit** → 2D for images,
Create-Video / Video-editor for videos.
- **Adapter handoff (new)** in `adapter.ts`: `openImageInEditor`, `openVideoInEditor(target)`,
`uploadMedia` + `PageSceneArtifact`. Host-wired in `apps/artcraft-webapp/.../pagescene` :
image → eager-upload + `applyEditOnCanvasFromImage` (`/edit-image`); video "generate" →
`useCreateVideoStore.setPendingRecreate` reference-video (lazy upload at generate, `/create-video`);
video "edit" → upload + `/video-editor` (TODO: auto-drop on timeline).
- **Camera-view regression fixes** (Phase 0): Esc + viewport double-click exit; double-click the
frustum to enter; robust enter/exit pose handling (removed the dead `hot_items` toggle);
frustum **pick-proxy mesh** (line raycasting disabled) so selection matches the wireframe;
`CameraStatusPill` (top-left) shows Viewport vs render-camera and toggles.
Deferred: real MCP backend; pagedraw file-only (lazy) base image; video-editor timeline
auto-import; audio in recordings; `snapShotOfCurrentFrame` render-camera reconciliation.
### Persistence + undo (timeline & split camera)
- **Timeline persists in scene JSON**: `save_manager.getSceneJson` now writes `timeline:
getTimeline() ?? null` (was a `""` stub); `loadFromJson` restores via `loadTimeline` (guards
legacy `""`/null). Flows through both backend save and the unmount→cache→remount roundtrip.
Loaded objects keep their saved `object_uuid` (proxy load, `obj.uuid = json_object.object_uuid`),
so timeline tracks (keyed by uuid) resolve.
- **Render camera persists**: at save, the render-camera entry in `cameras[]` is overridden with the
**live `::CAM::` transform** (`getRenderCameraTransform` dep) so it reflects gizmo moves (the store
config otherwise goes stale). The load path strips `::CAM::` and never recreated it — fixed:
`loadFromJson` calls `recreateCameraObject()` (→ `Scene._create_camera_obj()`, now idempotent)
AFTER `CamerasReplacedEvent`, so the frustum lands at the restored transform. This also fixes the
standalone "load loses the render camera" bug.
- **Undo/redo**: timeline is **transactional** — `Save` is one undo step (`SaveTimelineAction`;
`loadTimeline` re-seeks + emits on undo/redo); `Cancel` reverts to last saved (not history).
Render-camera gizmo moves were already undoable (`TransformAction` on `::CAM::`, no special-casing);
because save reads the live `cam_obj`, a save after an undo persists the undone position.
Per-edit timeline undo remains deferred.
### Implementation status (branch `feature/scene-builder`)
- **DONE — Build/Record pill**: `comps/SceneModePill/` (reuses `@storyteller/ui-tab-selector`), mounted above `Controls3D` in `Stage3DBody`. Store: `sceneMode` (default `build`) + `setSceneMode`.
- **DONE — Manual/Prompted toggle + prompted layout**: store `buildMode: "manual" | "prompted"` (default `manual`) + `setBuildMode`. The toggle + prompted layout live in `PromptBox3D` (`@storyteller/ui-promptbox`), gated by a new optional `buildMode` / `onBuildModeChange` / `onAddTimeline` prop trio passed from `Stage3DBody`. Toggle = a `ButtonIconSelect` (Manual = pointer icon, Prompted = wand-sparkles) in a floating glass pill above the prompt card. Prompted layout hides the manual toolbar and shows: textarea + "Add animation timeline" (`onAddTimeline`, stubbed) + prompt-history popover (`usePrompt3DStore.promptHistory`) + **Update**. `handlePromptedUpdate` is a **stub**: records the prompt on the engine + pushes to history + toasts; it does NOT call the MCP backend yet.
- **DONE — Animation timeline (in-memory)**: keyframe system + playback + full UI.
- Data model: `engine/timeline/types.ts` (`Keyframe`/`TimelineTrack`/`TimelineData`, `EasingSpec` = cubic-bezier ctrl pts, presets) + `interpolation.ts` (`cubicBezierYForX`, `sampleTrackAt`). Reuses `TransformSnap`/`snapshotTransform`/`writeTransform`.
- `engine/editor/TimelineController.ts` owns the timeline; ticked in `editor.ts` `renderSingleFrame()` after `entranceAnimator`. **Writes transforms only while playing or on seek** (idle never touches objects, so manual editing is safe).
- Events `TimelineChangedEvent`/`TimelinePlayheadEvent` → `EngineStoreBridge` → store fields (`timelineExists/Expanded/Playhead/IsPlaying/Duration/Tracks/SelectedKeyframeId`). Dispatchers in `actions/timeline.ts`.
- Undo: `engine/editor/actions/SaveTimelineAction.ts`; `Save` records it, `Cancel` reverts to last saved.
- UI in `comps/Timeline/`: `TimelineBar` (collapsed, via promptbox `aboveStackSlot`), `TimelineEditor` (expanded, replaces the prompt box; `Stage3DBody` hides `PromptBox3D` when `timelineExpanded`), `TimelineTrackRow` (diamonds, drag, add), `MotionPopover` (presets + draggable cubic-bezier). `onAddTimeline` → `createTimeline` + expand; `showAddTimelineButton={!timelineExists}` hides the promptbox button once a timeline exists.
- **In-memory only** — not persisted to scene JSON yet (reload clears). Rotation is euler-lerped (quaternion slerp is a possible refinement). Camera track is absent until the camera-split slice creates `::CAM::`.
- **DONE — Camera split (single camera) + camera view**: the render camera is now a selectable
wireframe-frustum placeholder in the scene.
- `scene.ts` `_create_camera_obj()` rewritten to build a `THREE.Group` named `"::CAM::"` (line
frustum, layer 1 = editor-only/excluded from render), positioned at the first non-`main`
store camera; called in `initialize()`/`clear()`; `refreshCamObj` resolves it. No
`userData.media_id` → the save path (`saveToScene`, which only serializes `media_id` children)
skips it, so scenes aren't corrupted.
- Enter/exit camera view reuses the existing `CameraController.switchCameraView()` +
`tickPerFrame` sync (canvas renders through the viewport camera, which is moved to the render
camera — no render-path change). Trigger: new `CameraViewToggleRequestedEvent` → `editor.ts`
subscription → `switchCameraView()`. Emitted by (a) `scene_manager_api.ts` `double_click()`
when the selected object is `::CAM::` (covers viewport + outliner-row double-click), and
(b) `actions/cameraView.ts` `toggleCameraView` (outliner icon + exit button).
- Outliner: `OutlinerItem.isCamera` (set in `convert_object`) → `Outliner.tsx` renders a
view-from-camera button (`faArrowRightToBracket`) left of the lock icon.
- `comps/ExitCameraView/` shows the "EXIT CAMERA VIEW" pill while `editorState === CAMERA_VIEW`.
- **DONE — Record mode**: `sceneMode === "record"` is read-only playback through the render camera.
- `Stage3DBody` effect on `isRecord`: `cameraController.enterCameraView()` + `setLocked(true)` on
entry; `setLocked(false)` + `exitCameraView()` on exit. `switchCameraView()` was refactored into
idempotent `enterCameraView()`/`exitCameraView()`; new `CameraController.locked` gates the
FreeCam integration in `tickPerFrame` (viewport can't be flown in record).
- Build chrome hidden in record (ControlsTopButtons, Controls3D, top-right cluster, Outliner,
PreviewEngineCamera, ControlPanelSceneObject, PromptBox3D, TimelineEditor). The **mode pill
stays**. `ExitCameraView` is suppressed in record (the pill leaves record).
- `comps/RecordControls/`: bottom-center read-only `TimelineBar` (`readOnly` hides the expand
chevron — playback only, no editing) + **Capture** (`editor.snapShotOfCurrentFrame(true)` — still
image download) and **Record** (red; **stub** — no video pipeline in pagescene yet).
- **NOT STARTED**: timeline persistence to scene JSON; natural-language motion clips; multi-camera;
render-camera transform round-trip to store `cameras[]`; **video render behind the Record button**;
the record-mode "expand timeline into a grayed-out read-only editor" (currently the expand chevron
is simply hidden in record).
### Feature: Mixamo animation clips on the timeline (IN PROGRESS)
Characters can carry skeletal (Mixamo) animation clips on the timeline, alongside the transform-
keyframe tracks. Answers locked with the user: retarget onto the character's skeleton; clips +
keyframes coexist; default clip source = the 37 curated demo clips. **Playback is sequential, one
clip at a time** — a character's clips live on a **single row** and cannot overlap (see the overlap
guard below). The earlier "stacked lanes" idea was dropped: stacking only pays off for *additive
layering* (e.g. a base walk + an upper-body wave playing together), which needs per-lane weights +
bone masking and is out of scope. If layering is ever wanted, that's the feature to build; until
then one row per character is the right model.
- **Data model (phase A, done)**: `engine/timeline/types.ts` — `ClipStrip` (`sourceMediaId`,
`startTime`, `duration`, `loop`) + `ClipLane` (`objectUuid`, `strip`); `TimelineData.clipLanes[]`.
Only the reference + placement is serialized; the `AnimationClip` is resolved at runtime.
`TimelineController` has `addClipLane` / `moveClipLane` / `removeClipLane` / `clipLanesFor`;
`TimelineChangedEvent` carries `clipLanes` → store `timelineClipLanes`. Persists via the existing
`getTimeline`/`loadTimeline` (rides `TimelineData`).
- **Playback engine (phase B, done — UNVERIFIED at runtime)**: `engine/animation/CharacterAnimationManager.ts`.
One `THREE.AnimationMixer` per character; each lane → an `AnimationAction`. Playhead-driven +
**deterministic**: on every `TimelineController.evaluate()` it sets each action's absolute `time`
from the playhead (paused actions) then `mixer.update(0)` — scrub, play, and frame-accurate record
all pose identically. `syncClipLanes()` reconciles the runtime on every clip-lane change; a
monotonic load token guards against superseded async clip loads. Clips are loaded via
`Scene.loadRawGlb(media_id)` (raw GLTF, not added to the scene) and bound **by node name** —
relies on the shared `mixorig:*` naming so a clip's tracks resolve against the character's bones
directly (direct-bind first; `SkeletonUtils.retargetClip` is the documented fallback if a rig uses
different bone names — **not yet wired**, pending the runtime spike).
- **Trigger UI (done)**: `comps/AnimationsDrawer/` — right-docked, only when a character is
selected (build mode; hidden via CSS in record). Clips are **click-to-add** (drops at the playhead)
**and draggable** onto the timeline (`ANIMATION_CLIP_MIME` payload `{media_id, name}`).
- **Timeline clip UI (phase C, done)**: `comps/Timeline/TimelineClipRow.tsx` renders **one row per
character** (rendered even when empty, as a drop hint) holding all that character's strips
end-to-end under its keyframe row in `TimelineEditor`. Strip body drags to **move**
(`moveClipLane`), the right edge drags to **trim** length (`resizeClipLane`), a loop chip toggles
repeat (`setClipLoop`), the × removes it (`removeClipLane`). Each character row is a **drop target**
for clip drags (`dropClipOnCharacter` → `addClipToCharacter(…, atTime)`, time from the ruler rect).
Non-character rows ignore clip drags. No per-op undo by design — clip edits ride the timeline
**Save/Cancel** session exactly like keyframes (`cancel()` restores + re-syncs).
- **Overlap guard (single row)**: `TimelineController.resolveFreeStart` snaps add/move to the nearest
gap so a character's strips never overlap; `nextStartAfter` bounds trim (`resizeClipLane`) and
auto-length (`resolveClipDuration`) against the following clip. `evaluateAt` then has at most one
active clip at any playhead, so playback is an unambiguous sequence.
- **Bind-pose in gaps**: a disabled three.js action leaves the skeleton frozen on its last frame, so
`evaluateAt` resets any character with no clip under the playhead to its **bind (T) pose** via
`resetToBindPose` → `Skeleton.pose()` (skeletons cached per character in `getMixer`). Gaps between
strips, before the first, and after the last therefore show the default T-pose, not a held frame.
- **Drag preview**: dragging a clip drives the shared `DragGhost` (tilt card) by setting
`dragItem`/`dragPosition`/`assetDraggingUnder` on the store in the drawer's native-DnD handlers,
and suppresses the browser's default drag image — matching object/character pickup motion.
- **Real clip length (fix, done)**: a fresh drop seeds `strip.duration` with a placeholder and flags
`autoDuration`; when the GLB loads, `CharacterAnimationManager` calls
`TimelineController.resolveClipDuration(laneId, clip.duration)` to adopt the clip's true length
(and re-clamp start). A user trim clears `autoDuration` so the natural length never clobbers a
hand-set duration on reload.
- **Rig-mismatch diagnostic**: `CharacterAnimationManager.clipBindsToCharacter` warns (console) when
a clip's tracks resolve to **0** nodes on the character — the signal that `SkeletonUtils.retargetClip`
is needed. Retarget itself is still **not wired** (direct-bind first).
- **⚠️ Runtime unknown to verify**: whether the demo clips and character rigs share `mixamorig:*`
bone names (→ direct bind works) or need retargeting. Drop/scrub a clip and watch for motion (and
the console warn) to decide whether to wire the retarget fallback.
### ⚠️ Unverified — compiles but NOT runtime-tested
Everything in slices 1–3 has been validated **only** by `tsc -b` (both `pagescene` and
`promptbox` exit 0). The app has **not** been launched; no visual, behavioral, lint, or test
verification has been done. Outstanding items to verify live:
- **Slice 1 (pill)**: placement/spacing above `Controls3D` (possible double `pt-3`); white-active
styling vs Figma; that clicking actually flips `sceneMode`.
- **Slice 2 (prompted)**: Manual/Prompted toggle renders/positions correctly above the card;
prompted layout hides the toolbar and shows textbox + add-timeline + history + Update; Update
stub toasts + records history; history popover lists/restores prompts; toggle only appears when
the host wires `onBuildModeChange`.
- **Slice 3 (timeline)**: nothing behaviorally verified — playback actually interpolates objects;
scrub/seek updates transforms; add/move/delete keyframes; Save+undo / Cancel; collapsed bar
positioning via `aboveStackSlot`; expanded editor replacing the prompt box; **playhead-line
alignment** (the `LANE_LEFT=8.75rem`/`LANE_RIGHT=2.5rem` offsets are hand-computed guesses);
`MotionPopover` bezier drag coordinate math + popover anchoring; range-scrubber styling; euler
rotation lerp quality; the assumption that the render loop stays alive during playback
(`shouldRender`); "selected object gets an empty track row" behavior.
- **Slice 4 (camera split)**: nothing runtime-verified — that the frustum appears at the render
camera and is excluded from the render; that selecting/moving it moves the render framing;
double-click (viewport + outliner row) and the outliner icon enter camera view; the aspect
frame + focal label show in camera view; EXIT returns to the prior free-cam position; the
frustum is/ isn't viewport-click-selectable (**may need `raycaster.layers.enable(1)`** in
MouseControls — outliner selection works regardless); `::CAM::` really is skipped by save.
- **Slice 5 (record mode)**: nothing runtime-verified — that switching the pill to Record enters a
locked render-camera view (viewport truly can't be flown), hides all build chrome but keeps the
pill, and shows the read-only playback bar + Capture/Record; that Capture downloads the composed
frame; that switching back to Build restores the prior free-cam view and chrome; interaction
between record and a manually-entered camera view; whether `switchCameraView` refactor (idempotent
enter/exit) preserved the original toggle behavior. Record button is a **no-op stub**.
Treat all of the above as needing a live pass before shipping.
### Build mode has two usability sub-modes
- **Manual** — identical to current behavior. User manually moves/places/transforms objects with the gizmo and toolbar. This is what exists today.
- **Prompted (Builder)** — the prompt bar changes to show only:
- a text box,
- an **"Add animation timeline"** button,
- a **history** button (previous prompts),
- an **"Update"** primary button.
Prompted mode drives a **JSON-style, MCP-esque backend integration** (in development). The flow: an LLM (e.g. Claude) reads the scene's **scene-descriptor data** + the user's prompt, makes edits, **outputs a new scene descriptor**, which is then applied to the 3D scene.
The **"Add animation timeline"** button only appears when the scene is **static** (no keyframes/animations on any object). Clicking it reveals the animation timeline bar (see below).
### Animation timeline
**Collapsed state** (a slim bar that sits above the prompt box, replacing the "Add animation timeline" button once a timeline exists):
- ▶ play/pause · current-time readout (`0:00`) · scrubber (blue fill to playhead) · total duration (`0:10`) · **chevron ▾**.
- The **chevron ▾ expands the timeline downward, collapsing the prompt box** (they trade vertical space).
**Expanded state** (full multi-track keyframe editor):
- **Transport (top-left):** skip-to-start ⏮ · play ▶ · skip-to-end ⏭.
- **Time ruler** with a draggable **playhead** (`0:00 … 0:04 … 0:10`).
- **One track per object.** A track can hold either:
- **Diamond keyframes** — each diamond stores the object's **full transform (position, rotation, scale)** at that moment. Tapping a diamond jumps the playhead to it. Each row has an "add keyframe" diamond button at its right edge.
- **Natural-language motion clip** — a text clip spanning a time range (e.g. *"walks to car - opens door - gets in"*) that drives generated motion.
- **Any object can have either type.** For simplicity, an early version may show only raw keyframes and drop the natural-language grouping.
- **Motion popover** — appears when a keyframe is selected; edits the **easing/interpolation curve into the next keyframe**. Contains ~4 **easing presets** at top plus a large **editable bezier curve** with draggable **handles**; a **dotted line** shows the linear reference.
- **Cancel / Save** (bottom-right):
- **Cancel** — drops all changes back to the last saved timeline state.
- **Save** — commits the changes. **This is an undoable action** (record it via `HistoryManager` / an `UndoableAction`).
### Camera model change (Blender-style)
Introduce a distinction between:
- **Viewport (scene) camera** — what the user flies around with; the editing view.
- **Render camera(s)** — the camera(s) that define the final render output.
Render cameras become **objects within the 3D scene**: visible/selectable in the editor (with a camera gizmo/frustum, as in the Figma), but **excluded from the final render**. This mirrors professional 3D software (Blender).
Render cameras appear in the **Outliner** (left "Scene" panel). A camera row shows a **"view from camera"** icon positioned **right of the camera item's name and left of the lock icon** (visible in the reference frames). Clicking it **switches the viewport camera to the selected render camera** and lets the user control it with the usual navigation keybinds.
**This is the big architectural change** — but the engine is **already partially scaffolded** for it. Verify these anchors before building on them:
- **Two cameras already exist** in `engine/editor/CameraController.ts`: `camera` (viewport/free-fly, sees layer 1 incl. gizmos) and `render_camera` (layer 0 only, clean output, its own aspect ratio). Created in `editor.ts` `initialize()` from the store's camera configs.
- **`Camera` type** lives in `frontend/libs/common/src/lib/interfaces/Camera.ts`: `{ id, label, focalLength, position, rotation, lookAt }`. Store seeds two: `"main"` (Main View, viewport) and `"cam2"` (Camera 2, render). So the store models multiple camera *configs*, but only **one** `render_camera` THREE object is currently instantiated (first non-main).
- **Placeholder object `"::CAM::"`** is referenced throughout (`scene.getObjectByName("::CAM::")`, `CameraController.refreshCamObj`, layer 1 so it's editor-only/excluded from render) — **but `_create_camera_obj()` is commented out** in `scene.ts`, so no camera mesh is actually in the scene graph today. The Outliner already knows to render a `::CAM::` object as a "Camera" row (`scene_manager_api.ts` `convert_object`) — it just never appears because the object isn't created.
- **`CAMERA_VIEW` state** (`enums/EditorState.ts`) + `CameraController.switchCameraView()` already implement enter/exit: on enter it copies `cam_obj` transform → viewport camera, hides `cam_obj`/objects/gizmo; on exit it restores. **Caveat:** today it *repositions the single viewport camera* rather than truly swapping to render through a separate scene camera. `PreviewEngineCamera.tsx` + an unused `camViewCanvasEl` exist as a secondary preview surface.
- **Letterbox/aspect overlay** already built: `comps/SceneContainer/Letterbox.tsx` reads `cameraAspectRatio`, draws mattes, toggled via `editorLetterBox`. `CameraAspectRatio` enum + `getRenderDimensions()` map to pixel sizes.
- **Focal length** is per-camera (`Camera.focalLength`), converted via `CameraController.focalLengthToFov()`, applied per-frame in `tickPerFrame()`; `focalLengthDragging` tracks the slider. The `35mm` overlay label maps to this.
- **Render path**: `editor.ts` `renderScene()` renders `activeScene.scene` through `render_camera`; but `snapShotOfCurrentFrame()` currently snapshots through the **viewport** `camera`, not `render_camera` — reconcile so exports use the render camera.
**Gaps to close for the true split:** (1) actually create camera placeholder meshes as scene objects (uncomment/rework `_create_camera_obj`); (2) map `cameras: Camera[]` 1:1 to scene objects; (3) make `CAMERA_VIEW` swap which camera the viewport renders through (not just reposition one); (4) support N render cameras, not just the first non-main; (5) make cameras keyframeable timeline targets. Reconcile all of this with the existing `cameras[]` + `selectedCameraId` store state.
### Camera view
Entered by **double-clicking the camera placeholder in the viewport** or the **view-from-camera button in the outliner**. In this state the **viewport camera becomes the selected render camera** — the user flies it with the usual navigation keybinds, and that motion is what gets keyframed.
- **Framing overlay**: the render **aspect frame** with **rule-of-thirds** guides; everything **outside the frame is dimmed** (won't be captured).
- **Focal-length label** (e.g. `35mm`) shown top-left inside the frame. Editable; may become a keyframable value in the future.
- The camera's frustum is not drawn while looking through it.
- **Timeline filters to just the selected camera's track** while in camera view.
- **Selected keyframe** renders as a **blue diamond** (at the playhead); other keyframes are white.
- **Exit** via the **EXIT CAMERA VIEW** button (bottom-right), or **Esc** / **double-click out**. (The text button is a dev placeholder; a proper icon comes later.)
Camera keyframes store the camera's transform just like object keyframes (and later, potentially focal length).
### Record mode
Record mode is the **read-only output/preview** mode — practically **immutable**. It plays back the render timeline **as seen from the main render camera** only.
- **All build chrome is hidden**: no outliner, no transform toolbar, no prompt box, no gizmos.
- The **viewport is locked to the render camera** (no free-fly, no object/camera editing). Framing overlay (aspect frame, dimmed exterior) is shown.
- **The only interactive controls are play/pause and scrubber position.** Nothing in the scene can be mutated.
- The **chevron can still expand the timeline**, but it renders **grayed out / read-only** — no keyframe editing.
- Two actions:
- **Capture** — export the **current frame as a still image**.
- **Record** — render the **full animation to video**.
### Build ↔ Record relationship (summary of axes)
- **Top pill**: `sceneMode` = `build | record` (net-new).
- **Within Build**: a usability sub-mode = `manual | prompted`.
- **Camera view** is an orthogonal viewport state (looking through a render camera) available while editing — not a top-level mode.
- Keep these distinct from the pre-existing `editorState` (`EDIT | CAMERA_VIEW`) and `poseMode` (`select | pose`) fields; do not overload them.
### Notes / deferred
- **JSON scene descriptor / MCP backend contract** is out of scope for the UI work — the UI and other features can be built independently. The one thing that won't function until the backend lands is the **Update button in Prompted mode** (acceptable for dev).
- The **view-from-camera icon** is already shown in the reference frames (outliner, right of the camera name, left of the lock icon) — no separate spec needed.
- **EXIT CAMERA VIEW** text button is a dev placeholder; a proper icon comes later.
@@ -7,6 +7,7 @@ export type { Stage3DProps } from "./lib/Stage3D";
// Adapter (platform abstraction)
export type {
PageSceneAdapter,
PageSceneArtifact,
PageSceneGenerateRequest,
PageSceneSavePayload,
ListMediaFilesQuery,
@@ -6,12 +6,41 @@ import {
AssetFilterOption,
CameraAspectRatio,
ClipGroup,
DEFAULT_CAMERA_ASPECT_RATIO,
EditorStates,
} from "./enums";
import { MediaItem } from "./models";
import { Simple3DVector } from "./datastructures/common";
export type { Camera, FocalLengthDragging };
export type {
EasingSpec,
Keyframe,
TimelineTrack,
TimelineData,
ClipLane,
ClipStrip,
} from "./engine/timeline/types";
import type { TimelineTrack, ClipLane } from "./engine/timeline/types";
// A still (Capture) or video (Record) produced by Record mode, cached
// locally (object URL) before any upload. Powers the completion modal's
// preview/playback and the 2D/video handoff.
export interface ProducedArtifact {
kind: "image" | "video";
blob: Blob;
objectUrl: string;
fileName: string;
mimeType: string;
aspectRatio: CameraAspectRatio;
}
// Progress of an in-flight Capture/Record so the RenderOverlay can cover the
// viewport (freeing GPU) and show progress while frames encode.
export interface RecordingProgress {
phase: "capturing" | "encoding" | "uploading";
pct: number; // 0..1
}
// Scene metadata — what host tracks in its `signalScene`. Mirrored
// into the store so ControlsTopButtons (lib-resident) can read it
@@ -48,6 +77,9 @@ export interface OutlinerItem {
type: string;
visible: boolean;
locked: boolean;
// True for the render-camera placeholder ("::CAM::") — the outliner shows a
// view-from-camera button for these rows.
isCamera?: boolean;
}
// The currently inspected object in the right-hand control panel.
@@ -76,6 +108,7 @@ export interface SelectedSceneObject {
id: string;
}
export type SceneMode = "build" | "record";
export type TransformMode = "move" | "rotate" | "scale";
export type TransformSpace = "world" | "local";
export type PoseMode = "select" | "pose";
@@ -100,11 +133,13 @@ export interface EditorLoader {
const DEFAULT_CAMERAS: Camera[] = [
{
id: "main",
// Blender-style default: viewport pulled back + elevated at a 3/4 angle so
// the whole render-camera frustum (cam2, below) is visible in the scene.
label: "Main View",
focalLength: 17,
position: { x: -2.5, y: 2.5, z: 2.5 },
position: { x: -4.5, y: 4, z: 6 },
rotation: { x: 0, y: 0, z: 0 },
lookAt: { x: 0, y: 0, z: 0 },
lookAt: { x: 0, y: 0.5, z: 0.6 },
},
{
id: "cam2",
@@ -131,6 +166,7 @@ interface PageSceneState {
cameraFilter: AssetFilterOption;
// editor mode
sceneMode: SceneMode;
editorState: EditorStates;
transformMode: TransformMode;
transformSpace: TransformSpace;
@@ -142,6 +178,24 @@ interface PageSceneState {
hotkeyStatus: HotkeyStatus;
isPromptBoxFocused: boolean;
// timeline (mirrors TimelineController; see engine/editor/TimelineController.ts)
timelineExists: boolean;
timelineExpanded: boolean;
timelinePlayhead: number;
timelineIsPlaying: boolean;
timelineDuration: number;
timelineTracks: TimelineTrack[];
timelineClipLanes: ClipLane[];
timelineSelectedKeyframeId: string | null;
// Left keyframe of the segment whose easing curve is being edited in the
// Motion popover (opened from the curve chip BETWEEN two keyframes).
// Distinct from timelineSelectedKeyframeId, which drives selection/delete.
timelineEasingKeyframeId: string | null;
// record output
producedArtifact: ProducedArtifact | null;
recordingProgress: RecordingProgress | null;
// layout / panels
assetModalVisible: boolean;
assetModalVisibleDuringDrag: boolean;
@@ -227,6 +281,7 @@ interface PageSceneState {
setCameraFilter: (filter: AssetFilterOption) => void;
// editor mode
setSceneMode: (mode: SceneMode) => void;
setEditorState: (state: EditorStates) => void;
setTransformMode: (mode: TransformMode) => void;
setTransformSpace: (space: TransformSpace) => void;
@@ -234,6 +289,18 @@ interface PageSceneState {
setPoseMode: (mode: PoseMode) => void;
setShowPoseControls: (visible: boolean) => void;
setGridVisible: (visible: boolean) => void;
setTimelineExists: (exists: boolean) => void;
setTimelineExpanded: (expanded: boolean) => void;
setTimelinePlayhead: (time: number) => void;
setTimelineIsPlaying: (playing: boolean) => void;
setTimelineDuration: (duration: number) => void;
setTimelineTracks: (tracks: TimelineTrack[]) => void;
setTimelineClipLanes: (clipLanes: ClipLane[]) => void;
setTimelineSelectedKeyframe: (id: string | null) => void;
setTimelineEasingKeyframe: (id: string | null) => void;
setProducedArtifact: (artifact: ProducedArtifact | null) => void;
clearProducedArtifact: () => void;
setRecordingProgress: (progress: RecordingProgress | null) => void;
toggleStats: () => void;
setIgnoreKeyDelete: (ignore: boolean) => void;
disableHotkeyInput: (level: DomLevels) => void;
@@ -306,17 +373,29 @@ export const usePageSceneStore = create<PageSceneState>((set, get) => ({
cameras: DEFAULT_CAMERAS,
selectedCameraId: "main",
cameraAspectRatio: CameraAspectRatio.HORIZONTAL_3_2,
cameraAspectRatio: DEFAULT_CAMERA_ASPECT_RATIO,
focalLengthDragging: { isDragging: false, focalLength: 35 },
cameraFilter: AssetFilterOption.ALL,
editorState: EditorStates.EDIT,
sceneMode: "build",
transformMode: "move",
transformSpace: "world",
selectedMode: "move",
poseMode: "select",
showPoseControls: false,
gridVisible: true,
timelineExists: false,
timelineExpanded: false,
timelinePlayhead: 0,
timelineIsPlaying: false,
timelineDuration: 10,
timelineTracks: [],
timelineClipLanes: [],
timelineSelectedKeyframeId: null,
timelineEasingKeyframeId: null,
producedArtifact: null,
recordingProgress: null,
ignoreKeyDelete: false,
hotkeyStatus: { disabled: false, disabledBy: DomLevels.NONE },
isPromptBoxFocused: false,
@@ -342,7 +421,7 @@ export const usePageSceneStore = create<PageSceneState>((set, get) => ({
outlinerItems: [],
outlinerSelectedItem: null,
outlinerShowing: false,
outlinerShowing: true,
precisionSelectorShowing: false,
precisionSelectorCoords: { x: 0, y: 0 },
@@ -405,6 +484,7 @@ export const usePageSceneStore = create<PageSceneState>((set, get) => ({
setCameraFilter: (filter) => set({ cameraFilter: filter }),
// editor mode actions
setSceneMode: (mode) => set({ sceneMode: mode }),
setEditorState: (state) => set({ editorState: state }),
setTransformMode: (mode) => set({ transformMode: mode }),
setTransformSpace: (space) => set({ transformSpace: space }),
@@ -412,6 +492,22 @@ export const usePageSceneStore = create<PageSceneState>((set, get) => ({
setPoseMode: (mode) => set({ poseMode: mode }),
setShowPoseControls: (visible) => set({ showPoseControls: visible }),
setGridVisible: (visible) => set({ gridVisible: visible }),
setTimelineExists: (exists) => set({ timelineExists: exists }),
setTimelineExpanded: (expanded) => set({ timelineExpanded: expanded }),
setTimelinePlayhead: (time) => set({ timelinePlayhead: time }),
setTimelineIsPlaying: (playing) => set({ timelineIsPlaying: playing }),
setTimelineDuration: (duration) => set({ timelineDuration: duration }),
setTimelineTracks: (tracks) => set({ timelineTracks: tracks }),
setTimelineClipLanes: (clipLanes) => set({ timelineClipLanes: clipLanes }),
setTimelineSelectedKeyframe: (id) => set({ timelineSelectedKeyframeId: id }),
setTimelineEasingKeyframe: (id) => set({ timelineEasingKeyframeId: id }),
setProducedArtifact: (artifact) => set({ producedArtifact: artifact }),
clearProducedArtifact: () =>
set((s) => {
if (s.producedArtifact) URL.revokeObjectURL(s.producedArtifact.objectUrl);
return { producedArtifact: null };
}),
setRecordingProgress: (progress) => set({ recordingProgress: progress }),
toggleStats: () => set((s) => ({ statsVisible: !s.statsVisible })),
setIgnoreKeyDelete: (ignore) => set({ ignoreKeyDelete: ignore }),
disableHotkeyInput: (level) => {
@@ -38,14 +38,15 @@ import {
useCostBreakdownModalStore,
} from "@storyteller/ui-pricing-modal";
import { LoadingDots } from "@storyteller/ui-loading";
import { PromptBox3D, commonToCameraAspect } from "@storyteller/ui-promptbox";
import type { PopoverItem } from "@storyteller/ui-popover";
import { v4 as uuidv4 } from "uuid";
import { EngineContext } from "./contexts/EngineContext/EngineContext";
import { AnonHintChip } from "./comps/AnonHintChip";
import { AspectRatioMenu } from "./comps/AspectRatioMenu";
import { ControlPanelSceneObject } from "./comps/ControlPanelSceneObject";
import { Controls3D } from "./comps/Controls3D";
import { SceneModePill } from "./comps/SceneModePill";
import { ControlsTopButtons } from "./comps/ControlsTopButtons";
import { EditorCanvas } from "./comps/EngineCanvases";
import { FocalLengthDisplay } from "./comps/FocalLengthDisplay/FocalLengthDisplay";
@@ -53,11 +54,18 @@ import { OnboardingHelper } from "./comps/OnboardingHelper";
import { PerfStatsOverlay } from "./comps/PerfStatsOverlay";
import { EntranceDebugPanel } from "./comps/EntranceDebugPanel";
import { Outliner } from "./comps/Outliner";
import { AnimationsDrawer } from "./comps/AnimationsDrawer";
import { PoseModeSelector } from "./comps/PoseModeSelector";
import { ExitCameraView } from "./comps/ExitCameraView";
import { RecordControls } from "./comps/RecordControls";
import { CameraStatusPill } from "./comps/CameraStatusPill";
import { RenderOverlay } from "./comps/RenderOverlay";
import { CompletionModal } from "./comps/CompletionModal";
import { PreviewBox } from "./comps/PreviewBox";
import { PreviewEngineCamera } from "./comps/PreviewEngineCamera";
import { SceneContainer } from "./comps/SceneContainer";
import { addCharacter, addObject, setCameraAspect } from "./actions";
import { addCharacter, addObject } from "./actions";
import { TimelineBar, TimelineEditor } from "./comps/Timeline";
import { useEditorCanvas } from "./hooks/useEditorCanvas";
import { useFreeCam } from "./hooks/useFreeCam";
import { useViewportPointer } from "./hooks/useViewportPointer";
@@ -126,6 +134,13 @@ export const Stage3DBody = ({
(s) => s.setIsPromptBoxFocused,
);
const gridVisible = usePageSceneStore((s) => s.gridVisible);
const timelineExists = usePageSceneStore((s) => s.timelineExists);
const timelineExpanded = usePageSceneStore((s) => s.timelineExpanded);
const setTimelineExpanded = usePageSceneStore((s) => s.setTimelineExpanded);
const timelineTracks = usePageSceneStore((s) => s.timelineTracks);
const is3DSceneLoaded = usePageSceneStore((s) => s.is3DSceneLoaded);
const sceneMode = usePageSceneStore((s) => s.sceneMode);
const isRecord = sceneMode === "record";
const previewImageUrl = usePageSceneStore((s) => s.sceneMeta.previewImageUrl);
const isVisitingOthersScene = useIsVisitingOthersScene();
const addCamera = usePageSceneStore((s) => s.addCamera);
@@ -219,6 +234,30 @@ export const Stage3DBody = ({
const editor = useContext(EngineContext);
// Record mode is read-only playback through the render camera: enter
// camera view + lock the viewport on entry, restore on exit.
useEffect(() => {
if (!editor) return;
if (isRecord) {
editor.cameraController.enterCameraView();
editor.cameraController.setLocked(true);
} else {
editor.cameraController.setLocked(false);
editor.cameraController.exitCameraView();
}
}, [isRecord, editor]);
// Auto-expand the timeline once when a loaded scene has keyframes or an
// imported Three.js animation; otherwise it stays collapsed by default.
const didAutoExpandTimelineRef = useRef(false);
useEffect(() => {
if (!editor || !is3DSceneLoaded || didAutoExpandTimelineRef.current) return;
didAutoExpandTimelineRef.current = true;
if (timelineTracks.length > 0 || editor.sceneHasAnimation()) {
setTimelineExpanded(true);
}
}, [editor, is3DSceneLoaded, timelineTracks, setTimelineExpanded]);
// Reactive viewport sizing. useViewportSize listens to window
// resize and re-renders the component. Falls back to
// window.innerWidth/innerHeight when the host adapter doesn't
@@ -343,28 +382,6 @@ export const Stage3DBody = ({
}
};
const onAspectRatioSelect = (newRatio: CameraAspectRatio) => {
if (!editor) return;
setCameraAspect(editor, newRatio);
};
// Cold-load sync: align the editor letterbox with the picker's
// initial display once when the engine + a `supportsNewAspectRatio()`
// model are both ready. Per-model-switch sync is intentionally NOT
// done because every model defaults to Square, which would override
// the user's pick.
const didColdSyncRef = useRef(false);
useEffect(() => {
if (didColdSyncRef.current) return;
if (!editor || !selectedImageModel?.supportsNewAspectRatio()) return;
const def = selectedImageModel.defaultAspectRatio;
if (!def) return;
const mapped = commonToCameraAspect(def);
if (!mapped) return;
setCameraAspect(editor, mapped);
didColdSyncRef.current = true;
}, [editor, selectedImageModel]);
// Gallery → 3D scene drop handler. Stage3D mounts only when 3D is
// active so this is implicitly 3D-only.
useEffect(() => {
@@ -443,6 +460,8 @@ export const Stage3DBody = ({
{import.meta.env.DEV && <EntranceDebugPanel />}
<FocalLengthDisplay />
<PoseModeSelector />
<RenderOverlay />
<CompletionModal />
<div
className="absolute left-0 top-0 w-full"
@@ -450,87 +469,87 @@ export const Stage3DBody = ({
>
<div className="grid grid-cols-3 gap-4">
<div className="flex flex-col items-start gap-2">
<ControlsTopButtons />
{topBarStartSlot && (
{!isRecord && <ControlsTopButtons />}
{!isRecord && topBarStartSlot && (
<div className="pl-3">{topBarStartSlot}</div>
)}
</div>
<Controls3D showImageTo3DButton={showImageTo3DButton} />
<div className="flex flex-col items-center">
<SceneModePill />
{!isRecord && (
<Controls3D showImageTo3DButton={showImageTo3DButton} />
)}
</div>
<div className="flex items-start justify-end gap-2 pr-3 pt-3">
{topBarEndSlot}
<AnonHintChip />
{/* Camera cluster: which camera drives the viewport +
the render-frame aspect ratio. */}
{!isRecord && <CameraStatusPill />}
{!isRecord && <AspectRatioMenu />}
{!isRecord && topBarEndSlot}
{!isRecord && <AnonHintChip />}
</div>
</div>
</div>
{/* Outliner: vertically centered against the viewport. Sized in
vh — deliberately independent of the camera aspect ratio. */}
<div
className={`absolute left-4 top-1/2 -translate-y-1/2${
isRecord ? " hidden" : ""
}`}
>
<Outliner />
</div>
{/* Animations drawer: right-docked, only when a character is
selected. Gated to build mode (hide via CSS, never unmount). */}
<div
className={`absolute right-4 top-1/2 -translate-y-1/2${
isRecord ? " hidden" : ""
}`}
>
<AnimationsDrawer />
</div>
<div
className="absolute bottom-0 left-0 right-0"
onClick={handleOverlayClick}
>
{/* Always mounted: PreviewEngineCamera owns the cam-view canvas
that EngineProvider depends on — unmounting it in record mode
tears down + recreates the whole engine (wiping the scene).
Hide with CSS in record instead. */}
<div
className="absolute bottom-20 mb-4 ml-4 flex origin-bottom-left flex-col gap-2"
className={`absolute bottom-20 mb-4 ml-4 flex origin-bottom-left flex-col gap-2${
isRecord ? " hidden" : ""
}`}
style={{ transform: `scale(${getScale()})` }}
>
<Outliner />
<PreviewEngineCamera />
</div>
<ControlPanelSceneObject />
{!isRecord && <ControlPanelSceneObject />}
</div>
{isVisitingOthersScene && <PreviewBox imageUrl={previewImageUrl} />}
<PromptBox3D
cameras={cameras}
cameraAspectRatio={camAspect}
disableHotkeyInput={disableHotkeyInput}
enableHotkeyInput={enableHotkeyInput}
gridVisibility={gridVisible}
setGridVisibility={(visible: boolean) =>
editor?.bus.emit(new GridVisibleChangedEvent(visible))
}
selectedCameraId={selectedCameraId}
deleteCamera={deleteCamera}
focalLengthDragging={focalLengthDragging}
setFocalLengthDragging={setFocalLengthDragging}
isPromptBoxFocused={isPromptBoxFocused}
setIsPromptBoxFocused={setIsPromptBoxFocused}
uploadImage={
editor
? (((arg: Parameters<typeof editor.adapter.uploadImage>[0]) =>
editor.adapter.uploadImage(arg)) as never)
: undefined
}
handleCameraSelect={handleCameraSelect}
handleAddCamera={handleAddCamera}
handleCameraNameChange={handleCameraNameChange}
handleCameraFocalLengthChange={handleCameraFocalLengthChange}
onAspectRatioSelect={onAspectRatioSelect}
selectedImageModel={selectedImageModel}
selectedProvider={selectedProvider}
credits={imageCredits}
setEnginePrompt={(prompt) => {
if (!editor) return;
editor.positive_prompt = prompt;
}}
snapshotCurrentFrame={editor?.snapShotOfCurrentFrame.bind(editor)}
modelSelector={inlineModelSelector}
aboveStackSlot={
<>
<OnboardingHelper />
{promptboxAboveStackSlot}
</>
}
onBeforeSubmit={() => {
const currentUserToken =
usePageSceneStore.getState().currentUserToken;
if (!currentUserToken && editor?.adapter.promptSignup) {
editor.adapter.promptSignup("generate");
return false;
}
return true;
}}
/>
{/* Build mode: the animation timeline is the sole bottom UI —
collapsed bar by default, expanded editor via the chevron.
The exit-camera-view button docks to the bar's right edge. */}
{!isRecord && !timelineExpanded && (
<div className="absolute bottom-4 left-1/2 w-[90vw] max-w-3xl -translate-x-1/2">
<TimelineBar />
<ExitCameraView className="absolute -right-2 top-1/2 -translate-y-1/2 translate-x-full" />
</div>
)}
{!isRecord && timelineExpanded && (
<>
<TimelineEditor />
<ExitCameraView className="absolute bottom-4 right-4" />
</>
)}
{isRecord && <RecordControls />}
<LoadingDots
className="absolute left-0 top-0 z-50"
@@ -0,0 +1,9 @@
import type Editor from "../engine/editor";
import { CameraViewToggleRequestedEvent } from "../engine/events/EngineEvent";
// Toggle "camera view" (look through / exit the render camera). Used by the
// outliner view-from-camera icon and the exit-camera-view button. Routes
// through the bus so the CameraController owns the actual transition.
export function toggleCameraView(editor: Editor): void {
editor.bus.emit(new CameraViewToggleRequestedEvent());
}
@@ -8,4 +8,7 @@ export * from "./setCameraAspect";
export * from "./setObjectColor";
export * from "./toggleObjectLock";
export * from "./toggleObjectVisibility";
export * from "./openAssetModal";
export * from "./transformObject";
export * from "./timeline";
export * from "./cameraView";
@@ -0,0 +1,12 @@
import { usePageSceneStore } from "../PageSceneStore";
// Open the add-asset (ArtCraft presets) modal. Shared by the Controls3D "+"
// popover and the Outliner's "+ Add" button. Clears any leftover drag-under
// state so the panel opens fully shown (a reopen-off drag leaves it
// faded-hidden until the next open).
export function openAssetModal(): void {
const store = usePageSceneStore.getState();
store.setAssetModalVisibleDuringDrag(true);
store.setAssetModalVisible(true);
store.setAssetDraggingUnder(false);
}
@@ -0,0 +1,125 @@
// UI-facing dispatchers for the animation timeline. Thin wrappers that
// forward to editor.timelineController (mirrors the other files in this
// folder). Pure-UI state (expanded, selected keyframe) is set directly on
// the store by components and has no dispatcher here.
import type Editor from "../engine/editor";
import type { EasingSpec } from "../engine/timeline/types";
// The clip source only needs an id + label — both a full MediaItem (drawer
// click) and a drag payload (drop) satisfy this.
interface ClipSource {
media_id: string;
name?: string;
}
// Placeholder strip length (seconds) until the clip GLB reports its real
// duration; playback uses the loaded clip's actual length regardless.
const DEFAULT_CLIP_DURATION = 2;
export function createTimeline(editor: Editor): void {
editor.timelineController.create();
}
export function playTimeline(editor: Editor): void {
editor.timelineController.play();
}
export function pauseTimeline(editor: Editor): void {
editor.timelineController.pause();
}
export function seekTimeline(editor: Editor, time: number): void {
editor.timelineController.seekTo(time);
}
export function setTimelineDuration(editor: Editor, seconds: number): void {
editor.timelineController.setDuration(seconds);
}
export function addKeyframe(
editor: Editor,
objectUuid: string,
atTime?: number,
): void {
editor.timelineController.addKeyframe(objectUuid, atTime);
}
export function deleteKeyframe(editor: Editor, keyframeId: string): void {
editor.timelineController.deleteKeyframe(keyframeId);
}
export function moveKeyframe(
editor: Editor,
keyframeId: string,
time: number,
): void {
editor.timelineController.moveKeyframe(keyframeId, time);
}
export function setKeyframeEasing(
editor: Editor,
keyframeId: string,
easing: EasingSpec,
): void {
editor.timelineController.setEasing(keyframeId, easing);
}
// Place a skeletal-animation clip (a Mixamo demo item) as a new stacked lane
// under `characterUuid`. Drops at the current playhead by default, or at an
// explicit `atTime` when dropped onto a specific point in the timeline. The
// real clip length replaces DEFAULT_CLIP_DURATION once the GLB loads (the
// strip starts at the placeholder width, flagged autoDuration).
export function addClipToCharacter(
editor: Editor,
characterUuid: string,
item: ClipSource,
atTime?: number,
): string | null {
if (!item.media_id) return null;
const startTime = atTime ?? editor.timelineController.getPlayhead();
return editor.timelineController.addClipLane(characterUuid, {
sourceMediaId: item.media_id,
name: item.name ?? "Animation",
startTime,
duration: DEFAULT_CLIP_DURATION,
loop: false,
autoDuration: true,
});
}
export function moveClipLane(
editor: Editor,
laneId: string,
startTime: number,
): void {
editor.timelineController.moveClipLane(laneId, startTime);
}
export function resizeClipLane(
editor: Editor,
laneId: string,
duration: number,
): void {
editor.timelineController.resizeClipLane(laneId, duration);
}
export function setClipLoop(
editor: Editor,
laneId: string,
loop: boolean,
): void {
editor.timelineController.setClipLoop(laneId, loop);
}
export function removeClipLane(editor: Editor, laneId: string): void {
editor.timelineController.removeClipLane(laneId);
}
export function saveTimeline(editor: Editor): void {
editor.timelineController.save();
}
export function cancelTimeline(editor: Editor): void {
editor.timelineController.cancel();
}
@@ -29,6 +29,9 @@ export function beginTransformSession(
commit() {
if (action.commit()) {
editor.history.record(action);
// Auto-key: an already-keyframed object edited via the panel updates
// its keyframe at the playhead (or creates one at the scrub point).
editor.timelineController.autoKeyIfTracked(uuid);
}
// Push the engine's post-edit transform back into the Zustand
// objectPanel slice so the panel's currentSceneObject vectors
@@ -75,6 +75,16 @@ export interface PageSceneSavePayload {
// ─── Adapter ───────────────────────────────────────────────────────────
// A still (Capture) or video (Record) produced by Record mode, cached
// locally (object URL) — handed to the 2D/video editor or uploaded on demand.
export interface PageSceneArtifact {
kind: "image" | "video";
blob: Blob;
objectUrl: string;
fileName: string;
mimeType: string;
}
export interface PageSceneAdapter {
// Generation enqueue. Same shape as PageDrawAdapter.enqueueEditImage.
enqueueGeneration(
@@ -215,6 +225,22 @@ export interface PageSceneAdapter {
// implements via useTabStore.setActiveTab; web host via router push.
navigateToImageTo3D(): void;
// Persist a produced still/video to the user's media library and return its
// media token. Images auto-upload after Capture; videos upload on demand
// (they're large). Host: MediaUploadApi via uploadByKind.
uploadMedia?(args: {
kind: "image" | "video";
blob: Blob;
fileName: string;
title?: string;
}): Promise<string>;
// Open the app's media Lightbox on an uploaded token — the shared modal that
// offers every destination for the media type (Edit-on-Canvas, Make-Video,
// Recreate, Share, Download). Host resolves the CDN URL from the token and
// renders <Lightbox>. Replaces bespoke per-destination handoffs.
openMediaLightbox?(token: string, kind: "image" | "video"): void;
// Auth/logout — the SettingsModal in Controls3D needs a logout
// callback. Tauri host: setLogoutStates. Web host: web auth flow.
performLogout(): void;
@@ -0,0 +1,100 @@
// Animations drawer: when a character is selected, lists the curated Mixamo
// clips. Click a clip to drop it on the character's timeline at the playhead,
// or drag it onto a character's timeline row (drop time follows the cursor).
// While dragging, the shared DragGhost (tilt card) previews the clip's
// thumbnail — the same pickup motion as object/character drags.
import { useContext } from "react";
import { EngineContext } from "../../contexts/EngineContext";
import { usePageSceneStore } from "../../PageSceneStore";
import { addClipToCharacter } from "../../actions";
import { demoAnimationItems } from "../../signals/demoAssets/demoAnimationItems";
import { ANIMATION_CLIP_MIME } from "../Timeline/timelineUtils";
// 1×1 transparent GIF used to suppress the browser's default drag image so only
// the DragGhost tilt card shows. Created once at module load so it's decoded by
// the time a drag starts.
const TRANSPARENT_DRAG_IMAGE =
typeof Image !== "undefined" ? new Image() : null;
if (TRANSPARENT_DRAG_IMAGE) {
TRANSPARENT_DRAG_IMAGE.src =
"data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7";
}
export const AnimationsDrawer = () => {
const editor = useContext(EngineContext);
const selectedObject = usePageSceneStore((s) => s.selectedObject);
const characters = usePageSceneStore((s) => s.characters);
const character = characters.find(
(c) => c.kind === "character" && c.id === selectedObject?.id,
);
if (!character) return null;
return (
<div className="glass glass-no-hover pointer-events-auto flex w-52 flex-col gap-2 rounded-2xl p-3 shadow-xl">
<div className="px-1 text-xs font-medium text-base-fg/70">
Animations · {character.name}
</div>
<div className="grid max-h-[46vh] grid-cols-2 gap-2 overflow-y-auto pe-1">
{demoAnimationItems.map((item) => (
<button
key={item.media_id}
type="button"
draggable
title={`Add “${item.name}” to ${character.name} — click, or drag onto the timeline`}
className="group flex flex-col items-stretch gap-1 rounded-lg border border-white/10 bg-black/30 p-1 text-left transition-colors hover:border-white/40"
onDragStart={(e) => {
e.dataTransfer.setData(
ANIMATION_CLIP_MIME,
JSON.stringify({ media_id: item.media_id, name: item.name }),
);
e.dataTransfer.effectAllowed = "copy";
// Hide the native drag image; the DragGhost renders the preview.
if (TRANSPARENT_DRAG_IMAGE) {
e.dataTransfer.setDragImage(TRANSPARENT_DRAG_IMAGE, 0, 0);
}
const store = usePageSceneStore.getState();
store.setDragItem(item);
store.setDragPosition({ currX: e.pageX, currY: e.pageY });
store.setAssetDraggingUnder(true);
}}
onDrag={(e) => {
// The final native drag event can report (0,0) — ignore it so the
// ghost doesn't snap to the corner on release.
if (e.pageX === 0 && e.pageY === 0) return;
usePageSceneStore
.getState()
.setDragPosition({ currX: e.pageX, currY: e.pageY });
}}
onDragEnd={() => {
const store = usePageSceneStore.getState();
store.setDragItem(null);
store.setAssetDraggingUnder(false);
store.setDragPosition({ currX: 0, currY: 0 });
}}
onClick={() =>
editor && addClipToCharacter(editor, character.id, item)
}
>
<div className="aspect-square w-full overflow-hidden rounded-md bg-black/40">
{item.thumbnail && (
<img
src={item.thumbnail}
alt={item.name ?? "Animation"}
className="h-full w-full object-cover opacity-90 transition-opacity group-hover:opacity-100"
draggable={false}
/>
)}
</div>
<span className="truncate px-0.5 text-[11px] text-base-fg/90">
{item.name}
</span>
</button>
))}
</div>
</div>
);
};
export default AnimationsDrawer;
@@ -0,0 +1 @@
export { AnimationsDrawer } from "./AnimationsDrawer";
@@ -1,96 +1,60 @@
import {
faRectangle,
faRectangleVertical,
faSquare,
} from "@fortawesome/pro-solid-svg-icons";
import { useContext } from "react";
import { PopoverMenu, type PopoverItem } from "@storyteller/ui-popover";
import { Tooltip } from "@storyteller/ui-tooltip";
import { AspectRatioIcon } from "@storyteller/ui-promptbox";
import { CameraAspectRatio } from "../../enums";
import { setCameraAspect } from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { EngineContext } from "../../contexts/EngineContext";
import { ButtonDropdown } from "@storyteller/ui-button-dropdown";
const ICON_BY_RATIO: Record<CameraAspectRatio, typeof faRectangle> = {
[CameraAspectRatio.HORIZONTAL_16_9]: faRectangle,
[CameraAspectRatio.VERTICAL_9_16]: faRectangleVertical,
[CameraAspectRatio.HORIZONTAL_3_2]: faRectangle,
[CameraAspectRatio.VERTICAL_2_3]: faRectangleVertical,
[CameraAspectRatio.SQUARE_1_1]: faSquare,
};
// Render-frame aspect ratio picker. Same popover pattern as the prompt
// boxes' AspectRatioPicker, but driving the 3D editor's camera letterbox
// (CameraAspectRatio) instead of a generation model's ratio list.
const LABEL_BY_RATIO: Record<CameraAspectRatio, string> = {
[CameraAspectRatio.HORIZONTAL_16_9]: "16:9 Horizontal",
[CameraAspectRatio.VERTICAL_9_16]: "9:16 Vertical",
[CameraAspectRatio.HORIZONTAL_3_2]: "3:2 Horizontal",
[CameraAspectRatio.VERTICAL_2_3]: "2:3 Vertical",
[CameraAspectRatio.SQUARE_1_1]: "1:1 Squared",
};
interface RatioOption {
ratio: CameraAspectRatio;
label: string;
proportions: [number, number];
}
const RATIO_OPTIONS: RatioOption[] = [
{ ratio: CameraAspectRatio.HORIZONTAL_16_9, label: "16:9", proportions: [16, 9] },
{ ratio: CameraAspectRatio.HORIZONTAL_3_2, label: "3:2", proportions: [3, 2] },
{ ratio: CameraAspectRatio.SQUARE_1_1, label: "1:1", proportions: [1, 1] },
{ ratio: CameraAspectRatio.VERTICAL_2_3, label: "2:3", proportions: [2, 3] },
{ ratio: CameraAspectRatio.VERTICAL_9_16, label: "9:16", proportions: [9, 16] },
];
export const AspectRatioMenu = () => {
const aspect = usePageSceneStore((s) => s.cameraAspectRatio);
const editor = useContext(EngineContext);
const handleChangeAspectRatio = (newRatio: CameraAspectRatio) => {
if (!editor) return;
setCameraAspect(editor, newRatio);
const current =
RATIO_OPTIONS.find((o) => o.ratio === aspect) ?? RATIO_OPTIONS[0];
const items: PopoverItem[] = RATIO_OPTIONS.map((option) => ({
label: option.label,
selected: option.ratio === aspect,
icon: <AspectRatioIcon ratio={option.proportions} />,
}));
const handleSelect = (item: PopoverItem) => {
const option = RATIO_OPTIONS.find((o) => o.label === item.label);
if (option && editor) setCameraAspect(editor, option.ratio);
};
return (
<div className="absolute right-0 top-0 m-2 flex flex-col items-end">
<ButtonDropdown
label={LABEL_BY_RATIO[aspect]}
className="shadow-xl"
icon={ICON_BY_RATIO[aspect]}
align="right"
showSelected={true}
options={[
{
label: "16:9",
icon: faRectangle,
className: "pl-4",
description: "Horizontal",
selected: aspect === CameraAspectRatio.HORIZONTAL_16_9,
onClick: () =>
handleChangeAspectRatio(CameraAspectRatio.HORIZONTAL_16_9),
},
{
label: "3:2",
icon: faRectangle,
className: "pl-4",
description: "Horizontal",
selected: aspect === CameraAspectRatio.HORIZONTAL_3_2,
onClick: () =>
handleChangeAspectRatio(CameraAspectRatio.HORIZONTAL_3_2),
},
{
label: "2:3",
icon: faRectangleVertical,
className: "pl-4",
description: "Vertical",
selected: aspect === CameraAspectRatio.VERTICAL_2_3,
onClick: () =>
handleChangeAspectRatio(CameraAspectRatio.VERTICAL_2_3),
},
{
label: "9:16",
icon: faRectangleVertical,
className: "pl-4",
description: "Vertical",
selected: aspect === CameraAspectRatio.VERTICAL_9_16,
onClick: () =>
handleChangeAspectRatio(CameraAspectRatio.VERTICAL_9_16),
},
{
label: "1:1",
icon: faSquare,
className: "pl-4",
description: "Squared",
selected: aspect === CameraAspectRatio.SQUARE_1_1,
onClick: () =>
handleChangeAspectRatio(CameraAspectRatio.SQUARE_1_1),
},
]}
<Tooltip content="Aspect ratio" position="bottom" delay={300} closeOnClick>
<PopoverMenu
items={items}
onSelect={handleSelect}
mode="toggle"
position="bottom"
panelTitle="Aspect Ratio"
showIconsInList
buttonClassName="glass glass-no-hover h-[34px] rounded-full px-3 text-xs font-medium shadow-xl"
triggerIcon={<AspectRatioIcon ratio={current.proportions} />}
/>
</div>
</Tooltip>
);
};
@@ -0,0 +1,50 @@
import { useContext } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faVideo, faUpDownLeftRight } from "@fortawesome/pro-solid-svg-icons";
import { Tooltip } from "@storyteller/ui-tooltip";
import { EngineContext } from "../../contexts/EngineContext";
import { toggleCameraView } from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { EditorStates } from "../../enums";
// Small status pill (top-left) showing which camera the viewport is driving:
// the free "Viewport" camera vs looking through a render camera. Clicking it
// toggles camera view. Hidden in record mode (that mode is self-evident).
export const CameraStatusPill = () => {
const editor = useContext(EngineContext);
const editorState = usePageSceneStore((s) => s.editorState);
const sceneMode = usePageSceneStore((s) => s.sceneMode);
const cameras = usePageSceneStore((s) => s.cameras);
if (sceneMode === "record") return null;
const inCameraView = editorState === EditorStates.CAMERA_VIEW;
const renderCameraLabel =
cameras.find((c) => c.id !== "main")?.label ?? "Camera";
return (
<Tooltip
content={inCameraView ? "Looking through render camera" : "Free viewport camera"}
position="bottom"
delay={300}
>
<button
type="button"
onClick={() => editor && toggleCameraView(editor)}
className={
inCameraView
? "flex h-[34px] items-center gap-2 rounded-full bg-brand-primary px-3 text-xs font-medium text-white shadow-xl transition-all duration-200"
: "glass glass-no-hover flex h-[34px] items-center gap-2 rounded-full px-3 text-xs font-medium text-base-fg/70 shadow-xl transition-all duration-200 hover:text-base-fg"
}
>
<FontAwesomeIcon
icon={inCameraView ? faVideo : faUpDownLeftRight}
className="h-3 w-3"
/>
{inCameraView ? renderCameraLabel : "Viewport"}
</button>
</Tooltip>
);
};
export default CameraStatusPill;
@@ -0,0 +1 @@
export { CameraStatusPill } from "./CameraStatusPill";
@@ -0,0 +1,148 @@
import { useContext, useEffect, useRef, useState } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faTrash,
faCloudArrowUp,
faSpinnerThird,
faTriangleExclamation,
faRotateRight,
} from "@fortawesome/pro-solid-svg-icons";
import { EngineContext } from "../../contexts/EngineContext";
import { usePageSceneStore } from "../../PageSceneStore";
// Output modal shown immediately after Capture/Record — it previews the local
// artifact (object URL, works offline), then uploads to the gallery and shows
// status. On success it opens the app Lightbox (all destinations) on the
// uploaded token; on failure it stays put with a Retry. Images auto-upload;
// videos wait for a manual Upload (they're large).
type UploadStatus = "idle" | "uploading" | "error";
export const CompletionModal = () => {
const editor = useContext(EngineContext);
const artifact = usePageSceneStore((s) => s.producedArtifact);
const clearProducedArtifact = usePageSceneStore(
(s) => s.clearProducedArtifact,
);
const [status, setStatus] = useState<UploadStatus>("idle");
const autoStartedRef = useRef(false);
const isVideo = artifact?.kind === "video";
const upload = async () => {
if (!editor?.adapter.uploadMedia || !artifact) return;
setStatus("uploading");
try {
const token = await editor.adapter.uploadMedia({
kind: artifact.kind,
blob: artifact.blob,
fileName: artifact.fileName,
});
// Hand off to the app Lightbox (destinations for this kind). It shows via
// the uploaded CDN URL, so revoking the local object URL now is safe.
editor.adapter.openMediaLightbox?.(token, artifact.kind);
clearProducedArtifact();
} catch {
setStatus("error");
}
};
// Images auto-upload as soon as the modal appears; reset when it closes.
useEffect(() => {
if (artifact?.kind === "image" && !autoStartedRef.current) {
autoStartedRef.current = true;
void upload();
}
if (!artifact) {
autoStartedRef.current = false;
setStatus("idle");
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [artifact]);
if (!artifact) return null;
const busy = status === "uploading";
return (
<div className="absolute inset-0 z-[70] flex items-center justify-center bg-black/60 p-6 backdrop-blur-sm">
<div className="glass glass-no-hover flex w-full max-w-xl flex-col gap-4 rounded-2xl p-5 text-white shadow-2xl">
<div className="text-sm font-semibold text-base-fg/90">
{isVideo ? "Recording complete" : "Capture complete"}
</div>
<div className="flex max-h-[52vh] items-center justify-center overflow-hidden rounded-xl bg-black/30">
{isVideo ? (
<video
src={artifact.objectUrl}
controls
autoPlay
loop
className="max-h-[52vh] w-full object-contain"
/>
) : (
<img
src={artifact.objectUrl}
alt="Captured frame"
className="max-h-[52vh] w-full object-contain"
/>
)}
</div>
{/* upload status */}
{busy && (
<div className="flex items-center gap-2 text-sm text-base-fg/70">
<FontAwesomeIcon
icon={faSpinnerThird}
className="h-4 w-4 animate-spin"
/>
Uploading to gallery…
</div>
)}
{status === "error" && (
<div className="flex items-center gap-2 text-sm text-red">
<FontAwesomeIcon
icon={faTriangleExclamation}
className="h-4 w-4"
/>
Upload failed — check your connection and retry.
</div>
)}
{/* actions */}
<div className="flex items-center justify-between gap-2">
<button
type="button"
onClick={() => clearProducedArtifact()}
disabled={busy}
className="flex h-9 items-center gap-2 rounded-full px-3 text-sm text-base-fg/60 transition-colors hover:bg-white/10 hover:text-base-fg disabled:opacity-50"
>
<FontAwesomeIcon icon={faTrash} className="h-3.5 w-3.5" />
Discard
</button>
{status === "error" ? (
<button
type="button"
onClick={() => void upload()}
className="flex h-9 items-center gap-2 rounded-full bg-brand-primary px-4 text-sm font-medium text-white transition-transform hover:scale-[1.03]"
>
<FontAwesomeIcon icon={faRotateRight} className="h-3.5 w-3.5" />
Retry
</button>
) : isVideo && status === "idle" ? (
<button
type="button"
onClick={() => void upload()}
className="flex h-9 items-center gap-2 rounded-full bg-brand-primary px-4 text-sm font-medium text-white transition-transform hover:scale-[1.03]"
>
<FontAwesomeIcon icon={faCloudArrowUp} className="h-3.5 w-3.5" />
Upload &amp; continue
</button>
) : null}
</div>
</div>
</div>
);
};
export default CompletionModal;
@@ -0,0 +1 @@
export { CompletionModal } from "./CompletionModal";
@@ -25,7 +25,7 @@ import {
import { twMerge } from "tailwind-merge";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import { setTransformMode } from "../../actions";
import { openAssetModal, setTransformMode } from "../../actions";
import { usePageSceneStore, type TransformMode } from "../../PageSceneStore";
import { AssetModal } from "../AssetMenu";
@@ -42,9 +42,6 @@ export const Controls3D = ({
const editor = useContext(EngineContext);
const {
assetModalVisible,
setAssetModalVisible,
setAssetModalVisibleDuringDrag,
setAssetDraggingUnder,
selectedMode,
transformSpace,
currentUserToken,
@@ -52,9 +49,6 @@ export const Controls3D = ({
} = usePageSceneStore(
useShallow((s) => ({
assetModalVisible: s.assetModalVisible,
setAssetModalVisible: s.setAssetModalVisible,
setAssetModalVisibleDuringDrag: s.setAssetModalVisibleDuringDrag,
setAssetDraggingUnder: s.setAssetDraggingUnder,
selectedMode: s.selectedMode,
transformSpace: s.transformSpace,
currentUserToken: s.currentUserToken,
@@ -100,14 +94,6 @@ export const Controls3D = ({
}
};
const handleOpenModal = () => {
setAssetModalVisibleDuringDrag(true);
setAssetModalVisible(true);
// Clear any leftover drag-under state so the panel opens fully shown (a
// reopen-off drag leaves it faded-hidden until the next open).
setAssetDraggingUnder(false);
};
const handleOpenCreate3dModal = () => {
editor?.adapter.navigateToImageTo3D();
};
@@ -132,7 +118,7 @@ export const Controls3D = ({
}
switch (action) {
case "presets":
handleOpenModal();
openAssetModal();
break;
case "library":
handleOpenGalleryModal();
@@ -16,7 +16,7 @@ import {
usePageSceneStore,
useIsVisitingOthersScene,
} from "../../PageSceneStore";
import { CameraAspectRatio, ToastTypes } from "../../enums";
import { DEFAULT_CAMERA_ASPECT_RATIO, ToastTypes } from "../../enums";
import { getSceneGenerationMetaData } from "../../sceneMetadata";
import { LoadUserScenes } from "./LoadUserScenes";
import { Help } from "./Help/Help";
@@ -54,7 +54,7 @@ export const ControlsTopButtons = () => {
const handleResetScene = () => {
editor?.cameraController.changeRenderCameraAspectRatio(
CameraAspectRatio.HORIZONTAL_3_2,
DEFAULT_CAMERA_ASPECT_RATIO,
);
};
@@ -0,0 +1,39 @@
import { useContext } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faArrowRightFromBracket } from "@fortawesome/pro-solid-svg-icons";
import { twMerge } from "tailwind-merge";
import { EngineContext } from "../../contexts/EngineContext";
import { toggleCameraView } from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { EditorStates } from "../../enums";
// Shown only while looking through a render camera (CAMERA_VIEW). Exits back
// to the free editing view. Positionless by default — the host (Stage3DBody)
// docks it next to the timeline bar via `className`.
export const ExitCameraView = ({ className }: { className?: string }) => {
const editor = useContext(EngineContext);
const editorState = usePageSceneStore((s) => s.editorState);
const sceneMode = usePageSceneStore((s) => s.sceneMode);
// In record mode the viewport is also CAMERA_VIEW, but the mode pill
// handles leaving — don't show the manual exit button there.
if (editorState !== EditorStates.CAMERA_VIEW || sceneMode === "record") {
return null;
}
return (
<button
type="button"
onClick={() => editor && toggleCameraView(editor)}
className={twMerge(
"flex h-11 items-center gap-2 whitespace-nowrap rounded-xl bg-brand-primary px-4 text-sm font-medium text-white shadow-xl transition-colors hover:bg-brand-primary/90",
className,
)}
>
<FontAwesomeIcon icon={faArrowRightFromBracket} className="h-3.5 w-3.5" />
Exit
</button>
);
};
export default ExitCameraView;
@@ -0,0 +1 @@
export { ExitCameraView } from "./ExitCameraView";
@@ -1,41 +1,32 @@
import { ChangeEvent, useContext, useEffect, useState } from "react";
import { useContext } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faCircleXmark,
faArrowRightToBracket,
faEye,
faEyeSlash,
faListTree,
faLock,
faLockOpen,
faPlus,
faXmark,
} from "@fortawesome/pro-solid-svg-icons";
import { Input } from "@storyteller/ui-input";
import { Button } from "@storyteller/ui-button";
import { Transition } from "@headlessui/react";
import { twMerge } from "tailwind-merge";
import { useSignals } from "@preact/signals-react/runtime";
import { EngineContext } from "../../contexts/EngineContext";
import { OutlinerItem, usePageSceneStore } from "../../PageSceneStore";
import { useViewportSize } from "../../hooks/useViewportSize";
import { CameraAspectRatio } from "../../enums";
import { openAssetModal } from "../../actions/openAssetModal";
import { toggleObjectLock } from "../../actions/toggleObjectLock";
import { toggleObjectVisibility } from "../../actions/toggleObjectVisibility";
import { toggleCameraView } from "../../actions/cameraView";
const OutlinerRow = ({ item }: { item: OutlinerItem }) => {
const [hovered, setHovered] = useState(false);
const isSelected = usePageSceneStore(
(s) => s.outlinerSelectedItem?.id === item.id,
);
const editorEngine = useContext(EngineContext);
// Delete object logic here
const handleDeleteKeyPress = (event: React.KeyboardEvent<HTMLDivElement>) => {
if (event.key === "Delete") {
//console.log("Delete key pressed for item:", item.id);
}
};
// Double click logic here
const handleDoubleClick = () => {
editorEngine?.sceneManager?.double_click();
@@ -58,55 +49,71 @@ const OutlinerRow = ({ item }: { item: OutlinerItem }) => {
toggleObjectVisibility(editorEngine, item.id);
};
const handleViewFromCamera = (e: React.MouseEvent) => {
e.stopPropagation();
if (!editorEngine) return;
toggleCameraView(editorEngine);
};
return (
<div
role="button"
className={twMerge(
"flex cursor-pointer justify-between px-4 py-[7px] text-[13px] font-normal text-white/80 outline-none transition-all duration-100 hover:bg-action-900/35 focus:outline-none",
"flex cursor-pointer items-center justify-between gap-2 rounded-xl border border-transparent px-3 py-2 outline-none transition-colors duration-100 hover:bg-white/5 focus:outline-none",
isSelected &&
"bg-brand-primary/80 font-medium text-white hover:bg-brand-primary/80",
"border-brand-primary bg-brand-primary/15 hover:bg-brand-primary/15",
)}
onMouseEnter={() => setHovered(true)}
onMouseLeave={() => setHovered(false)}
onDoubleClick={handleDoubleClick}
onKeyDown={handleDeleteKeyPress}
onClick={handleSelect}
tabIndex={0}
>
<span className="flex items-center gap-2.5">
<div className="flex w-4 items-center justify-center">
<span className="flex min-w-0 items-center gap-3">
<div className="flex w-5 shrink-0 items-center justify-center text-base text-white/85">
<FontAwesomeIcon icon={item.icon} />
</div>
{item.name}
<span className="min-w-0">
<span className="block truncate text-[13px] font-semibold text-white">
{item.name}
</span>
<span className="block truncate text-xs text-white/50">
{item.type}
</span>
</span>
</span>
<div className="flex gap-3">
<div className="flex shrink-0 items-center gap-3">
{item.isCamera && (
<button
onClick={handleViewFromCamera}
title="View from camera"
className="text-white/60 transition-colors duration-100 hover:text-white text-xs"
>
<div className="w-3">
<FontAwesomeIcon icon={faArrowRightToBracket} />
</div>
</button>
)}
<button
onClick={handleToggleLock}
style={{
opacity: hovered || item.locked ? 1 : 0,
}}
title={item.locked ? "Unlock" : "Lock"}
className={twMerge(
"text-white/60 transition-colors duration-100 hover:text-white text-xs",
item.locked && "text-white/90",
)}
>
<div className="w-3">
<FontAwesomeIcon
icon={item.locked ? faLock : faLockOpen}
className="opacity-80 transition-opacity duration-100 hover:opacity-100"
/>
<FontAwesomeIcon icon={item.locked ? faLock : faLockOpen} />
</div>
</button>
<button
onClick={handleToggleVisibility}
style={{
opacity: hovered || !item.visible ? 1 : 0,
}}
title={item.visible ? "Hide" : "Show"}
className={twMerge(
"text-white/60 transition-colors duration-100 hover:text-white text-xs",
!item.visible && "text-white/90",
)}
>
<div className="w-4">
<FontAwesomeIcon
icon={item.visible ? faEye : faEyeSlash}
className={twMerge(
"opacity-80 transition-opacity duration-100 hover:opacity-100",
item.locked && "text-white/90",
)}
/>
<div className="w-3">
<FontAwesomeIcon icon={item.visible ? faEye : faEyeSlash} />
</div>
</button>
</div>
@@ -116,58 +123,16 @@ const OutlinerRow = ({ item }: { item: OutlinerItem }) => {
export const Outliner = () => {
useSignals();
const camAspect = usePageSceneStore((s) => s.cameraAspectRatio);
const items = usePageSceneStore((s) => s.outlinerItems);
const showing = usePageSceneStore((s) => s.outlinerShowing);
const [searchTerm, setSearchTerm] = useState("");
const [editorHeight, setEditorHeight] = useState(0);
const handleSearchChange = (e: ChangeEvent<HTMLInputElement>) => {
setSearchTerm(e.target.value);
};
const clearSearch = () => {
setSearchTerm("");
};
// Filter items based on search term
const filteredItems = items.filter((item) =>
item.name.toLowerCase().includes(searchTerm.toLowerCase()),
);
const viewport = useViewportSize();
useEffect(() => {
setEditorHeight(viewport.height - 56);
}, [viewport.height]);
const getOutlinerHeightClass = () => {
if (viewport.width >= 2000) {
if (camAspect === CameraAspectRatio.VERTICAL_9_16) {
return `${editorHeight * 0.5 - 120}px`;
} else if (camAspect === CameraAspectRatio.SQUARE_1_1) {
return `${editorHeight * 0.42}px`;
} else {
return `${editorHeight * 0.54}px`;
}
}
if (camAspect === CameraAspectRatio.VERTICAL_9_16) {
return `${editorHeight * 0.7 - 10}px`;
} else if (camAspect === CameraAspectRatio.SQUARE_1_1) {
return `${editorHeight * 0.7}px`;
} else {
return `${editorHeight * 0.7}px`;
}
};
return (
<Transition
as="div"
show={showing}
className={twMerge(
"glass flex max-h-[34vh] w-[240px] origin-bottom-left flex-col overflow-hidden rounded-lg shadow-lg",
"flex h-[45vh] w-[260px] flex-col overflow-hidden rounded-xl border border-ui-panel-border bg-ui-panel shadow-lg",
)}
style={{ height: getOutlinerHeightClass() }}
enter="transition-opacity duration-150"
enterFrom="opacity-0"
enterTo="opacity-100"
@@ -175,11 +140,15 @@ export const Outliner = () => {
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
<div className="flex items-center px-4 pt-3">
<h1 className="grow text-base font-semibold">
<FontAwesomeIcon icon={faListTree} className="mb-0 mr-2" />
Outliner
</h1>
<div className="flex items-center px-4 pb-1 pt-3">
<h1 className="grow text-base font-semibold">Scene</h1>
<Button
icon={faPlus}
className="h-6 bg-transparent px-1.5 text-sm font-medium text-white/80 hover:bg-transparent hover:text-white"
onClick={openAssetModal}
>
Add
</Button>
<Button
icon={faXmark}
className="h-5 bg-transparent p-0 text-xl opacity-50 hover:bg-transparent hover:opacity-90"
@@ -189,26 +158,12 @@ export const Outliner = () => {
/>
</div>
<div className="relative mx-4 my-2.5">
<Input
inputClassName="h-8 rounded-lg text-sm pr-8"
placeholder="Search..."
value={searchTerm}
onInput={handleSearchChange}
/>
{searchTerm && (
<FontAwesomeIcon
icon={faCircleXmark}
className="absolute right-2 top-1/2 -translate-y-1/2 transform cursor-pointer opacity-50 transition-all duration-100 hover:opacity-100"
onClick={clearSearch}
/>
)}
</div>
<div className="grow overflow-auto">
{filteredItems.map((item) => (
<OutlinerRow key={item.id} item={item} />
))}
<div className="grow overflow-auto p-2">
<div className="flex flex-col gap-1">
{items.map((item) => (
<OutlinerRow key={item.id} item={item} />
))}
</div>
</div>
</Transition>
);
@@ -0,0 +1,115 @@
import { useContext } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faCamera, faCircle } from "@fortawesome/pro-solid-svg-icons";
import { Tooltip } from "@storyteller/ui-tooltip";
import { EngineContext } from "../../contexts/EngineContext";
import { usePageSceneStore } from "../../PageSceneStore";
import { recordTimeline } from "../../engine/recording/TimelineRecorder";
import { TimelineBar } from "../Timeline";
// Record-mode controls: a read-only playback bar (when a timeline exists) plus
// Capture (still) and Record (timeline→video).
// Capture → auto-upload to gallery → open the app Lightbox on the token.
// Record → produce the clip → hand to the video review modal (manual upload,
// since videos are large) via producedArtifact.
export const RecordControls = () => {
const editor = useContext(EngineContext);
const timelineExists = usePageSceneStore((s) => s.timelineExists);
const aspectRatio = usePageSceneStore((s) => s.cameraAspectRatio);
const recordingProgress = usePageSceneStore((s) => s.recordingProgress);
const setProducedArtifact = usePageSceneStore((s) => s.setProducedArtifact);
const setRecordingProgress = usePageSceneStore((s) => s.setRecordingProgress);
const busy = recordingProgress !== null;
// Snapshot the composed frame → hand to the review modal, which previews it
// immediately and auto-uploads to the gallery (with status) → app Lightbox.
const handleCapture = () => {
if (!editor || busy) return;
setRecordingProgress({ phase: "capturing", pct: 0 });
// Defer so the overlay paints before the (sync) snapshot work.
requestAnimationFrame(() => {
try {
const snap = editor.snapShotOfCurrentFrame(false);
if (snap) {
setProducedArtifact({
kind: "image",
blob: snap.file,
objectUrl: URL.createObjectURL(snap.file),
fileName: snap.file.name,
mimeType: "image/png",
aspectRatio,
});
}
} finally {
setRecordingProgress(null);
}
});
};
// Encode the timeline; the produced clip opens the review modal for a manual
// upload (videos are large, so we don't auto-upload).
const handleRecord = async () => {
if (!editor || busy || !timelineExists) return;
setRecordingProgress({ phase: "encoding", pct: 0 });
try {
const result = await recordTimeline(editor, {
onProgress: (pct) => setRecordingProgress({ phase: "encoding", pct }),
});
if (result) {
setProducedArtifact({
kind: "video",
blob: result.blob,
objectUrl: URL.createObjectURL(result.blob),
fileName: result.fileName,
mimeType: result.mimeType,
aspectRatio,
});
}
} finally {
setRecordingProgress(null);
}
};
return (
<div className="absolute bottom-6 left-1/2 z-30 flex -translate-x-1/2 flex-col items-center gap-3">
{timelineExists && (
<div className="w-[70vw] max-w-3xl">
<TimelineBar readOnly />
</div>
)}
<div className="flex items-center gap-3">
<button
type="button"
onClick={handleCapture}
disabled={busy}
className="glass glass-no-hover flex items-center gap-2 rounded-full px-5 py-2.5 text-sm font-medium text-base-fg shadow-xl hover:bg-white/10 disabled:opacity-50"
>
<FontAwesomeIcon icon={faCamera} className="h-3.5 w-3.5" />
Capture
</button>
<Tooltip
content={
timelineExists
? "Render the timeline to video"
: "Add an animation timeline first"
}
position="top"
delay={200}
>
<button
type="button"
onClick={handleRecord}
disabled={busy || !timelineExists}
className="flex items-center gap-2 rounded-full bg-red px-5 py-2.5 text-sm font-semibold text-white shadow-xl transition-transform hover:scale-[1.03] disabled:opacity-50 disabled:hover:scale-100"
>
<FontAwesomeIcon icon={faCircle} className="h-3 w-3" />
Record
</button>
</Tooltip>
</div>
</div>
);
};
export default RecordControls;
@@ -0,0 +1 @@
export { RecordControls } from "./RecordControls";
@@ -0,0 +1,27 @@
import { LoadingDots } from "@storyteller/ui-loading";
import { usePageSceneStore } from "../../PageSceneStore";
// Opaque full-viewport cover shown while a Capture/Record is in flight. It
// hides the 3D scene (freeing GPU during frame encoding) and shows progress.
export const RenderOverlay = () => {
const progress = usePageSceneStore((s) => s.recordingProgress);
const pct = Math.round((progress?.pct ?? 0) * 100);
const message =
progress?.phase === "encoding"
? `Rendering video… ${pct}%`
: progress?.phase === "uploading"
? "Uploading to gallery…"
: "Capturing frame…";
return (
<LoadingDots
className="absolute left-0 top-0 z-[60]"
isShowing={progress !== null}
type="bricks"
message={message}
/>
);
};
export default RenderOverlay;
@@ -0,0 +1 @@
export { RenderOverlay } from "./RenderOverlay";
@@ -0,0 +1,180 @@
import { useContext, useEffect, useMemo, useState } from "react";
import { usePageSceneStore } from "../../PageSceneStore";
import { EngineContext } from "../../contexts/EngineContext";
import { CameraAspectRatio } from "../../enums";
// Camera-view framing overlay: an inset, aspect-correct frame showing the
// EXACT render framing (the engine offsets the viewport projection into this
// rect via CameraController.setCameraFrameRect), with the surrounding scene
// dimmed, rule-of-thirds guides, and the focal-length label. The frame is
// fitted into the area left free by the surrounding chrome so the bottom
// timeline/record controls never cover the shot.
// Reserved chrome, in px. Vertical insets differ per mode:
// build/camera-view has the mode pill + transform toolbar up top and the
// collapsed timeline bar + exit-view button at the bottom; record mode has
// only the pill up top but the playback bar + capture/record cluster below.
const TOP_INSET_BUILD = 124;
const TOP_INSET_RECORD = 76;
const BOTTOM_INSET_BUILD = 88;
// Fallback while the expanded editor hasn't been measured yet.
const BOTTOM_INSET_BUILD_TIMELINE_EXPANDED = 240;
const BOTTOM_INSET_RECORD = 152;
// The expanded timeline editor sits at bottom-4 (16px); keep a little air
// between it and the frame.
const TIMELINE_EDITOR_CLEARANCE = 28;
const SIDE_INSET = 24;
// Below this the insets don't fit; fall back to a slim uniform margin so
// tiny windows still get a usable frame.
const MIN_FRAME_EDGE = 120;
const FALLBACK_MARGIN = 16;
const RATIO_VALUE: Record<CameraAspectRatio, number> = {
[CameraAspectRatio.HORIZONTAL_16_9]: 16 / 9,
[CameraAspectRatio.HORIZONTAL_3_2]: 3 / 2,
[CameraAspectRatio.SQUARE_1_1]: 1,
[CameraAspectRatio.VERTICAL_2_3]: 2 / 3,
[CameraAspectRatio.VERTICAL_9_16]: 9 / 16,
};
type FrameRect = { x: number; y: number; width: number; height: number };
const fitFrame = (
width: number,
height: number,
ratio: number,
top: number,
bottom: number,
): FrameRect => {
let safeWidth = width - SIDE_INSET * 2;
let safeHeight = height - top - bottom;
let safeTop = top;
if (safeWidth < MIN_FRAME_EDGE || safeHeight < MIN_FRAME_EDGE) {
safeWidth = width - FALLBACK_MARGIN * 2;
safeHeight = height - FALLBACK_MARGIN * 2;
safeTop = FALLBACK_MARGIN;
}
const frameWidth = Math.min(safeWidth, safeHeight * ratio);
const frameHeight = frameWidth / ratio;
return {
x: (width - frameWidth) / 2,
y: safeTop + (safeHeight - frameHeight) / 2,
width: frameWidth,
height: frameHeight,
};
};
export const CameraFrame = ({
width,
height,
}: {
width: number;
height: number;
}) => {
const editor = useContext(EngineContext);
const camAspect = usePageSceneStore((s) => s.cameraAspectRatio);
const sceneMode = usePageSceneStore((s) => s.sceneMode);
const timelineExpanded = usePageSceneStore((s) => s.timelineExpanded);
const cameras = usePageSceneStore((s) => s.cameras);
const selectedCameraId = usePageSceneStore((s) => s.selectedCameraId);
const isRecord = sceneMode === "record";
// The expanded timeline editor's height depends on its track count, so
// measure it live instead of guessing a fixed inset.
const [editorPanelHeight, setEditorPanelHeight] = useState(0);
useEffect(() => {
if (isRecord || !timelineExpanded) return undefined;
const el = document.getElementById("timeline-editor");
if (!el) return undefined;
setEditorPanelHeight(el.offsetHeight);
const observer = new ResizeObserver(() =>
setEditorPanelHeight(el.offsetHeight),
);
observer.observe(el);
return () => observer.disconnect();
}, [isRecord, timelineExpanded]);
const frame = useMemo(() => {
if (width <= 0 || height <= 0) return null;
const top = isRecord ? TOP_INSET_RECORD : TOP_INSET_BUILD;
const bottom = isRecord
? BOTTOM_INSET_RECORD
: timelineExpanded
? editorPanelHeight > 0
? editorPanelHeight + TIMELINE_EDITOR_CLEARANCE
: BOTTOM_INSET_BUILD_TIMELINE_EXPANDED
: BOTTOM_INSET_BUILD;
return fitFrame(width, height, RATIO_VALUE[camAspect], top, bottom);
}, [width, height, camAspect, isRecord, timelineExpanded, editorPanelHeight]);
// Hand the rect to the engine so the viewport projection maps the exact
// render framing into it; clear on unmount (leaving camera view/record).
useEffect(() => {
if (!editor || !frame) return;
editor.cameraController.setCameraFrameRect({
x: frame.x,
y: frame.y,
width: frame.width,
height: frame.height,
canvasWidth: width,
canvasHeight: height,
});
}, [editor, frame, width, height]);
useEffect(() => {
return () => editor?.cameraController.setCameraFrameRect(null);
}, [editor]);
if (!frame) return null;
const focalLength = cameras.find((c) => c.id === selectedCameraId)
?.focalLength;
const right = frame.x + frame.width;
const bottomEdge = frame.y + frame.height;
return (
<div
id="letterbox"
className="pointer-events-none absolute inset-0 select-none"
>
{/* Dim everything outside the frame (still visible, won't render). */}
<div
className="absolute left-0 right-0 top-0 bg-black/60"
style={{ height: frame.y }}
/>
<div
className="absolute bottom-0 left-0 right-0 bg-black/60"
style={{ height: height - bottomEdge }}
/>
<div
className="absolute left-0 bg-black/60"
style={{ top: frame.y, height: frame.height, width: frame.x }}
/>
<div
className="absolute right-0 bg-black/60"
style={{ top: frame.y, height: frame.height, width: width - right }}
/>
{/* The frame itself: border, rule-of-thirds guides, focal label. */}
<div
className="absolute border border-white/40"
style={{
left: frame.x,
top: frame.y,
width: frame.width,
height: frame.height,
}}
>
<div className="absolute bottom-0 left-1/3 top-0 w-px bg-white/15" />
<div className="absolute bottom-0 left-2/3 top-0 w-px bg-white/15" />
<div className="absolute left-0 right-0 top-1/3 h-px bg-white/15" />
<div className="absolute left-0 right-0 top-2/3 h-px bg-white/15" />
{focalLength !== undefined && (
<div className="absolute left-3 top-2 text-sm font-medium text-white/80 drop-shadow">
{focalLength}mm
</div>
)}
</div>
</div>
);
};
@@ -1,63 +0,0 @@
import { twMerge } from "tailwind-merge";
import { usePageSceneStore } from "../../PageSceneStore";
import {
calcMatteWidth,
calcMatteHeight,
getMatteOrientation,
MatteOrientation,
} from "./helpers";
export const Letterbox = ({
isShowing,
width,
height,
}: {
isShowing: boolean;
width: number;
height: number;
}) => {
const camAspect = usePageSceneStore((s) => s.cameraAspectRatio);
const matteOri = getMatteOrientation({ camAspect, width, height });
const matteWidth = calcMatteWidth({ matteOri, camAspect, width, height });
const matteHeight = calcMatteHeight({ matteOri, camAspect, width, height });
return (
<div
id="letterbox"
className={twMerge(
"user-select-none pointer-events-none absolute left-0 top-0 flex h-full w-full justify-between",
matteOri === MatteOrientation.TOP_BOTTOM ? "flex-col" : null,
)}
>
<Matte matteOri={matteOri} width={matteWidth} height={matteHeight} />
<Matte matteOri={matteOri} width={matteWidth} height={matteHeight} />
</div>
);
};
const Matte = ({
matteOri,
width,
height,
}: {
matteOri: MatteOrientation;
width?: number;
height?: number;
}) => {
if (matteOri === MatteOrientation.TOP_BOTTOM) {
return (
<div
className="pointer-events-none h-20 w-full bg-black/30 brightness-75"
style={{ height: `${height}px` }}
/>
);
}
return (
<div
className="pointer-events-none h-full w-80 bg-black/30 brightness-75"
style={{ width: `${width}px` }}
/>
);
};
@@ -1,9 +1,17 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { usePageSceneStore } from "../../PageSceneStore";
import { Letterbox } from "./Letterbox";
import { EditorStates } from "../../enums";
import { CameraFrame } from "./CameraFrame";
export const SceneContainer = ({ children }: { children: React.ReactNode }) => {
const editorLetterBox = usePageSceneStore((s) => s.editorLetterBox);
const editorState = usePageSceneStore((s) => s.editorState);
const sceneMode = usePageSceneStore((s) => s.sceneMode);
// The framing overlay only makes sense while rendering through a camera:
// camera view and record mode. The free viewport shows no aspect mattes.
const showCameraFrame =
editorLetterBox &&
(editorState === EditorStates.CAMERA_VIEW || sceneMode === "record");
const [size, setSize] = useState({ width: 0, height: 0 });
const nodeRef = useRef<HTMLDivElement | null>(null);
@@ -36,11 +44,9 @@ export const SceneContainer = ({ children }: { children: React.ReactNode }) => {
className="relative h-full w-full"
>
{children}
<Letterbox
isShowing={editorLetterBox}
width={size.width}
height={size.height}
/>
{showCameraFrame && (
<CameraFrame width={size.width} height={size.height} />
)}
</div>
);
};
@@ -1,117 +0,0 @@
import { CameraAspectRatio } from "../../enums";
export enum MatteOrientation {
TOP_BOTTOM,
LEFT_RIGHT,
}
export const getMatteOrientation = ({
camAspect,
width,
height,
}: {
camAspect: CameraAspectRatio;
width: number;
height: number;
}): MatteOrientation => {
switch (camAspect) {
case CameraAspectRatio.SQUARE_1_1: {
return width > height
? MatteOrientation.LEFT_RIGHT
: MatteOrientation.TOP_BOTTOM;
}
case CameraAspectRatio.VERTICAL_9_16: {
return (height / 16) * 9 - width >= 0
? MatteOrientation.TOP_BOTTOM
: MatteOrientation.LEFT_RIGHT;
}
case CameraAspectRatio.VERTICAL_2_3: {
return (height / 3) * 2 - width >= 0
? MatteOrientation.TOP_BOTTOM
: MatteOrientation.LEFT_RIGHT;
}
case CameraAspectRatio.HORIZONTAL_3_2: {
return (width / 3) * 2 - height >= 0
? MatteOrientation.LEFT_RIGHT
: MatteOrientation.TOP_BOTTOM;
}
case CameraAspectRatio.HORIZONTAL_16_9:
default: {
return (width / 16) * 9 - height >= 0
? MatteOrientation.LEFT_RIGHT
: MatteOrientation.TOP_BOTTOM;
}
}
};
export const calcMatteWidth = ({
matteOri,
camAspect,
width,
height,
}: {
matteOri: MatteOrientation;
camAspect: CameraAspectRatio;
width: number;
height: number;
}): number | undefined => {
if (matteOri === MatteOrientation.TOP_BOTTOM) {
return undefined;
}
switch (camAspect) {
case CameraAspectRatio.SQUARE_1_1: {
return (width - height) / 2;
}
case CameraAspectRatio.HORIZONTAL_16_9: {
return (width - (height / 9) * 16) / 2;
}
case CameraAspectRatio.VERTICAL_9_16: {
return (width - (height / 16) * 9) / 2;
}
case CameraAspectRatio.HORIZONTAL_3_2: {
return (width - (height / 2) * 3) / 2;
}
case CameraAspectRatio.VERTICAL_2_3: {
return (width - (height / 3) * 2) / 2;
}
default: {
return undefined;
}
}
};
export const calcMatteHeight = ({
matteOri,
camAspect,
width,
height,
}: {
matteOri: MatteOrientation;
camAspect: CameraAspectRatio;
width: number;
height: number;
}): number | undefined => {
if (matteOri === MatteOrientation.LEFT_RIGHT) {
return undefined;
}
switch (camAspect) {
case CameraAspectRatio.SQUARE_1_1: {
return (height - width) / 2;
}
case CameraAspectRatio.HORIZONTAL_16_9: {
return (height - (width / 16) * 9) / 2;
}
case CameraAspectRatio.VERTICAL_9_16: {
return (height - (width / 9) * 16) / 2;
}
case CameraAspectRatio.HORIZONTAL_3_2: {
return (height - (width / 3) * 2) / 2;
}
case CameraAspectRatio.VERTICAL_2_3: {
return (height - (width / 2) * 3) / 2;
}
default: {
return undefined;
}
}
};
@@ -0,0 +1,34 @@
import { useShallow } from "zustand/shallow";
import { TabSelector, type TabItem } from "@storyteller/ui-tab-selector";
import { usePageSceneStore, type SceneMode } from "../../PageSceneStore";
const TABS: TabItem[] = [
{ id: "build", label: "Build" },
{ id: "record", label: "Record" },
];
export const SceneModePill = () => {
const { sceneMode, setSceneMode } = usePageSceneStore(
useShallow((s) => ({
sceneMode: s.sceneMode,
setSceneMode: s.setSceneMode,
})),
);
return (
<div className="flex justify-center pt-3">
<TabSelector
tabs={TABS}
activeTab={sceneMode}
onTabChange={(id) => setSceneMode(id as SceneMode)}
className="w-auto"
listClassName="rounded-full px-1"
tabClassName="rounded-full px-5 py-1"
indicatorClassName="rounded-full bg-white/95"
selectedTabClassName="text-black"
/>
</div>
);
};
export default SceneModePill;
@@ -0,0 +1 @@
export { SceneModePill } from "./SceneModePill";
@@ -0,0 +1,65 @@
import { useContext, useEffect, useRef, useState } from "react";
import { EngineContext } from "../../contexts/EngineContext";
import { setTimelineDuration } from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { formatTimecode } from "./timelineUtils";
// The total-duration readout, click-to-edit. Click → number input (seconds);
// Enter/blur commits, Escape cancels. Backed by TimelineController.setDuration
// (clamped to [1,60], covers the required 5–30s range).
export const DurationLabel = ({ className = "" }: { className?: string }) => {
const editor = useContext(EngineContext);
const duration = usePageSceneStore((s) => s.timelineDuration);
const [editing, setEditing] = useState(false);
const [value, setValue] = useState("");
const inputRef = useRef<HTMLInputElement>(null);
useEffect(() => {
if (editing) inputRef.current?.select();
}, [editing]);
const commit = () => {
const secs = parseFloat(value);
if (editor && Number.isFinite(secs)) setTimelineDuration(editor, secs);
setEditing(false);
};
if (editing) {
return (
<input
ref={inputRef}
type="number"
min={1}
max={60}
step={1}
value={value}
onChange={(e) => setValue(e.target.value)}
onBlur={commit}
onKeyDown={(e) => {
if (e.key === "Enter") commit();
if (e.key === "Escape") setEditing(false);
}}
onClick={(e) => e.stopPropagation()}
onMouseDown={(e) => e.stopPropagation()}
className={`w-12 rounded bg-white/10 px-1 text-center tabular-nums text-base-fg outline-none ${className}`}
/>
);
}
return (
<button
type="button"
title="Set timeline duration (seconds)"
onClick={(e) => {
e.stopPropagation();
setValue(String(Math.round(duration)));
setEditing(true);
}}
className={`tabular-nums transition-colors hover:text-base-fg ${className}`}
>
{formatTimecode(duration)}
</button>
);
};
export default DurationLabel;
@@ -0,0 +1,189 @@
import { useContext, useRef, useState } from "react";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import { setKeyframeEasing } from "../../actions";
import {
EASING_PRESETS,
type EasingPresetName,
type EasingSpec,
type Keyframe,
} from "../../engine/timeline/types";
// The Motion popover edits a keyframe's easing curve (the interpolation
// INTO the next keyframe). Presets across the top, an editable cubic-bezier
// curve below with two draggable handles and a dotted linear reference.
const BOX = 150; // svg drawing box (px), domain [0,1]×[0,1]
const PAD = 14; // padding so handles/labels aren't clipped
const SPAN = BOX - PAD * 2;
// Map normalized (x,y) in [0,1] → svg coords (y is inverted so 1 is at top).
const sx = (x: number) => PAD + x * SPAN;
const sy = (y: number) => PAD + (1 - y) * SPAN;
const PRESETS: { name: EasingPresetName; label: string }[] = [
{ name: "linear", label: "Linear" },
{ name: "easeIn", label: "In" },
{ name: "easeOut", label: "Out" },
{ name: "easeInOut", label: "In-Out" },
];
interface MotionPopoverProps {
keyframe: Keyframe;
// Horizontal anchor as a percentage across the editor (playhead/keyframe pos).
leftPercent: number;
}
export const MotionPopover = ({ keyframe, leftPercent }: MotionPopoverProps) => {
const editor = useContext(EngineContext);
const svgRef = useRef<SVGSVGElement>(null);
const [dragging, setDragging] = useState<null | "p1" | "p2">(null);
const easing = keyframe.easing;
const commit = (next: EasingSpec) => {
if (editor) setKeyframeEasing(editor, keyframe.id, next);
};
const pointerToNorm = (e: React.PointerEvent) => {
const svg = svgRef.current;
if (!svg) return null;
const rect = svg.getBoundingClientRect();
// rect may be scaled vs the BOX viewBox; normalize via SPAN in view units.
const vx = ((e.clientX - rect.left) / rect.width) * BOX;
const vy = ((e.clientY - rect.top) / rect.height) * BOX;
const x = Math.max(0, Math.min(1, (vx - PAD) / SPAN));
const y = Math.max(0, Math.min(1, 1 - (vy - PAD) / SPAN));
return { x, y };
};
const onPointerMove = (e: React.PointerEvent) => {
if (!dragging) return;
const n = pointerToNorm(e);
if (!n) return;
if (dragging === "p1") commit({ ...easing, p1x: n.x, p1y: n.y });
else commit({ ...easing, p2x: n.x, p2y: n.y });
};
const curvePath = `M ${sx(0)} ${sy(0)} C ${sx(easing.p1x)} ${sy(easing.p1y)} ${sx(
easing.p2x,
)} ${sy(easing.p2y)} ${sx(1)} ${sy(1)}`;
return (
<div
id="motion-popover"
className="glass glass-no-hover pointer-events-auto absolute bottom-full z-50 mb-2 w-[180px] -translate-x-1/2 select-none rounded-2xl p-3 shadow-xl backdrop-blur-md"
style={{ left: `${Math.max(12, Math.min(88, leftPercent))}%` }}
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<div className="mb-2 text-xs font-semibold text-base-fg/80">Motion</div>
<div className="mb-3 grid grid-cols-4 gap-1">
{PRESETS.map((p) => {
const preset = EASING_PRESETS[p.name];
const active =
preset.p1x === easing.p1x &&
preset.p1y === easing.p1y &&
preset.p2x === easing.p2x &&
preset.p2y === easing.p2y;
return (
<button
key={p.name}
type="button"
title={p.label}
onClick={() => commit({ ...preset })}
className={`flex h-8 items-center justify-center rounded-md border transition-colors ${
active
? "border-brand-primary bg-brand-primary/20"
: "border-ui-controls-border/60 hover:bg-ui-controls/40"
}`}
>
<svg width="20" height="20" viewBox="0 0 20 20" fill="none">
<path
d={`M 2 18 C ${2 + preset.p1x * 16} ${18 - preset.p1y * 16} ${
2 + preset.p2x * 16
} ${18 - preset.p2y * 16} 18 2`}
stroke="currentColor"
strokeWidth="1.5"
className="text-base-fg/80"
/>
</svg>
</button>
);
})}
</div>
<svg
ref={svgRef}
viewBox={`0 0 ${BOX} ${BOX}`}
className="w-full touch-none rounded-lg bg-black/20"
onPointerMove={onPointerMove}
onPointerUp={() => setDragging(null)}
onPointerLeave={() => setDragging(null)}
>
{/* dotted linear reference */}
<line
x1={sx(0)}
y1={sy(0)}
x2={sx(1)}
y2={sy(1)}
stroke="currentColor"
strokeWidth="1"
strokeDasharray="3 3"
className="text-base-fg/25"
/>
{/* handle stems */}
<line
x1={sx(0)}
y1={sy(0)}
x2={sx(easing.p1x)}
y2={sy(easing.p1y)}
stroke="currentColor"
strokeWidth="1"
className="text-brand-primary/50"
/>
<line
x1={sx(1)}
y1={sy(1)}
x2={sx(easing.p2x)}
y2={sy(easing.p2y)}
stroke="currentColor"
strokeWidth="1"
className="text-brand-primary/50"
/>
{/* the curve */}
<path
d={curvePath}
stroke="currentColor"
strokeWidth="2"
fill="none"
className="text-base-fg"
/>
{/* endpoints */}
<circle cx={sx(0)} cy={sy(0)} r="2.5" className="fill-base-fg/50" />
<circle cx={sx(1)} cy={sy(1)} r="2.5" className="fill-base-fg/50" />
{/* draggable handles */}
<circle
cx={sx(easing.p1x)}
cy={sy(easing.p1y)}
r="5"
className="cursor-grab fill-brand-primary"
onPointerDown={(e) => {
e.currentTarget.setPointerCapture(e.pointerId);
setDragging("p1");
}}
/>
<circle
cx={sx(easing.p2x)}
cy={sy(easing.p2y)}
r="5"
className="cursor-grab fill-brand-primary"
onPointerDown={(e) => {
e.currentTarget.setPointerCapture(e.pointerId);
setDragging("p2");
}}
/>
</svg>
</div>
);
};
export default MotionPopover;
@@ -0,0 +1,76 @@
import { useContext } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faPlay, faPause, faChevronDown } from "@fortawesome/pro-solid-svg-icons";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import { pauseTimeline, playTimeline, seekTimeline } from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { formatTimecode } from "./timelineUtils";
import { DurationLabel } from "./DurationLabel";
// Collapsed timeline bar: play/pause · time · scrubber · duration · expand.
// Sits above the prompt card (rendered via the promptbox aboveStackSlot).
// `readOnly` (record mode) hides the expand chevron — playback only.
export const TimelineBar = ({ readOnly = false }: { readOnly?: boolean }) => {
const editor = useContext(EngineContext);
const playhead = usePageSceneStore((s) => s.timelinePlayhead);
const isPlaying = usePageSceneStore((s) => s.timelineIsPlaying);
const duration = usePageSceneStore((s) => s.timelineDuration);
const setExpanded = usePageSceneStore((s) => s.setTimelineExpanded);
const togglePlay = () => {
if (!editor) return;
if (isPlaying) pauseTimeline(editor);
else playTimeline(editor);
};
return (
<div
className="glass glass-no-hover flex w-full select-none items-center gap-3 rounded-2xl px-4 py-2 text-white shadow-xl"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
<button
type="button"
onClick={togglePlay}
className="flex h-7 w-7 items-center justify-center rounded-full text-base-fg/80 hover:bg-white/10"
>
<FontAwesomeIcon icon={isPlaying ? faPause : faPlay} className="h-3.5 w-3.5" />
</button>
<span className="w-9 shrink-0 tabular-nums text-xs text-base-fg/70">
{formatTimecode(playhead)}
</span>
<input
type="range"
min={0}
max={duration}
step={0.01}
value={playhead}
onChange={(e) => {
if (!editor) return;
pauseTimeline(editor);
seekTimeline(editor, parseFloat(e.target.value));
}}
className="h-1 flex-1 cursor-pointer accent-white"
/>
{readOnly ? (
<span className="w-9 shrink-0 text-right tabular-nums text-xs text-base-fg/70">
{formatTimecode(duration)}
</span>
) : (
<DurationLabel className="shrink-0 text-right text-xs text-base-fg/70" />
)}
{!readOnly && (
<button
type="button"
title="Expand timeline"
onClick={() => setExpanded(true)}
className="flex h-7 w-7 items-center justify-center rounded-full text-base-fg/60 hover:bg-white/10"
>
<FontAwesomeIcon icon={faChevronDown} className="h-3 w-3" />
</button>
)}
</div>
);
};
export default TimelineBar;
@@ -0,0 +1,167 @@
import { useContext, useRef } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faFilm,
faRepeat,
faXmark,
} from "@fortawesome/pro-solid-svg-icons";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import {
moveClipLane,
removeClipLane,
resizeClipLane,
setClipLoop,
} from "../../actions";
import {
usePageSceneStore,
type ClipLane,
} from "../../PageSceneStore";
import { DEFAULT_TIMELINE_FPS } from "../../engine/timeline/types";
import { fractionToTime, quantizeToFrame, timeToFraction } from "./timelineUtils";
interface TimelineClipRowProps {
lanes: ClipLane[];
duration: number;
}
// Drag intent for the current pointer gesture.
type Drag =
| { laneId: string; kind: "move"; grabOffset: number } // seconds pointer→start
| { laneId: string; kind: "resize" };
// A character's skeletal-animation clips on ONE shared row (single row per
// character; the overlap guard in TimelineController keeps them a
// non-overlapping sequence). Each strip drags to move, its right edge trims
// length, the loop chip toggles repeat, the × removes it. Dropping a clip from
// the Animations drawer is handled by the character block wrapper in
// TimelineEditor, so an empty row still accepts drops.
export const TimelineClipRow = ({ lanes, duration }: TimelineClipRowProps) => {
const editor = useContext(EngineContext);
const laneRef = useRef<HTMLDivElement>(null);
const drag = useRef<Drag | null>(null);
const fps =
editor?.timelineController.getTimeline()?.fps ?? DEFAULT_TIMELINE_FPS;
const laneTimeFromEvent = (clientX: number): number => {
const el = laneRef.current;
if (!el) return 0;
const rect = el.getBoundingClientRect();
return quantizeToFrame(
fractionToTime((clientX - rect.left) / rect.width, duration),
fps,
);
};
const onPointerMove = (e: React.PointerEvent) => {
if (!drag.current || !editor) return;
const t = laneTimeFromEvent(e.clientX);
if (drag.current.kind === "move") {
moveClipLane(editor, drag.current.laneId, t - drag.current.grabOffset);
} else {
const lane = lanes.find((l) => l.id === drag.current?.laneId);
if (lane) resizeClipLane(editor, lane.id, t - lane.strip.startTime);
}
};
const endDrag = () => {
drag.current = null;
};
return (
<div className="flex items-center gap-3 py-0.5">
<div className="flex w-32 shrink-0 items-center gap-2 truncate ps-6 text-xs text-base-fg/50">
<FontAwesomeIcon icon={faFilm} className="h-3 w-3 opacity-60" />
<span className="truncate">Animation</span>
</div>
<div
ref={laneRef}
className="relative h-6 flex-1 rounded-md bg-black/20"
onPointerMove={onPointerMove}
onPointerUp={endDrag}
onPointerLeave={endDrag}
onPointerCancel={endDrag}
>
{lanes.length === 0 && (
<div className="pointer-events-none absolute inset-0 flex items-center justify-center rounded-md border border-dashed border-white/15 text-[10px] text-base-fg/30">
Drag an animation here
</div>
)}
{lanes.map((lane) => {
const strip = lane.strip;
const leftPct = timeToFraction(strip.startTime, duration) * 100;
const widthPct = Math.max(
(strip.duration / Math.max(duration, 1e-6)) * 100,
1.5, // stay visible while a fresh clip's real length is loading
);
return (
<div
key={lane.id}
className="absolute inset-y-0 flex items-center gap-1 rounded-md border border-brand-primary/60 bg-brand-primary/25 pe-1 ps-1.5 text-[10px] text-white"
style={{ left: `${leftPct}%`, width: `${widthPct}%` }}
onPointerDown={(e) => {
e.currentTarget.setPointerCapture(e.pointerId);
drag.current = {
laneId: lane.id,
kind: "move",
grabOffset: laneTimeFromEvent(e.clientX) - strip.startTime,
};
}}
>
<button
type="button"
title={
strip.loop
? "Looping — click to play once"
: "Play once — click to loop"
}
className={`flex h-3.5 w-3.5 shrink-0 items-center justify-center rounded-[3px] ${
strip.loop
? "bg-white/90 text-brand-primary"
: "text-white/70 hover:text-white"
}`}
onPointerDown={(e) => e.stopPropagation()}
onClick={() =>
editor && setClipLoop(editor, lane.id, !strip.loop)
}
>
<FontAwesomeIcon icon={faRepeat} className="h-2.5 w-2.5" />
</button>
<span className="min-w-0 flex-1 truncate">{strip.name}</span>
<button
type="button"
title="Remove animation"
className="flex h-3.5 w-3.5 shrink-0 items-center justify-center rounded-[3px] text-white/70 hover:bg-white/20 hover:text-white"
onPointerDown={(e) => e.stopPropagation()}
onClick={() => editor && removeClipLane(editor, lane.id)}
>
<FontAwesomeIcon icon={faXmark} className="h-2.5 w-2.5" />
</button>
{/* right-edge trim handle */}
<div
className="absolute inset-y-0 right-0 w-1.5 cursor-ew-resize rounded-r-md bg-white/40 hover:bg-white/70"
title="Trim clip length"
onPointerDown={(e) => {
e.stopPropagation();
e.currentTarget.setPointerCapture(e.pointerId);
drag.current = { laneId: lane.id, kind: "resize" };
}}
/>
</div>
);
})}
</div>
{/* spacer to match the keyframe row's add-button column (keeps lane
widths aligned with the ruler) */}
<div className="w-7 shrink-0" />
</div>
);
};
export default TimelineClipRow;
@@ -0,0 +1,416 @@
import { useContext, useEffect, useRef } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faPlay,
faPause,
faBackwardStep,
faForwardStep,
faChevronUp,
faTrash,
} from "@fortawesome/pro-solid-svg-icons";
import { Button } from "@storyteller/ui-button";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import {
addClipToCharacter,
cancelTimeline,
deleteKeyframe,
pauseTimeline,
playTimeline,
saveTimeline,
seekTimeline,
} from "../../actions";
import { usePageSceneStore } from "../../PageSceneStore";
import { DEFAULT_TIMELINE_FPS } from "../../engine/timeline/types";
import {
ANIMATION_CLIP_MIME,
formatTimecode,
formatTimecodeFrames,
fractionToTime,
quantizeToFrame,
timeToFraction,
} from "./timelineUtils";
import { TimelineTrackRow } from "./TimelineTrackRow";
import { TimelineClipRow } from "./TimelineClipRow";
import { MotionPopover } from "./MotionPopover";
import { DurationLabel } from "./DurationLabel";
// Lane geometry: label column (w-32=8rem) + gap-3(0.75rem) … gap-3 + add-btn(w-7=1.75rem).
const LANE_LEFT = "8.75rem";
const LANE_RIGHT = "2.5rem";
// Expanded multi-track keyframe editor. Replaces the prompt box while open.
export const TimelineEditor = () => {
const editor = useContext(EngineContext);
const rulerRef = useRef<HTMLDivElement>(null);
const isScrubbing = useRef(false);
const tracks = usePageSceneStore((s) => s.timelineTracks);
const clipLanes = usePageSceneStore((s) => s.timelineClipLanes);
const characters = usePageSceneStore((s) => s.characters);
const duration = usePageSceneStore((s) => s.timelineDuration);
const playhead = usePageSceneStore((s) => s.timelinePlayhead);
const isPlaying = usePageSceneStore((s) => s.timelineIsPlaying);
const outlinerItems = usePageSceneStore((s) => s.outlinerItems);
const selectedKeyframeId = usePageSceneStore(
(s) => s.timelineSelectedKeyframeId,
);
const setExpanded = usePageSceneStore((s) => s.setTimelineExpanded);
// Every scene object gets a track row (empty lanes included), so the
// editor never looks blank while the scene has content — no selection
// required to see where keyframes can go.
const trackByUuid = new Map(tracks.map((t) => [t.objectUuid, t]));
const rows = outlinerItems;
// Only characters get skeletal-animation clip lanes. Clip drags are accepted
// on a character's block; other rows ignore them.
const characterIds = new Set(characters.map((c) => c.id));
const clipLanesByChar = new Map<string, typeof clipLanes>();
for (const lane of clipLanes) {
const list = clipLanesByChar.get(lane.objectUuid) ?? [];
list.push(lane);
clipLanesByChar.set(lane.objectUuid, list);
}
const selectedKeyframe = tracks
.flatMap((t) => t.keyframes)
.find((k) => k.id === selectedKeyframeId);
// The Motion popover edits the easing of the segment BETWEEN two
// keyframes (stored on the left one) and anchors at the segment midpoint.
const easingKeyframeId = usePageSceneStore((s) => s.timelineEasingKeyframeId);
let easingAnchor: {
keyframe: (typeof tracks)[number]["keyframes"][number];
leftPercent: number;
} | null = null;
if (easingKeyframeId) {
for (const track of tracks) {
const sorted = [...track.keyframes].sort((a, b) => a.time - b.time);
const index = sorted.findIndex((k) => k.id === easingKeyframeId);
if (index === -1) continue;
if (index < sorted.length - 1) {
const mid = (sorted[index].time + sorted[index + 1].time) / 2;
easingAnchor = {
keyframe: sorted[index],
leftPercent: timeToFraction(mid, duration) * 100,
};
}
break;
}
}
// Selected keyframe: Delete/Backspace removes it (capture phase, so the
// engine's document-level Delete — which deletes the selected scene
// object — never sees the key), and clicking anywhere that isn't a
// keyframe or the Delete button deselects it.
useEffect(() => {
if (!selectedKeyframeId) return undefined;
const onKeyDown = (e: KeyboardEvent) => {
if (e.key !== "Delete" && e.key !== "Backspace") return;
const target = e.target as HTMLElement | null;
if (
target instanceof HTMLInputElement ||
target instanceof HTMLTextAreaElement ||
target?.isContentEditable
) {
return;
}
e.preventDefault();
e.stopPropagation();
if (editor) deleteKeyframe(editor, selectedKeyframeId);
usePageSceneStore.getState().setTimelineSelectedKeyframe(null);
};
const onPointerDown = (e: PointerEvent) => {
const target = e.target as HTMLElement | null;
if (
target?.closest("[data-keyframe]") ||
target?.closest("[data-keyframe-delete]")
) {
return;
}
usePageSceneStore.getState().setTimelineSelectedKeyframe(null);
};
document.addEventListener("keydown", onKeyDown, true);
document.addEventListener("pointerdown", onPointerDown, true);
return () => {
document.removeEventListener("keydown", onKeyDown, true);
document.removeEventListener("pointerdown", onPointerDown, true);
};
}, [selectedKeyframeId, editor]);
// Clicking anywhere outside the popover (canvas included) dismisses it;
// the easing chips opt out so they can toggle it themselves.
useEffect(() => {
if (!easingKeyframeId) return undefined;
const onPointerDown = (e: PointerEvent) => {
const target = e.target as HTMLElement | null;
if (
target?.closest("#motion-popover") ||
target?.closest("[data-easing-chip]")
) {
return;
}
usePageSceneStore.getState().setTimelineEasingKeyframe(null);
};
document.addEventListener("pointerdown", onPointerDown, true);
return () =>
document.removeEventListener("pointerdown", onPointerDown, true);
}, [easingKeyframeId]);
const playheadFraction = timeToFraction(playhead, duration);
const togglePlay = () => {
if (!editor) return;
if (isPlaying) pauseTimeline(editor);
else playTimeline(editor);
};
// The timeline runs on a frame grid (what Record encodes at).
const fps = editor?.timelineController.getTimeline()?.fps ?? DEFAULT_TIMELINE_FPS;
// Scrubbing follows the pointer 1:1 (no magnetism toward keyframes),
// quantized to the frame grid so times are never arbitrary floats.
const seekFromRuler = (clientX: number) => {
const ruler = rulerRef.current;
if (!ruler || !editor) return;
const rect = ruler.getBoundingClientRect();
pauseTimeline(editor);
seekTimeline(
editor,
quantizeToFrame(
fractionToTime((clientX - rect.left) / rect.width, duration),
fps,
),
);
};
// Drop an animation clip (dragged from the Animations drawer) onto a
// character at the pointer's time. Lane geometry matches the ruler, so the
// ruler rect converts pointer-x → time exactly as scrubbing does.
const dropClipOnCharacter = (characterId: string, e: React.DragEvent) => {
const ruler = rulerRef.current;
if (!ruler || !editor) return;
const raw = e.dataTransfer.getData(ANIMATION_CLIP_MIME);
if (!raw) return;
e.preventDefault();
let payload: { media_id?: string; name?: string };
try {
payload = JSON.parse(raw);
} catch {
return;
}
if (!payload.media_id) return;
const rect = ruler.getBoundingClientRect();
const time = quantizeToFrame(
fractionToTime((e.clientX - rect.left) / rect.width, duration),
fps,
);
addClipToCharacter(
editor,
characterId,
{ media_id: payload.media_id, name: payload.name },
time,
);
};
return (
<div
id="timeline-editor"
className="glass glass-no-hover absolute bottom-4 left-1/2 w-[90vw] max-w-5xl -translate-x-1/2 select-none rounded-2xl p-3 text-white shadow-xl"
onMouseDown={(e) => e.stopPropagation()}
onClick={(e) => e.stopPropagation()}
>
{/* transport + collapse */}
<div className="mb-2 flex items-center gap-2">
<button
type="button"
title="Go to start"
onClick={() => editor && seekTimeline(editor, 0)}
className="flex h-7 w-7 items-center justify-center rounded-full text-base-fg/70 hover:bg-white/10"
>
<FontAwesomeIcon icon={faBackwardStep} className="h-3.5 w-3.5" />
</button>
<button
type="button"
onClick={togglePlay}
className="flex h-7 w-7 items-center justify-center rounded-full text-base-fg/90 hover:bg-white/10"
>
<FontAwesomeIcon
icon={isPlaying ? faPause : faPlay}
className="h-3.5 w-3.5"
/>
</button>
<button
type="button"
title="Go to end"
onClick={() => editor && seekTimeline(editor, duration)}
className="flex h-7 w-7 items-center justify-center rounded-full text-base-fg/70 hover:bg-white/10"
>
<FontAwesomeIcon icon={faForwardStep} className="h-3.5 w-3.5" />
</button>
<span className="ml-2 tabular-nums text-xs text-base-fg/60">
{formatTimecodeFrames(playhead, fps)} / {formatTimecode(duration)}
</span>
<button
type="button"
title="Collapse timeline"
onClick={() => setExpanded(false)}
className="ml-auto flex h-7 w-7 items-center justify-center rounded-full text-base-fg/60 hover:bg-white/10"
>
<FontAwesomeIcon icon={faChevronUp} className="h-3 w-3" />
</button>
</div>
{/* ruler + tracks (playhead line spans this region) */}
<div className="relative">
<div className="mb-1 flex items-center gap-3">
<div className="w-32 shrink-0" />
<div
ref={rulerRef}
className="relative h-5 flex-1 cursor-pointer touch-none text-[10px] text-base-fg/50"
onPointerDown={(e) => {
isScrubbing.current = true;
e.currentTarget.setPointerCapture(e.pointerId);
seekFromRuler(e.clientX);
}}
onPointerMove={(e) => {
if (isScrubbing.current) seekFromRuler(e.clientX);
}}
onPointerUp={() => {
isScrubbing.current = false;
}}
onPointerCancel={() => {
isScrubbing.current = false;
}}
>
<span className="absolute left-0">{formatTimecode(0)}</span>
<span className="absolute left-1/2 -translate-x-1/2">
{formatTimecode(duration / 2)}
</span>
<DurationLabel className="absolute right-0" />
</div>
<div className="w-7 shrink-0" />
</div>
{rows.length === 0 ? (
<div className="py-6 text-center text-xs text-base-fg/40">
Add objects to the scene to start animating.
</div>
) : (
/* Cap at ~3.5 rows (row = 36px lane + 8px padding) so the panel
stays compact; the half row hints that the list scrolls. The
scrollbar is hidden so the lanes keep the exact same width as
the ruler — a visible scrollbar would shift keyframes out of
alignment with the ruler-driven playhead. */
<div className="max-h-[154px] overflow-y-auto overscroll-contain [&::-webkit-scrollbar]:hidden [-ms-overflow-style:none] [scrollbar-width:none]">
{rows.map((item) => {
const isCharacter = characterIds.has(item.id);
const lanes = isCharacter ? clipLanesByChar.get(item.id) : undefined;
return (
<div
key={item.id}
onDragOver={
isCharacter
? (e) => {
// Allow the drop only when it carries an animation clip.
if (e.dataTransfer.types.includes(ANIMATION_CLIP_MIME)) {
e.preventDefault();
}
}
: undefined
}
onDrop={
isCharacter
? (e) => dropClipOnCharacter(item.id, e)
: undefined
}
>
<TimelineTrackRow
item={item}
track={trackByUuid.get(item.id)}
duration={duration}
/>
{isCharacter && (
<TimelineClipRow lanes={lanes ?? []} duration={duration} />
)}
</div>
);
})}
</div>
)}
{/* playhead (diamond handle in the ruler + line over the lanes) */}
<div
className="pointer-events-none absolute inset-y-0"
style={{ left: LANE_LEFT, right: LANE_RIGHT }}
>
<div
className="absolute inset-y-0 w-px bg-white"
style={{ left: `${playheadFraction * 100}%` }}
/>
<div
className="absolute top-0 h-2.5 w-2.5 -translate-x-1/2 rotate-45 rounded-[2px] bg-white"
style={{ left: `${playheadFraction * 100}%` }}
/>
</div>
{/* Anchor the popover in the lane area (same geometry as the
playhead overlay) so its left % lines up with the easing chip. */}
{easingAnchor && (
<div
className="pointer-events-none absolute inset-y-0"
style={{ left: LANE_LEFT, right: LANE_RIGHT }}
>
<MotionPopover
keyframe={easingAnchor.keyframe}
leftPercent={easingAnchor.leftPercent}
/>
</div>
)}
</div>
{/* footer */}
<div className="mt-3 flex items-center justify-between gap-2">
<span className="ms-2 text-[11px] text-base-fg/40">
Each diamond stores an object's full position, rotation and scale at
that moment. Tap a diamond to jump to it.
</span>
<div className="flex items-center gap-2">
{selectedKeyframe && (
/* data-keyframe-delete: exempt from the click-away deselect —
pointerdown would otherwise clear the selection before this
button's click fires. */
<div data-keyframe-delete>
<Button
variant="secondary"
icon={faTrash}
className="flex h-9 items-center border border-ui-controls-border bg-ui-controls/60 px-3 text-sm text-base-fg hover:bg-ui-controls/90"
onClick={() =>
editor && deleteKeyframe(editor, selectedKeyframe.id)
}
>
Delete
</Button>
</div>
)}
<Button
variant="secondary"
className="flex h-9 items-center border border-ui-controls-border bg-ui-controls/60 px-3 text-sm text-base-fg hover:bg-ui-controls/90"
onClick={() => editor && cancelTimeline(editor)}
>
Cancel
</Button>
<Button
variant="primary"
className="flex h-9 items-center border-none bg-brand-primary px-3 text-sm text-white"
onClick={() => editor && saveTimeline(editor)}
>
Save
</Button>
</div>
</div>
</div>
);
};
export default TimelineEditor;
@@ -0,0 +1,179 @@
import { useContext, useRef } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { EngineContext } from "../../contexts/EngineContext/EngineContext";
import { addKeyframe, moveKeyframe, seekTimeline } from "../../actions";
import {
usePageSceneStore,
type OutlinerItem,
type TimelineTrack,
} from "../../PageSceneStore";
import { DEFAULT_TIMELINE_FPS } from "../../engine/timeline/types";
import {
fractionToTime,
quantizeToFrame,
timeToFraction,
} from "./timelineUtils";
interface TimelineTrackRowProps {
item: OutlinerItem;
track: TimelineTrack | undefined;
duration: number;
}
export const TimelineTrackRow = ({
item,
track,
duration,
}: TimelineTrackRowProps) => {
const editor = useContext(EngineContext);
const laneRef = useRef<HTMLDivElement>(null);
const draggingId = useRef<string | null>(null);
const selectedKeyframeId = usePageSceneStore(
(s) => s.timelineSelectedKeyframeId,
);
const setSelectedKeyframe = usePageSceneStore(
(s) => s.setTimelineSelectedKeyframe,
);
const easingKeyframeId = usePageSceneStore((s) => s.timelineEasingKeyframeId);
const setEasingKeyframe = usePageSceneStore(
(s) => s.setTimelineEasingKeyframe,
);
const keyframes = track?.keyframes ?? [];
const sorted = [...keyframes].sort((a, b) => a.time - b.time);
const laneTimeFromEvent = (clientX: number): number => {
const lane = laneRef.current;
if (!lane) return 0;
const rect = lane.getBoundingClientRect();
// Keyframes live on the timeline's frame grid, same as scrubbing.
const fps = editor?.timelineController.getTimeline()?.fps ?? DEFAULT_TIMELINE_FPS;
return quantizeToFrame(
fractionToTime((clientX - rect.left) / rect.width, duration),
fps,
);
};
const selectKeyframe = (id: string, time: number) => {
setSelectedKeyframe(id);
if (editor) seekTimeline(editor, time);
};
return (
<div className="flex items-center gap-3 py-1">
{/* Keep this column exactly w-32: the ruler row and playhead overlay
assume it. Indent with padding, never margin (margin widens the
column and shifts the lanes out of alignment). */}
<div className="flex w-32 shrink-0 items-center gap-2 truncate ps-2 text-sm text-base-fg/90">
<FontAwesomeIcon icon={item.icon} className="h-3.5 w-3.5 opacity-70" />
<span className="truncate">{item.name}</span>
</div>
<div
ref={laneRef}
className="relative h-9 flex-1 rounded-lg bg-black/30"
onPointerMove={(e) => {
if (!draggingId.current || !editor) return;
moveKeyframe(
editor,
draggingId.current,
laneTimeFromEvent(e.clientX),
);
}}
onPointerUp={() => {
draggingId.current = null;
}}
onPointerLeave={() => {
draggingId.current = null;
}}
>
{/* connecting line between the first and last keyframe */}
{sorted.length >= 2 && (
<div
className="pointer-events-none absolute top-1/2 h-px -translate-y-1/2 bg-white/30"
style={{
left: `${timeToFraction(sorted[0].time, duration) * 100}%`,
right: `${(1 - timeToFraction(sorted[sorted.length - 1].time, duration)) * 100}%`,
}}
/>
)}
{/* easing chip between each pair of keyframes — toggles the Motion
popover for this segment (the curve INTO the next keyframe,
stored on the left keyframe) */}
{sorted.slice(0, -1).map((kf, i) => {
const next = sorted[i + 1];
const mid =
(timeToFraction(kf.time, duration) +
timeToFraction(next.time, duration)) /
2;
const active = easingKeyframeId === kf.id;
return (
<button
key={`ease-${kf.id}`}
type="button"
title="Edit easing"
data-easing-chip
className={`absolute top-1/2 flex h-5 w-6 -translate-x-1/2 -translate-y-1/2 items-center justify-center rounded-md border bg-black/60 ${
active
? "border-brand-primary text-white"
: "border-white/40 text-white/80 hover:border-white/80 hover:text-white"
}`}
style={{ left: `${mid * 100}%` }}
onClick={() => setEasingKeyframe(active ? null : kf.id)}
>
<svg width="12" height="10" viewBox="0 0 12 10" fill="none">
<path
d="M1 9 C 5 9, 7 1, 11 1"
stroke="currentColor"
strokeWidth="1.25"
/>
</svg>
</button>
);
})}
{sorted.map((kf) => {
const selected = kf.id === selectedKeyframeId;
return (
<div
key={kf.id}
role="button"
tabIndex={0}
data-keyframe
title={`${kf.time.toFixed(2)}s`}
className={`absolute top-1/2 h-3 w-3 -translate-x-1/2 -translate-y-1/2 rotate-45 cursor-grab rounded-[2px] border transition-colors ${
selected
? "border-brand-primary bg-brand-primary"
: "border-white/70 bg-white"
}`}
style={{ left: `${timeToFraction(kf.time, duration) * 100}%` }}
onPointerDown={(e) => {
e.currentTarget.setPointerCapture(e.pointerId);
draggingId.current = kf.id;
// Selecting a keyframe never opens the Motion popover —
// that's the easing chip's job.
setEasingKeyframe(null);
selectKeyframe(kf.id, kf.time);
}}
/>
);
})}
</div>
<button
type="button"
title="Add keyframe at playhead"
className="flex h-7 w-7 shrink-0 items-center justify-center rounded-full text-base-fg/70 hover:bg-white/10 hover:text-base-fg"
onClick={() => {
if (editor) addKeyframe(editor, item.id);
}}
>
<span className="block h-2 w-2 rotate-45 rounded-[1px] border-[1.5px] border-current" />
</button>
</div>
);
};
export default TimelineTrackRow;
@@ -0,0 +1,2 @@
export { TimelineBar } from "./TimelineBar";
export { TimelineEditor } from "./TimelineEditor";
@@ -0,0 +1,37 @@
// Shared helpers for the timeline UI.
// DataTransfer MIME for dragging an animation clip from the Animations drawer
// onto a character's lane in the timeline. Payload is JSON: { media_id, name }.
export const ANIMATION_CLIP_MIME = "application/x-artcraft-animation-clip";
// Seconds → "m:ss" (e.g. 10 → "0:10", 65 → "1:05").
export const formatTimecode = (seconds: number): string => {
const clamped = Math.max(0, seconds);
const mins = Math.floor(clamped / 60);
const secs = Math.floor(clamped % 60);
return `${mins}:${secs.toString().padStart(2, "0")}`;
};
// Seconds → "m:ss:ff" frame timecode (e.g. 3.6667s @ 30fps → "0:03:20").
// Used for the transport readout so the scrub position reads in frames.
export const formatTimecodeFrames = (seconds: number, fps: number): string => {
const totalFrames = Math.round(Math.max(0, seconds) * fps);
const totalSeconds = Math.floor(totalFrames / fps);
const frames = totalFrames % fps;
return `${formatTimecode(totalSeconds)}:${frames
.toString()
.padStart(2, "0")}`;
};
// Quantize a scrubbed/dragged time to the timeline's frame grid so the
// playhead and keyframes land on frame boundaries, not arbitrary floats.
export const quantizeToFrame = (seconds: number, fps: number): number =>
Math.round(seconds * fps) / fps;
// Position (0..1) of a time within [0, duration].
export const timeToFraction = (time: number, duration: number): number =>
duration <= 0 ? 0 : Math.max(0, Math.min(1, time / duration));
// Convert a pointer x within a track lane element to a time in [0, duration].
export const fractionToTime = (fraction: number, duration: number): number =>
Math.max(0, Math.min(1, fraction)) * duration;
@@ -34,6 +34,8 @@ import {
SelectedCameraChangedEvent,
SelectedModeChangedEvent,
SelectionChangedEvent,
TimelineChangedEvent,
TimelinePlayheadEvent,
TransformModeChangedEvent,
TransformSpaceChangedEvent,
} from "./events/EngineEvent";
@@ -137,6 +139,20 @@ export class EngineStoreBridge {
usePageSceneStore.getState().updateCamera(e.cameraId, e.updates),
),
// timeline
bus.subscribe(TimelineChangedEvent, (e) => {
const store = usePageSceneStore.getState();
store.setTimelineExists(e.exists);
store.setTimelineDuration(e.duration);
store.setTimelineTracks(e.tracks);
store.setTimelineClipLanes(e.clipLanes);
}),
bus.subscribe(TimelinePlayheadEvent, (e) => {
const store = usePageSceneStore.getState();
store.setTimelinePlayhead(e.playhead);
store.setTimelineIsPlaying(e.isPlaying);
}),
// lifecycle
bus.subscribe(EngineInitializedEvent, (e) =>
usePageSceneStore.getState().setIs3DEditorInitialized(e.initialized),
@@ -0,0 +1,237 @@
// Drives skeletal (Mixamo) animation clips placed on the timeline as clip
// lanes. Each character gets one THREE.AnimationMixer; each clip lane becomes
// an AnimationAction on that mixer. Playback is DETERMINISTIC and playhead-
// driven: we never let the mixer advance on its own — on every timeline
// evaluate() we set each action's absolute time from the playhead and call
// mixer.update(0). This makes scrubbing, play, and frame-accurate recording all
// pose the character identically.
//
// Clips are authored on their own (source) armature. We bind them onto the
// character by node name: Mixamo rigs share the "mixamorig:*" bone naming, so a
// clip's tracks resolve directly against the character's bones. If a character
// uses a differently-named rig, retargeting (SkeletonUtils.retargetClip) is the
// fallback — see retargetOntoCharacter() (not wired until the direct path is
// verified, to keep the first slice simple).
import * as THREE from "three";
import type Editor from "../editor";
import type { ClipLane } from "../timeline/types";
// Per-lane runtime state. `action` is null while the clip GLB is still loading.
interface LaneRuntime {
laneId: string;
characterUuid: string;
sourceMediaId: string;
startTime: number;
loop: boolean;
action: THREE.AnimationAction | null;
clipDuration: number;
}
export class CharacterAnimationManager {
// characterUuid → mixer (root = the character Object3D).
private mixers = new Map<string, THREE.AnimationMixer>();
// laneId → runtime.
private lanes = new Map<string, LaneRuntime>();
// laneId → monotonic token, so a superseded async load can't clobber state.
private loadTokens = new Map<string, number>();
// characterUuid → its skeleton(s), cached so bind-pose resets don't re-traverse.
private skeletons = new Map<string, THREE.Skeleton[]>();
constructor(private readonly editor: Editor) {}
// Reconcile the live runtime to `clipLanes`: dispose lanes that vanished,
// load lanes that appeared. Idempotent — called on every timeline change.
sync(clipLanes: ClipLane[]): void {
const wanted = new Set(clipLanes.map((l) => l.id));
for (const laneId of [...this.lanes.keys()]) {
if (!wanted.has(laneId)) this.disposeLane(laneId);
}
for (const lane of clipLanes) {
const rt = this.lanes.get(lane.id);
if (!rt) {
void this.addLane(lane);
} else {
// Placement can change without a reload.
rt.startTime = lane.strip.startTime;
rt.loop = lane.strip.loop;
}
}
this.pruneMixers();
}
// Pose every character at `playhead` (seconds). Deterministic: sets each
// action's absolute time then applies via mixer.update(0).
evaluateAt(playhead: number): void {
// Characters that have a clip covering this playhead. Everyone else is
// reset to their bind (T) pose below.
const posed = new Set<string>();
for (const rt of this.lanes.values()) {
const action = rt.action;
if (!action) continue;
let local = playhead - rt.startTime;
const dur = rt.clipDuration;
const active = local >= 0 && (rt.loop || local <= dur);
if (!active) {
action.enabled = false;
continue;
}
if (rt.loop && dur > 0) local = local % dur;
else local = Math.min(local, dur);
action.enabled = true;
action.paused = true; // we drive time ourselves; don't auto-advance
action.time = local;
action.setEffectiveWeight(1);
posed.add(rt.characterUuid);
}
// Gaps between strips (and before the first / after the last) have no active
// clip. A disabled action leaves the skeleton frozen on its last evaluated
// frame, so explicitly restore the bind pose for those characters — the
// "default state" when nothing is under the scrubber.
for (const uuid of this.mixers.keys()) {
if (!posed.has(uuid)) this.resetToBindPose(uuid);
}
for (const mixer of this.mixers.values()) mixer.update(0);
}
clear(): void {
for (const laneId of [...this.lanes.keys()]) this.disposeLane(laneId);
for (const mixer of this.mixers.values()) mixer.stopAllAction();
this.mixers.clear();
this.skeletons.clear();
}
// ─── internals ────────────────────────────────────────────────────────
private async addLane(lane: ClipLane): Promise<void> {
const token = (this.loadTokens.get(lane.id) ?? 0) + 1;
this.loadTokens.set(lane.id, token);
const rt: LaneRuntime = {
laneId: lane.id,
characterUuid: lane.objectUuid,
sourceMediaId: lane.strip.sourceMediaId,
startTime: lane.strip.startTime,
loop: lane.strip.loop,
action: null,
clipDuration: lane.strip.duration,
};
this.lanes.set(lane.id, rt);
const character = this.findObject(lane.objectUuid);
if (!character) return;
const glb = await this.editor.activeScene.loadRawGlb(lane.strip.sourceMediaId);
// Bail if this lane was removed/replaced while the clip was loading.
if (this.loadTokens.get(lane.id) !== token || !this.lanes.has(lane.id)) {
return;
}
const clip = glb?.animations?.[0];
if (!clip) {
console.warn(
"Animation clip has no animations track:",
lane.strip.sourceMediaId,
);
return;
}
const mixer = this.getMixer(lane.objectUuid, character);
const action = mixer.clipAction(clip);
action.play();
action.paused = true;
action.enabled = false;
action.loop = rt.loop ? THREE.LoopRepeat : THREE.LoopOnce;
action.clampWhenFinished = true;
rt.action = action;
rt.clipDuration = clip.duration;
// Diagnostic for the direct-bind vs retarget question: if none of the
// clip's tracks resolve to a node on the character, the character won't
// move — that's the signal to wire SkeletonUtils.retargetClip.
if (!this.clipBindsToCharacter(clip, character)) {
console.warn(
`Animation clip "${clip.name}" bound 0 tracks on character ${lane.objectUuid} — likely a rig/bone-name mismatch (retarget needed).`,
);
}
// Adopt the clip's real length into the timeline data (fresh drops only;
// user trims are preserved), so the strip width and move/trim clamps are
// correct and survive save/reload.
this.editor.timelineController.resolveClipDuration(lane.id, clip.duration);
// Pose immediately at the current playhead so the drop is visible even
// when the timeline isn't playing.
this.evaluateAt(this.editor.timelineController.getPlayhead());
}
private disposeLane(laneId: string): void {
const rt = this.lanes.get(laneId);
if (!rt) return;
this.loadTokens.set(laneId, (this.loadTokens.get(laneId) ?? 0) + 1);
if (rt.action) {
const mixer = this.mixers.get(rt.characterUuid);
rt.action.stop();
mixer?.uncacheAction(rt.action.getClip());
mixer?.uncacheClip(rt.action.getClip());
}
this.lanes.delete(laneId);
}
// Drop mixers whose character no longer has any lanes.
private pruneMixers(): void {
const live = new Set([...this.lanes.values()].map((l) => l.characterUuid));
for (const uuid of [...this.mixers.keys()]) {
if (!live.has(uuid)) {
this.mixers.get(uuid)?.stopAllAction();
this.mixers.delete(uuid);
this.skeletons.delete(uuid);
}
}
}
private getMixer(uuid: string, root: THREE.Object3D): THREE.AnimationMixer {
let mixer = this.mixers.get(uuid);
if (!mixer) {
mixer = new THREE.AnimationMixer(root);
this.mixers.set(uuid, mixer);
this.skeletons.set(uuid, this.collectSkeletons(root));
}
return mixer;
}
// Reset a character's skeleton(s) to their bind (rest) pose. For Mixamo rigs
// the bind pose is the T-pose — the "default state" shown wherever no clip
// covers the playhead.
private resetToBindPose(characterUuid: string): void {
const skeletons = this.skeletons.get(characterUuid);
if (!skeletons) return;
for (const skeleton of skeletons) skeleton.pose();
}
private collectSkeletons(root: THREE.Object3D): THREE.Skeleton[] {
const skeletons: THREE.Skeleton[] = [];
root.traverse((o) => {
const sm = o as THREE.SkinnedMesh;
if (sm.isSkinnedMesh && sm.skeleton) skeletons.push(sm.skeleton);
});
return skeletons;
}
// True if at least one of the clip's tracks names a node that exists under
// `character`. Mixamo clips address bones as "mixamorig:Hips.position"; we
// strip the ".property" suffix and look the node up by name.
private clipBindsToCharacter(
clip: THREE.AnimationClip,
character: THREE.Object3D,
): boolean {
return clip.tracks.some((track) => {
const nodeName = track.name.split(".")[0];
if (!nodeName) return false;
return character.getObjectByName(nodeName) !== undefined;
});
}
private findObject(uuid: string): THREE.Object3D | undefined {
return this.editor.activeScene.scene.getObjectByProperty("uuid", uuid);
}
}
@@ -2,7 +2,11 @@ import * as THREE from "three";
import { OrbitControls } from "three/addons/controls/OrbitControls.js";
import Scene from "./scene";
import { PointerLockControls } from "three/addons/controls/PointerLockControls.js";
import { CameraAspectRatio, ClipGroup } from "../enums";
import {
CameraAspectRatio,
ClipGroup,
DEFAULT_CAMERA_ASPECT_RATIO,
} from "../enums";
import { SceneUtils } from "./helper";
import { MouseControls } from "./keybinds_controls";
import { SaveManager } from "./save_manager";
@@ -16,6 +20,8 @@ import { GizmoController } from "./editor/GizmoController";
import { SelectionBridge } from "./editor/SelectionBridge";
import { CameraController } from "./editor/CameraController";
import { HistoryManager } from "./editor/HistoryManager";
import { TimelineController } from "./editor/TimelineController";
import { CharacterAnimationManager } from "./animation/CharacterAnimationManager";
import { DeleteAction } from "./editor/actions/DeleteAction";
import { TransformAction } from "./editor/actions/TransformAction";
@@ -26,9 +32,11 @@ import { ensureAmmoLoaded } from "./ammoLoader";
import { EngineEventBus } from "./events/EngineEventBus";
import { EngineStoreBridge } from "./EngineStoreBridge";
import {
CameraAspectRatioChangedEvent,
EditorStateChangedEvent,
EngineInitializedEvent,
GridVisibleChangedEvent,
CameraViewToggleRequestedEvent,
InspectorPanelChangedEvent,
SceneLoadedEvent,
SceneResetEvent,
@@ -77,6 +85,11 @@ class Editor {
selection: SelectionBridge;
// Owns the camera state, FreeCam plumbing, and the per-frame camera tick.
cameraController: CameraController;
// Owns the animation timeline + keyframe playback (evaluated per frame).
timelineController: TimelineController;
// Drives skeletal (Mixamo) clip lanes onto characters, deterministically
// from the timeline playhead. See CharacterAnimationManager.
characterAnimationManager: CharacterAnimationManager;
// Owns the undo/redo stack. Mutation sites push UndoableAction
// instances via editor.history.record(...); each action class under
// engine/editor/actions/ encapsulates its own apply/revert.
@@ -96,6 +109,7 @@ class Editor {
// Store→engine reactor for grid visibility (toggling the gridHelper
// mesh in/out of the THREE.js scene). Cleared on unmountEngine.
private gridSubscription: () => void;
private cameraViewSubscription: () => void;
// Holds the in-flight transform action between gizmo dragstart and
// dragend. Null whenever no drag is in progress.
@@ -175,6 +189,13 @@ class Editor {
this.activeScene?.applyGridVisibility(e.visible),
);
// Look-through toggle. The double-click intercept and the outliner/exit
// buttons emit this; the controller owns the enter/exit transition.
this.cameraViewSubscription = this.bus.subscribe(
CameraViewToggleRequestedEvent,
() => this.cameraController.switchCameraView(),
);
// PostProcessingPipeline must exist before Scene because Scene's
// load paths invoke updateSurfaceIdAttributeToMesh as a callback.
this.postProcessing = new PostProcessingPipeline();
@@ -287,6 +308,18 @@ class Editor {
loadSceneState: (token) => this.adapter.loadScene(token),
getCameras: () => usePageSceneStore.getState().cameras,
getSelectedCameraId: () => usePageSceneStore.getState().selectedCameraId,
getTimeline: () => this.timelineController.getTimeline(),
loadTimeline: (timeline) =>
this.timelineController.loadTimeline(timeline),
getRenderCameraTransform: () => {
const cam = this.cameraController.cam_obj;
if (!cam) return null;
return {
position: { x: cam.position.x, y: cam.position.y, z: cam.position.z },
rotation: { x: cam.rotation.x, y: cam.rotation.y, z: cam.rotation.z },
};
},
recreateCameraObject: () => this.activeScene._create_camera_obj(),
bus: this.bus,
});
this.viewport = new ViewportController({
@@ -309,6 +342,14 @@ class Editor {
// apply/revert + dependencies. HistoryManager just stores them.
this.history = new HistoryManager({ capacity: 64 });
// Animation timeline. Holds keyframes + drives playback; ticked each
// frame in renderSingleFrame() after the entrance animator. An empty
// timeline always exists so the collapsed timeline bar (the sole
// build-mode bottom UI) is functional; a loaded scene overrides it.
this.characterAnimationManager = new CharacterAnimationManager(this);
this.timelineController = new TimelineController(this);
this.timelineController.create();
this.positive_prompt =
"((masterpiece, best quality, 8K, detailed)), colorful, epic, fantasy, (fox, red fox:1.2), no humans, 1other, ((koi pond)), outdoors, pond, rocks, stones, koi fish, ((watercolor))), lilypad, fish swimming around.";
@@ -490,6 +531,11 @@ class Editor {
} else if (this.activeTransform) {
if (this.activeTransform.commit()) {
this.history.record(this.activeTransform);
// Auto-key: if the dragged object is already keyframed, record
// its new transform at the playhead (replaces the keyframe there
// or creates one at the current scrub point).
const uuid = this.sceneManager?.selected_objects?.[0]?.uuid;
if (uuid) this.timelineController.autoKeyIfTracked(uuid);
}
this.activeTransform = null;
}
@@ -539,6 +585,7 @@ class Editor {
isHotkeyDisabled: () =>
usePageSceneStore.getState().hotkeyStatus.disabled,
getTransformSpace: () => usePageSceneStore.getState().transformSpace,
isViewportLocked: () => this.cameraController.locked,
});
this.sceneManager = new SceneManager(
@@ -601,7 +648,9 @@ class Editor {
}
public isMovable(): boolean {
return this.focused;
// A locked viewport (record mode) blocks mouse-look entirely —
// playback is read-only, so the camera must not turn.
return this.focused && !this.cameraController.locked;
}
// Toggle the three.js Stats panel (FPS / ms / mb). Bound to the
@@ -687,6 +736,14 @@ class Editor {
const originalRenderCameraAspect =
renderCamera?.aspect || originalCameraAspect;
// The camera-view framing offset maps the render frame into an inset
// rect of the viewport canvas — it must not leak into the snapshot,
// which renders the frame full-bleed at the target resolution.
// CameraController.applyFrameProjection reapplies it afterwards.
if (camera.view?.enabled) {
camera.clearViewOffset();
}
// Temporarily set renderer to high resolution
this.renderer.setSize(targetWidth, targetHeight, false);
this.renderer.setPixelRatio(1);
@@ -727,6 +784,10 @@ class Editor {
this.renderer.setSize(originalWidth, originalHeight, false);
this.renderer.setPixelRatio(originalPixelRatio);
// Reapply the camera-view framing offset (if active) before the
// restore render so the viewport doesn't flash an unframed frame.
this.cameraController.applyFrameProjection();
// Re-render at original resolution
if (this.postProcessing.composer) {
this.postProcessing.composer.setSize(originalWidth, originalHeight);
@@ -768,6 +829,14 @@ class Editor {
this.cameraController.cam_obj = this.activeScene.get_object_by_name(
this.cameraController.camera_name,
);
// Fresh scenes always start at the default aspect ratio — don't
// inherit whatever the previously loaded scene used.
this.cameraController.changeRenderCameraAspectRatio(
DEFAULT_CAMERA_ASPECT_RATIO,
);
this.bus.emit(
new CameraAspectRatioChangedEvent(DEFAULT_CAMERA_ASPECT_RATIO),
);
const sceneTitle =
sceneTitleInput && sceneTitleInput !== ""
? sceneTitleInput
@@ -824,6 +893,17 @@ class Editor {
});
}
// True if any object in the scene carries a THREE animation clip (e.g. an
// animated GLB import). Used to auto-expand the timeline on load.
sceneHasAnimation(): boolean {
let found = false;
this.activeScene.scene.traverse((o) => {
const clips = (o as unknown as { animations?: unknown[] }).animations;
if (clips && clips.length > 0) found = true;
});
return found;
}
deleteObject(uuid: string) {
const obj = this.activeScene.scene.getObjectByProperty("uuid", uuid);
if (obj) this.history.record(new DeleteAction(this, obj));
@@ -878,6 +958,10 @@ class Editor {
// Advance any in-flight entrance animations.
this.entranceAnimator.tick(delta_time);
// Evaluate the animation timeline (writes transforms only while
// playing or right after a seek).
this.timelineController.tick(delta_time);
this.activeScene.shader_objects.forEach((shader) => {
shader.material.uniforms["time"].value += 0.5 * delta_time;
});
@@ -969,6 +1053,7 @@ class Editor {
this.isMounted = false;
this.bus.emit(new EngineInitializedEvent(false));
this.gridSubscription();
this.cameraViewSubscription();
this.storeBridge.dispose();
console.log("3D Editor Engine unmounted");
}
@@ -5,7 +5,11 @@ import {
lookAtFromCamera,
type FreeCamControlState,
} from "../cameraMath";
import { CameraAspectRatio, EditorStates } from "../../enums";
import {
CameraAspectRatio,
DEFAULT_CAMERA_ASPECT_RATIO,
EditorStates,
} from "../../enums";
import type { Camera } from "@storyteller/common";
import type { EngineEventBus } from "../events/EngineEventBus";
import { CameraUpdatedEvent } from "../events/EngineEvent";
@@ -16,6 +20,20 @@ export type RenderDimensions = {
aspectRatio: number;
};
// Where the camera-view framing overlay wants the render framing to land,
// in canvas CSS pixels. While a rect is set (and we're in camera view) the
// viewport camera's projection is offset so the exact render framing maps
// into this rect instead of filling the whole canvas — the area outside the
// rect shows the surrounding scene (dimmed by the overlay).
export type CameraFrameRect = {
x: number;
y: number;
width: number;
height: number;
canvasWidth: number;
canvasHeight: number;
};
// Capabilities CameraController needs from outside its own state.
// Editor wires these in initialize() so CameraController never imports
// sibling subsystems directly.
@@ -46,7 +64,7 @@ export class CameraController {
camera: THREE.PerspectiveCamera | null = null;
render_camera: THREE.PerspectiveCamera | null = null;
render_camera_aspect_ratio: CameraAspectRatio = CameraAspectRatio.HORIZONTAL_3_2;
render_camera_aspect_ratio: CameraAspectRatio = DEFAULT_CAMERA_ASPECT_RATIO;
cam_obj: THREE.Object3D | undefined;
camera_person_mode: boolean = false;
@@ -54,6 +72,15 @@ export class CameraController {
lockControls: PointerLockControls | undefined;
freeCamState: FreeCamControlState | null = null;
// When true, FreeCam input is ignored (record mode: read-only viewport).
locked = false;
// Camera-view framing rect set by the CameraFrame overlay; null = none.
private frameRect: CameraFrameRect | null = null;
// Canvas aspect captured when the frame projection is applied, so the
// viewport camera's aspect can be restored the moment the frame clears
// (the resize cascade only corrects it on the next actual resize).
private frameCanvasAspect: number | null = null;
last_cam_pos: THREE.Vector3 = new THREE.Vector3(0, 0, 0);
last_cam_rot: THREE.Euler = new THREE.Euler(0, 0, 0);
@@ -83,69 +110,132 @@ export class CameraController {
}
// Toggles between free-camera (edit) mode and the active render
// camera's perspective. Lifted from SceneUtils.switchCameraView so
// the camera state stays in one place.
// camera's perspective. Split into idempotent enter/exit so callers
// (record mode) can force a specific state without double-toggling.
switchCameraView() {
this.camera_person_mode = !this.camera_person_mode;
if (this.freeCamState) {
this.freeCamState.velocity.set(0, 0, 0);
if (this.camera_person_mode) this.exitCameraView();
else this.enterCameraView();
}
// Look through the render camera. No-op if already in camera view.
enterCameraView() {
// Require a live viewport camera: we MUST capture its pose before
// flipping state, or exit would restore the default (0,0,0) pose and
// leave the viewport pointed at empty space (looks like "everything
// disappeared").
if (this.camera_person_mode || !this.cam_obj || !this.camera) return;
this.last_cam_pos.copy(this.camera.position);
this.last_cam_rot.copy(this.camera.rotation);
this.camera_person_mode = true;
if (this.freeCamState) this.freeCamState.velocity.set(0, 0, 0);
this.camera.position.copy(this.cam_obj.position);
this.camera.rotation.copy(this.cam_obj.rotation);
if (this.lockControls) {
this.deps.getThreeScene().add(this.lockControls.getObject());
}
if (!this.cam_obj) return;
this.cam_obj.scale.set(0, 0, 0);
if (this.camera_person_mode && this.camera) {
this.last_cam_pos.copy(this.camera.position);
this.last_cam_rot.copy(this.camera.rotation);
this.deps.removeTransformControls();
this.deps.setSelected(this.cam_obj);
this.deps.setEditorState(EditorStates.CAMERA_VIEW);
this.camera.position.copy(this.cam_obj.position);
this.camera.rotation.copy(this.cam_obj.rotation);
// Workaround: in camera mode, a right-click should pan rather than
// open the browser context menu. Defer until the letterbox settles.
setTimeout(
() =>
document
.getElementById("letterbox")
?.addEventListener("contextmenu", function (event) {
event.preventDefault();
}),
250,
);
}
if (this.lockControls) {
this.deps.getThreeScene().add(this.lockControls.getObject());
}
// useFreeCam reads editorState from the store and enables
// itself when CAMERA_VIEW; nothing to do here.
this.cam_obj.scale.set(0, 0, 0);
// Return to the free editing camera. No-op if not in camera view.
exitCameraView() {
if (!this.camera_person_mode || !this.cam_obj) return;
this.camera_person_mode = false;
if (this.freeCamState) this.freeCamState.velocity.set(0, 0, 0);
this.deps.removeTransformControls();
this.deps.setSelected(this.cam_obj);
this.deps.setEditorState(EditorStates.CAMERA_VIEW);
const hot = this.deps.getHotItems();
if (hot) {
hot.forEach((element) => {
element.visible = false;
});
}
// Workaround: in camera mode, a right-click should pan rather
// than open the browser context menu. Defer the listener attach
// until after the letterbox swap settles.
setTimeout(
() =>
document
.getElementById("letterbox")
?.addEventListener("contextmenu", function (event) {
event.preventDefault();
}),
250,
);
} else if (this.camera) {
if (this.camera) {
this.camera.position.copy(this.last_cam_pos);
this.camera.rotation.copy(this.last_cam_rot);
if (this.lockControls) {
this.deps.getThreeScene().remove(this.lockControls.getObject());
}
this.cam_obj.scale.set(1, 1, 1);
this.camera.updateProjectionMatrix();
}
// Drop the camera-view framing offset now that we're free-flying again.
this.applyFrameProjection();
if (this.lockControls) {
this.deps.getThreeScene().remove(this.lockControls.getObject());
}
this.cam_obj.scale.set(1, 1, 1);
const hot = this.deps.getHotItems();
if (hot) {
hot.forEach((element) => {
element.visible = true;
});
}
this.deps.hideObjectPanel();
this.deps.setEditorState(EditorStates.EDIT);
}
this.deps.hideObjectPanel();
this.deps.setEditorState(EditorStates.EDIT);
// Record mode locks the viewport: input is ignored so playback is
// strictly read-only. Gated in tickPerFrame's FreeCam integration.
setLocked(locked: boolean): void {
this.locked = locked;
}
// Set (or clear, with null) the camera-view framing rect. Applied
// immediately and kept in sync per-frame by tickPerFrame, so resizes and
// snapshot-time projection resets self-heal on the next tick.
setCameraFrameRect(rect: CameraFrameRect | null): void {
this.frameRect = rect;
this.applyFrameProjection();
}
// Reconcile the viewport camera's projection with the framing rect.
// While in camera view with a rect set: aspect = the render camera's
// aspect, and a view offset maps the full render framing into the rect
// (the canvas renders a wider surround around it). Otherwise: clear any
// offset and restore the full-canvas aspect. Cheap when nothing changed.
applyFrameProjection(): void {
const cam = this.camera;
if (!cam) return;
const rect = this.camera_person_mode ? this.frameRect : null;
if (rect) {
const renderAspect = this.getRenderDimensions().aspectRatio;
const view = cam.view;
const unchanged =
cam.aspect === renderAspect &&
view?.enabled === true &&
view.fullWidth === rect.width &&
view.fullHeight === rect.height &&
view.offsetX === -rect.x &&
view.offsetY === -rect.y &&
view.width === rect.canvasWidth &&
view.height === rect.canvasHeight;
if (unchanged) return;
this.frameCanvasAspect = rect.canvasWidth / rect.canvasHeight;
cam.aspect = renderAspect;
// Virtual "full view" = the framing rect; the canvas renders the
// larger region around it, so the rect shows the exact render frame.
cam.setViewOffset(
rect.width,
rect.height,
-rect.x,
-rect.y,
rect.canvasWidth,
rect.canvasHeight,
);
cam.updateProjectionMatrix();
} else if (cam.view?.enabled) {
cam.clearViewOffset();
if (this.frameCanvasAspect !== null) {
cam.aspect = this.frameCanvasAspect;
this.frameCanvasAspect = null;
}
cam.updateProjectionMatrix();
}
}
@@ -208,7 +298,11 @@ export class CameraController {
}
}
if (this.freeCamState && this.camera) {
// Keep the camera-view framing projection in sync (no-op when nothing
// changed; also heals resize/snapshot-time aspect resets).
this.applyFrameProjection();
if (this.freeCamState && this.camera && !this.locked) {
const moved = freeCamFrameTick(
this.camera,
this.freeCamState,
@@ -0,0 +1,498 @@
// TimelineController owns the animation timeline + playback for the 3D
// editor. It is the authoritative holder of the keyframe data; the store
// only mirrors it (via events → EngineStoreBridge) for the React UI.
//
// Playback contract: transforms are written to THREE objects ONLY while
// playing or on an explicit seek. When idle, the controller does not touch
// object transforms, so manual editing in build mode is unaffected.
import * as THREE from "three";
import type Editor from "../editor";
import { snapshotTransform, writeTransform } from "./actions/snapshots";
import { SaveTimelineAction } from "./actions/SaveTimelineAction";
import { sampleTrackAt } from "../timeline/interpolation";
import {
cloneTimeline,
DEFAULT_EASING,
DEFAULT_TIMELINE_DURATION,
DEFAULT_TIMELINE_FPS,
type ClipLane,
type ClipStrip,
type EasingSpec,
type Keyframe,
type TimelineData,
type TimelineTrack,
} from "../timeline/types";
import {
TimelineChangedEvent,
TimelinePlayheadEvent,
} from "../events/EngineEvent";
export class TimelineController {
private timeline: TimelineData | null = null;
private saved: TimelineData | null = null;
private playhead = 0;
private isPlaying = false;
constructor(private readonly editor: Editor) {}
// ─── queries ──────────────────────────────────────────────────────────
exists(): boolean {
return this.timeline !== null;
}
getTimeline(): TimelineData | null {
return this.timeline;
}
getPlayhead(): number {
return this.playhead;
}
// ─── lifecycle ────────────────────────────────────────────────────────
create(): void {
if (this.timeline) return;
this.timeline = {
duration: DEFAULT_TIMELINE_DURATION,
fps: DEFAULT_TIMELINE_FPS,
tracks: [],
clipLanes: [],
};
this.saved = cloneTimeline(this.timeline);
this.playhead = 0;
this.isPlaying = false;
this.emitChanged();
this.emitPlayhead();
}
clear(): void {
this.timeline = null;
this.saved = null;
this.playhead = 0;
this.isPlaying = false;
this.editor.characterAnimationManager.clear();
this.emitChanged();
this.emitPlayhead();
}
// Replace the live timeline wholesale (used by undo/redo of a save).
loadTimeline(data: TimelineData): void {
this.timeline = cloneTimeline(data);
// Backfill defensively so legacy/partial saved timelines don't break the
// frame-quantized UI (which assumes a valid fps) or produce a 0 duration.
if (!this.timeline.fps || this.timeline.fps <= 0) {
this.timeline.fps = DEFAULT_TIMELINE_FPS;
}
if (!this.timeline.duration || this.timeline.duration <= 0) {
this.timeline.duration = DEFAULT_TIMELINE_DURATION;
}
if (!Array.isArray(this.timeline.tracks)) this.timeline.tracks = [];
if (!Array.isArray(this.timeline.clipLanes)) this.timeline.clipLanes = [];
this.saved = cloneTimeline(this.timeline);
this.playhead = Math.min(this.playhead, this.timeline.duration);
this.syncClipLanes();
this.evaluate();
this.emitChanged();
this.emitPlayhead();
}
// ─── transport ────────────────────────────────────────────────────────
play(): void {
if (!this.timeline) return;
if (this.playhead >= this.timeline.duration) this.playhead = 0;
this.isPlaying = true;
this.emitPlayhead();
}
pause(): void {
this.isPlaying = false;
this.emitPlayhead();
}
seekTo(time: number): void {
if (!this.timeline) return;
this.playhead = Math.max(0, Math.min(time, this.timeline.duration));
this.evaluate();
this.emitPlayhead();
}
// Set the max timeline duration (seconds). Clamped to [1, 60] — covers the
// required 5–30s range with headroom. Keeps the playhead in range.
setDuration(seconds: number): void {
if (!this.timeline || !Number.isFinite(seconds)) return;
this.timeline.duration = Math.max(1, Math.min(60, seconds));
if (this.playhead > this.timeline.duration) {
this.playhead = this.timeline.duration;
}
this.emitChanged();
this.emitPlayhead();
}
// Advanced once per frame from editor.renderSingleFrame().
tick(delta: number): void {
if (!this.timeline || !this.isPlaying) return;
this.playhead += delta;
if (this.playhead >= this.timeline.duration) {
this.playhead = this.timeline.duration;
this.isPlaying = false;
}
this.evaluate();
this.emitPlayhead();
}
// ─── keyframe editing ──────────────────────────────────────────────────
// Auto-key: when an ALREADY-keyframed object is mutated, capture its current
// transform at the playhead — replacing the keyframe there or creating one at
// the current scrub point. No-op for objects that have never been keyframed
// (the first keyframe is always added explicitly).
autoKeyIfTracked(objectUuid: string): void {
if (!this.timeline) return;
const track = this.timeline.tracks.find(
(t) => t.objectUuid === objectUuid,
);
if (!track || track.keyframes.length === 0) return;
this.addKeyframe(objectUuid, this.playhead);
}
addKeyframe(objectUuid: string, atTime?: number): void {
if (!this.timeline) return;
const obj = this.editor.activeScene.scene.getObjectByProperty(
"uuid",
objectUuid,
);
if (!obj) return;
const time = Math.max(
0,
Math.min(atTime ?? this.playhead, this.timeline.duration),
);
const keyframe: Keyframe = {
id: THREE.MathUtils.generateUUID(),
time,
transform: snapshotTransform(obj),
easing: { ...DEFAULT_EASING },
};
let track = this.timeline.tracks.find((t) => t.objectUuid === objectUuid);
if (!track) {
track = { objectUuid, keyframes: [] };
this.timeline.tracks.push(track);
}
// Replace an existing keyframe at (nearly) the same time, else insert.
const existingIdx = track.keyframes.findIndex(
(k) => Math.abs(k.time - time) < 1e-3,
);
if (existingIdx >= 0) track.keyframes[existingIdx] = keyframe;
else track.keyframes.push(keyframe);
this.sortTrack(track);
this.emitChanged();
}
deleteKeyframe(keyframeId: string): void {
if (!this.timeline) return;
for (const track of this.timeline.tracks) {
const idx = track.keyframes.findIndex((k) => k.id === keyframeId);
if (idx >= 0) {
track.keyframes.splice(idx, 1);
break;
}
}
// Drop now-empty tracks so the row disappears.
this.timeline.tracks = this.timeline.tracks.filter(
(t) => t.keyframes.length > 0,
);
this.emitChanged();
}
moveKeyframe(keyframeId: string, time: number): void {
if (!this.timeline) return;
const clamped = Math.max(0, Math.min(time, this.timeline.duration));
for (const track of this.timeline.tracks) {
const kf = track.keyframes.find((k) => k.id === keyframeId);
if (kf) {
kf.time = clamped;
this.sortTrack(track);
break;
}
}
this.evaluate();
this.emitChanged();
}
setEasing(keyframeId: string, easing: EasingSpec): void {
if (!this.timeline) return;
for (const track of this.timeline.tracks) {
const kf = track.keyframes.find((k) => k.id === keyframeId);
if (kf) {
kf.easing = { ...easing };
break;
}
}
this.evaluate();
this.emitChanged();
}
// ─── clip lanes (skeletal animation) ─────────────────────────────────────
// Add a new stacked clip lane for `objectUuid` (a character). The clip is
// dropped at `startTime`, clamped so it starts within the timeline. Returns
// the new lane id, or null if there is no timeline.
addClipLane(objectUuid: string, strip: Omit<ClipStrip, "id">): string | null {
if (!this.timeline) return null;
// Characters hold clips on a single row: snap the drop to the nearest free
// slot so strips never overlap (see resolveFreeStart).
const startTime = this.resolveFreeStart(
objectUuid,
null,
strip.startTime,
strip.duration,
);
const lane: ClipLane = {
id: THREE.MathUtils.generateUUID(),
objectUuid,
strip: { ...strip, id: THREE.MathUtils.generateUUID(), startTime },
};
this.timeline.clipLanes.push(lane);
this.syncClipLanes();
this.evaluate();
this.emitChanged();
return lane.id;
}
// Move a clip strip along its lane. Snaps to the nearest overlap-free slot so
// it never runs into a neighbour, and stays within the timeline.
moveClipLane(laneId: string, startTime: number): void {
if (!this.timeline) return;
const lane = this.timeline.clipLanes.find((l) => l.id === laneId);
if (!lane) return;
lane.strip.startTime = this.resolveFreeStart(
lane.objectUuid,
laneId,
startTime,
lane.strip.duration,
);
this.syncClipLanes();
this.evaluate();
this.emitChanged();
}
// User trim (right edge): set the strip's on-timeline length. Clamped to at
// least one frame and so its end never crosses the next clip or the timeline
// end. Marks the duration as hand-set so a later clip (re)load won't overwrite it.
resizeClipLane(laneId: string, duration: number): void {
if (!this.timeline) return;
const lane = this.timeline.clipLanes.find((l) => l.id === laneId);
if (!lane) return;
const minDur = 1 / (this.timeline.fps || DEFAULT_TIMELINE_FPS);
const nextStart = this.nextStartAfter(
lane.objectUuid,
laneId,
lane.strip.startTime,
);
const maxDur = nextStart - lane.strip.startTime;
lane.strip.duration = Math.max(minDur, Math.min(duration, maxDur));
lane.strip.autoDuration = false;
this.syncClipLanes();
this.evaluate();
this.emitChanged();
}
// Toggle whether a clip loops for the remainder of its strip.
setClipLoop(laneId: string, loop: boolean): void {
if (!this.timeline) return;
const lane = this.timeline.clipLanes.find((l) => l.id === laneId);
if (!lane) return;
lane.strip.loop = loop;
this.syncClipLanes();
this.evaluate();
this.emitChanged();
}
// Called by CharacterAnimationManager once a clip GLB has loaded and its
// real length is known. Adopts the natural length only for a fresh drop
// (autoDuration still set) — a user-trimmed strip keeps its hand-set length.
resolveClipDuration(laneId: string, naturalDuration: number): void {
if (!this.timeline || !(naturalDuration > 0)) return;
const lane = this.timeline.clipLanes.find((l) => l.id === laneId);
if (!lane || lane.strip.autoDuration === false) return;
// Adopt the natural length but never let it overrun the next clip or the
// timeline end (single-row, non-overlapping).
const nextStart = this.nextStartAfter(
lane.objectUuid,
laneId,
lane.strip.startTime,
);
const maxDur = nextStart - lane.strip.startTime;
lane.strip.duration = Math.min(naturalDuration, maxDur);
lane.strip.autoDuration = false;
this.emitChanged();
}
removeClipLane(laneId: string): void {
if (!this.timeline) return;
this.timeline.clipLanes = this.timeline.clipLanes.filter(
(l) => l.id !== laneId,
);
this.syncClipLanes();
this.evaluate();
this.emitChanged();
}
// All clip lanes for one character (stacked lanes, drop order).
clipLanesFor(objectUuid: string): ClipLane[] {
if (!this.timeline) return [];
return this.timeline.clipLanes.filter((l) => l.objectUuid === objectUuid);
}
// ─── save / cancel ──────────────────────────────────────────────────────
save(): void {
if (!this.timeline) return;
const before = this.saved
? cloneTimeline(this.saved)
: cloneTimeline(this.timeline);
const action = new SaveTimelineAction(this, before);
this.saved = cloneTimeline(this.timeline);
if (action.commit()) this.editor.history.record(action);
this.emitChanged();
}
cancel(): void {
if (!this.saved) return;
this.timeline = cloneTimeline(this.saved);
this.syncClipLanes();
this.evaluate();
this.emitChanged();
this.emitPlayhead();
}
// ─── internals ────────────────────────────────────────────────────────
private evaluate(): void {
if (!this.timeline) return;
const cameraController = this.editor.cameraController;
for (const track of this.timeline.tracks) {
const snap = sampleTrackAt(track, this.playhead);
if (!snap) continue;
writeTransform(this.editor, track.objectUuid, snap);
// In camera view the canvas renders through the viewport camera and
// CameraController.tickPerFrame copies camera → cam_obj (the user's
// flying drives the proxy). Timeline-driven camera motion goes the
// other way, so mirror it onto the viewport camera here — otherwise
// playback/scrubbing shows no movement while looking through the
// camera (it's only visible as the frustum moving in viewport mode).
if (
cameraController.getCameraPersonMode() &&
cameraController.cam_obj?.uuid === track.objectUuid &&
cameraController.camera
) {
cameraController.camera.position.copy(
cameraController.cam_obj.position,
);
cameraController.camera.rotation.copy(
cameraController.cam_obj.rotation,
);
}
}
// Pose characters from their skeletal clip lanes at the same playhead.
this.editor.characterAnimationManager.evaluateAt(this.playhead);
}
// ─── clip-lane overlap guard (single row per character) ─────────────────
// Snap `desiredStart` to the nearest position where a `duration`-long strip
// fits without overlapping the character's other clips, within the timeline.
// Used by add + move so a character's clips stay a non-overlapping sequence.
private resolveFreeStart(
characterUuid: string,
exceptLaneId: string | null,
desiredStart: number,
duration: number,
): number {
if (!this.timeline) return desiredStart;
const maxStart = Math.max(0, this.timeline.duration - duration);
const clamp = (s: number) => Math.max(0, Math.min(s, maxStart));
const others = this.otherStripIntervals(characterUuid, exceptLaneId);
const desired = clamp(desiredStart);
if (!this.overlapsAny(desired, duration, others)) return desired;
// Candidate slots: flush before/after each neighbour, plus the two ends.
const candidates = [0, maxStart];
for (const iv of others) {
candidates.push(iv.end);
candidates.push(iv.start - duration);
}
const valid = candidates
.map(clamp)
.filter((s) => !this.overlapsAny(s, duration, others));
if (valid.length === 0) return desired; // no room — accept the clamp
valid.sort((a, b) => Math.abs(a - desired) - Math.abs(b - desired));
return valid[0];
}
// Start of the nearest clip that begins at/after `start` (excluding
// `exceptLaneId`), or the timeline end if there is none. Bounds trim/length.
private nextStartAfter(
characterUuid: string,
exceptLaneId: string | null,
start: number,
): number {
if (!this.timeline) return start;
let next = this.timeline.duration;
for (const iv of this.otherStripIntervals(characterUuid, exceptLaneId)) {
if (iv.start >= start) next = Math.min(next, iv.start);
}
return next;
}
// Occupied [start, end] intervals of a character's OTHER clip strips, sorted.
private otherStripIntervals(
characterUuid: string,
exceptLaneId: string | null,
): Array<{ start: number; end: number }> {
if (!this.timeline) return [];
return this.timeline.clipLanes
.filter((l) => l.objectUuid === characterUuid && l.id !== exceptLaneId)
.map((l) => ({
start: l.strip.startTime,
end: l.strip.startTime + l.strip.duration,
}))
.sort((a, b) => a.start - b.start);
}
private overlapsAny(
start: number,
duration: number,
intervals: Array<{ start: number; end: number }>,
): boolean {
const end = start + duration;
return intervals.some((iv) => start < iv.end && end > iv.start);
}
// Reconcile the skeletal-animation runtime whenever the clip-lane set changes.
private syncClipLanes(): void {
this.editor.characterAnimationManager.sync(this.timeline?.clipLanes ?? []);
}
private sortTrack(track: TimelineTrack): void {
track.keyframes.sort((a, b) => a.time - b.time);
}
private emitChanged(): void {
this.editor.bus.emit(
new TimelineChangedEvent(
this.timeline !== null,
this.timeline?.duration ?? DEFAULT_TIMELINE_DURATION,
this.timeline ? cloneTimeline(this.timeline).tracks : [],
this.timeline ? cloneTimeline(this.timeline).clipLanes : [],
),
);
}
private emitPlayhead(): void {
this.editor.bus.emit(
new TimelinePlayheadEvent(this.playhead, this.isPlaying),
);
}
}

Some files were not shown because too many files have changed in this diff Show More