Compare commits

..
Author SHA1 Message Date
Tanish Manku b9ee643bf6 Stop the bootstrap scripts claiming the dev webapp targets the local backend
It never has: setDevelopment() is commented out in main.tsx, so the dev server
falls through to its /v1 proxy at api.storyteller.ai. Both bootstrap epilogues
and the seed script now say so and print the VITE_DEV_API_HOST needed to
actually reach the local backend.
2026-08-03 17:24:39 -07:00
Tanish Manku 07e9a35044 Add portable local MySQL/Redis bootstrap scripts for backend dev
Brings up a fully local storyteller-web: portable MySQL 8.4 and Redis under a
gitignored .devstack/ (plain user processes -- no admin, no services, no WSL,
no Docker), all 281 migrations, role/badge seeds, and a demo login with
credits. Windows and Linux variants, plus a doctor that reports what is and
isn't healthy.

Scripts and docs only. Nothing here is compiled into any binary, and no
application behaviour changes: the backend it starts is the one that ships.

Integrations that cannot be provided locally -- object storage, generation
providers, email -- get placeholder credentials, so those endpoints fail when
exercised. Swap in a real dev credential for the one you are working on. The
doc records exactly which areas that leaves working.

An earlier revision closed those gaps with development-only behaviour inside
storyteller-web (DEV_FAKE_GENERATION, a filesystem bucket mode, a CDN_BASE_URL
override). That is deliberately not here: a switch that exists in the shipped
binary can be flipped. Frontend work that wants a populated, working stack
should use frontend/apps/fake-storyteller-web instead.
2026-08-03 14:41:14 -07:00
hanashi 0eddb659fa fixup 3d on admin dash, add databox (#1823) 2026-07-30 20:20:44 -04:00
hanashi 17c12836ad seedream 5 plus (#1822) 2026-07-30 18:25:19 -04:00
BFlat 7654e95636 Desktop: drag & drop files onto the promptbox on create pages (#1821)
### What
- Drag files onto the promptbox on the Image, Video, and Audio create
pages to add them as references - same look and behavior as the webapp
- On those pages the promptbox is now the only drop target; dropping
elsewhere does nothing
- Every other page keeps the existing upload-modal behavior

### Notes
- The webapp's drop code doesn't work as-is on desktop - Tauri
intercepts file drags before the page sees them, so desktop needed its
own path
- Both now share one component, so the two apps stay in sync going
forward
- Model limits still apply: each box only accepts the reference types
its model supports, and existing slot/duration caps are unchanged
2026-07-30 22:37:34 +07:00
BFlat 1bb31f42d7 Bug fix: hand-typed mentions (#1819)
- **Hand-typed mentions always take the character's exact name** -
typing `@person` for a character named `Person` now converts to
`@Person` as you type, so the tag binds correctly instead of silently
failing at generation time.
2026-07-30 08:05:44 +07:00
BFlat bc4b3a6c1f Bug fix: mention textarea & reference deck (#1818)
- **Prompt input no longer freezes at a character tag** - the caret
could get stuck inside a chip (e.g. after deleting the space behind it),
blocking all typing.
- **Reference add-menu no longer hides behind the sidebar** - it now
renders above everything instead of getting clipped by the promptbox
layers.
- **Character autocomplete is scrollable in focus mode** - the
fullscreen prompt's scroll lock was blocking it; same fix for the chip
menu's Replace list.
2026-07-30 07:43:40 +07:00
BFlat 13b5e7b5c2 Mention textarea focus & dropdown fixes (#1817)
- Fix popovers (model selector etc.) closing instantly when the mention
textarea was focused
- Fix mention autocomplete not clickable in prompt focus mode
- Fix chip "Replace" reopening the autocomplete - now confirms
immediately
- Fix autocomplete appearing at top-left of screen when caret is next to
a leading chip
- Don't open the autocomplete when clicking the caret onto an
already-confirmed chip
2026-07-29 23:09:26 +07:00
BFlat 4ff5bdf475 Webapp: mobile audio ref fixes (#1816)
- **Fixed: picking an audio reference on mobile opened the photo/video
picker.** iPhones don't have an audio library, so the browser fell back
to Photos - the only thing you could "attach as audio" was a video, and
it silently uploaded as an audio ref. Audio pickers now open the Files
app filtered to real audio files (MP3, WAV, M4A, FLAC…).
- **Non-audio files are now rejected with a clear error toast** instead
of being silently attached (also covers drag & drop and Android's "any
file" picker).
- **Fixed: limit warnings never appeared in the web app.** Errors like
"Total audio duration cannot exceed 15s" from shared components were
firing into a toast system the web app doesn't render. They're now
routed to the app's own toasts, so length/limit errors actually show up.
- Applies to the create video, create audio, and desktop reference deck
flows
2026-07-29 22:17:04 +07:00
BFlat e1b16cf2de Character mention chips + mention fixes (#1815)
### New stuff
- Character `@mentions` in the video promptbox now show as **chips with
a thumbnail**, so you can see exactly which character is attached to the
prompt
- Clicking a chip opens a menu to **Replace** (pick another character),
**Preview** (view their image), or **Remove** the mention
- Chips behave like a single unit - one backspace deletes the whole
mention, and removing/replacing can be undone with Ctrl+Z
- Copying prompt text still gives clean plain text (`@name ...`), and
pasting it back restores the chips
- When two characters share a name, the one you actually picked is now
the one that gets sent

### Fixes
- Mobile: the `@` autocomplete no longer goes off-screen - it stays
inside the viewport, flips below the caret when needed, and taps select
reliably
- Focus mode: the autocomplete no longer pushes the prompt sideways /
causes overflow
- Enter-to-generate now works while the mention editor is active on the
webapp
2026-07-29 21:43:02 +07:00
BFlat 2d7bf65a23 Bug fix: created characters token (#1813)
- Create flow stored `inference_job_token` as the `character_token`, so
mentioning
a new character sent a bogus token - backend drops it and name-matches
the
  raw `@name` against stale characters.
- Don't add pending characters to the mention store; they aren't
generatable yet.
- Keep the pending poll running after the modal closes and sync real
tokens into
  the store once the character is active.
- Remove unused `addCharacter` store action.
- Same fix in all three copies: webapp, website, shared `promptbox` lib.
2026-07-28 22:03:30 +07:00
Brandon Thomas 6ee5518c55 api keys for everyone (#1811) 2026-07-27 14:55:54 -04:00
Brandon Thomas f708402e8a grab billing data for reports (#1810) 2026-07-27 10:03:13 -04:00
BFlat ffa86f0ba8 Webapp: Video Frame Extractor (#1809)
Ports the Video Frame Extractor from the desktop app to the webapp at
`/frame-extractor`, with a reworked UI.

### What's new
- New Frame Extractor page: upload a video, pick one from your library,
or drag and drop
- Scrub to any moment and capture the exact frame at the playhead with
one click
- Frame stepping buttons and keyboard shortcuts (arrows to step, Space
to play/pause, C or Enter to capture)
- Burst capture: grab a sequence of frames to find the sharpest one,
with progress and cancel
- Each captured frame can be used as an image reference, animated into a
video, saved to the library, or downloaded
- Save all and clear actions for captured frames

### Integrations
- "Extract frames" action on library videos (selection bar and lightbox)
opens the extractor with that video preloaded
- Sends frames straight into the create-image / create-video reference
deck, same flow as "Send to prompt"
- New sidebar entry (purple NEW badge) and home page card
- Page is public; saving and references prompt sign-up when logged out
2026-07-27 18:52:47 +07:00
BFlat b741cdd76c Webapp: send library media to the create-image / create-video promptbox (#1807)
- Adds a **Send to prompt** action to the library's multi-select bar
with two
  targets: **Create image** and **Create video**
- Selected images are attached as reference images on the target page
without
touching the user's typed prompt or settings; selection is cleared and
the
  app navigates to the create page
- Selected videos can be sent to the video prompt as reference videos;
models
without video-reference support (only Seedance has it) get an
explanatory
  toast instead of a silent drop
- Media travels via `pendingRefImages` / `pendingRefVideos` store slots
(mirroring the existing `pendingRecreate` pattern) and is merged on the
  receiving page once its model list is loaded, so the **real per-model
reference caps** apply - fixes silent no-ops when refs were already
attached
- Merging dedupes by media token, clamps to the model cap, and toasts
partial
  adds ("Added 3 of 5 …"), full decks, and unsupported models
- Keyframe-only video models take the first sent image as the start
frame;
reference-capable models switch into reference mode so the deck is
visible
- DRY: shared `galleryItemToRefImage` mapper (reused by the create-image
library picker) and `buildVideoRefsToAdd` (reused by the video-ref
picker,
  duration probing + total-duration cap)
- Bar UX: option list adapts to the selection (video-only hides Create
image),
  icon-only button on phones to avoid overflowing the bar
2026-07-25 15:07:21 +07:00
bflatastic 9fbbfb5877 Webapp: copy paste image references into promptbox 2026-07-25 09:10:01 +07:00
BFlat a49426b77d Webapp/desktop: multi-select batch download on the create-image page (#1806)
- Multi-select and batch download also enabled on the create-image page
(webapp + desktop); zip renamed to artcraft-media-*.zip
2026-07-23 20:37:40 +07:00
BFlat 9f28c525ec Webapp/desktop: batch download video results, refresh credits after failed generations (#1801)
## Changes

**Batch download**
- Add multi-select mode to the create-video results feed (webapp +
desktop) with a "Select" toggle in the topbar, checkboxes on completed
items, and a floating download bar
- Webapp: "Download" bundles selected videos into a single .zip
client-side (new jszip dependency, concurrent fetches, progress and
partial-failure toasts)
- Desktop: "Download" prompts for a folder and saves each selected video
as its own file via native Tauri save, with progress toasts
- Library: add a Download (zip) button to the existing bulk selection
bar
- Shared `ui-generation-list` lib gains a selection store, select
toggle, action bar, and optional selection props on the grid/list views
(existing hosts unaffected)

**Credits refresh on failed generations**
- Webapp: dispatch a credits refresh when the task queue or generation
feed observes a newly failed job (immediately and again after 2.5s so
server-side refunds settle)
- Desktop: also refresh credits when the 5s task poll discovers a
failure that the push event missed, and listen for `credits-change` in
the TopBar

**UI fixes**
- Selection download bar always sits above the promptbox regardless of
its size
- Mobile Generate/History header no longer overlaps the gallery toggles
(flex layout instead of absolute positioning)
- Reference deck top-aligns in the expanded prompt box instead of
centering
- "Play videos with sound" setting now defaults to on
2026-07-23 19:54:12 +07:00
BFlat d4239a3516 Webapp: drag & drop files onto the prompt box to add references (#1800)
- Drop images, videos, or audio anywhere on the prompt box - files are
sorted into the right reference type based on what the current model
supports
- Drag-over overlay shows which file kinds are accepted; turns red when
the hovered files aren't usable
- Video keyframe mode: dropped images fill the start frame, then the end
frame
- Unsupported kinds, full slots, and non-media files get a clear toast
instead of failing silently
- Reuses the existing `useDeckMedia` upload pipelines, so limits and
error handling match the "+" pickers
2026-07-23 03:29:49 +07:00
BFlat 109d7081db Fix: silent failures when submitting a generation (recreate "does nothing" bug) (#1799)
Users reported that after using **Recreate**, pressing the generate
arrow did nothing at all (no error, no card, no spinner). Root cause:
several failure paths gave zero feedback, so a rejected request looked
like a dead button or an outage.

### Changes

- **Show real error messages.** The API layer now keeps the server's
error message (e.g. "Image is required") instead of throwing it away, so
users see why a generation was rejected.
- **Toast on every failed submit.** Image and video pages (webapp +
website) now show an error toast when a generation fails to start.
Previously, most failures were recorded invisibly and never shown.
- **Out of credits feedback on the website.** 402 responses now show a
toast (webapp keeps its upgrade modal).
- **Network errors are announced** instead of failing silently.
- **No more silent submit blocks.** Cases that used to quietly ignore
the click now explain themselves:
  - Model list still loading
- Model needs a starting image (website; also un-disabled the button so
clicking gives feedback)
- **Recreate model fallback is announced.** If the original generation's
model is no longer available, recreate now tells the user it switched to
the default model instead of silently swapping.
2026-07-22 13:27:06 +07:00
BFlat 5faa94a9dd Webapp/desktop: video preview and lightbox sound settings (#1798) 2026-07-22 08:17:19 +07:00
Tanish Manku 9c83c20c46 Frontend: Moodboard persistence (#1793) 2026-07-20 19:08:56 -07:00
Tanish Mankuandbflatastic cb4374ea70 Frontend: video editor persistence (#1792)
Co-authored-by: bflatastic <bombayapirat@gmail.com>
2026-07-20 18:01:24 -07:00
BFlat eb59e7fcd7 Webapp: move audio/3D reference inputs into the prompt box reference deck (#1795)
## Summary

Compacts the legacy reference bands above the prompt box (audio ref row,
multi-view/input-mesh row, reference video row) into the reference deck
design already used on the image and video pages. Mobile forms keep
their existing rows.

## Changes

- **Create Audio** - audio tracks are now deck cards (playable tile,
duration badge) with Upload / From library in the deck's "+" menu; band
removed. Mutual-exclusion and "Audio track required" behavior unchanged.
- **Create 3D World** - guide video is now a deck card; store reworked
to the shared `RefVideo[]` shape.
- **Create 3D Object** - Front/Back/Left/Right are always-visible
labeled slot cards next to the image-ref deck (same design as First/Last
frame on video), plus a "Mesh" slot card for mesh-to-mesh models. Upload
logic deduplicated into `mesh-upload.ts`.
- **Webapp PromptBox** - removed the `mediaReferenceRow` band prop; deck
video/audio groups gate on `videoRefsSupported`/`audioRefsSupported` (no
longer video-page-only); new `referenceSlots` prop; `maxImages: 0`
blocks image uploads on pages whose model takes no image refs.
- **Deck lib** - exported `DeckSlotCard`; new `mesh` card kind;
`sortable: false` for fixed slots; non-finite duration caps hide the
seconds limit; collapsed fan width now tracks card count, so a single
ref sits snug against the prompt (multi-ref unchanged).
2026-07-20 21:12:01 +07:00
BFlat 185bf835a1 Desktop: tags in the gallery modal (#1794)
- Tags now work in the desktop app - no separate build, it all comes
through the shared gallery/lightbox components
- Open any item in the gallery → edit its tags in the details panel
(same chip editor as the webapp, with autocomplete)
- New "Tags" section in the gallery sidebar: your 5 most-used tags + an
"All tags" entry
- Click a tag to see all media with that tag in the normal grid
- "All tags" opens a word cloud (bigger = used more), click a word to
jump in
- Webapp sidebar now also caps at 5 tags with a "See all tags" link
- The tag editor is now one shared component used by both apps instead
of two copies
2026-07-17 20:44:49 +07:00
BFlat 65463ae14c Update character posing video link in README 2026-07-16 22:40:51 +07:00
BFlat fc61c6efd1 Update README.md character pose video 2026-07-16 22:34:47 +07:00
bflatastic 4788ee9a23 Website: character pose video swap 2026-07-16 21:53:00 +07:00
BFlat 77cee77258 Webapp: media tags (#1791)
- Tags editor in LightboxDetails (lightbox + /media/:id): chip input
with autocomplete over the user's existing tags, comma/paste splitting,
duplicate rejection, owner gating via session, and serialized+coalesced
SetMediaFileTags snapshot saves with optimistic chips and error revert
(components/tags/TagChipInput.tsx, TagsSection.tsx)
- Tag browser: "Tags" tab in the library with name/most-used sort, tag
view at /library/tag_… reusing the media grid, breadcrumb header with
rename (FolderNameDialog) and delete (confirm modal), tag context menu
- Sidebar: tags rows under the Assets group below folders, with
expand/collapse chevrons on both Folders (default open) and Tags
(default closed), animated via framer-motion; tag icons use violet to
distinguish them from folders
- New library-tags-store mirroring the folders store (cursor-paged media
caches, optimistic rename/delete with toast revert); shared item mappers
extracted to library-media-map.ts to avoid a circular import; media
deletion now purges tag caches; bulk-selection bar resolves thumbnails
from tag caches too
- Topbar breadcrumbs map /library/tags and tag views to "Tags"
- Tags show as word cloud
2026-07-15 21:57:21 +07:00
BFlat 9f69eb162a Webapp/Desktop: clear all button for promptboxes (#1790) 2026-07-15 19:49:21 +07:00
BFlat 46d05808cc Reference decks + new generate button for image/video promptboxes (#1783)
Redesigns how references are attached in the image and video
promptboxes, on both the desktop app and the webapp (desktop layout -
mobile is unchanged).

### Reference deck
- Replaces the image-toggle icon and the upload bands above the prompt
with a fanned stack of reference cards inside the promptbox, left of the
prompt text
- Hovering the deck expands it to show every attached ref - no
scrolling, it widens toward the end of the promptbox and wraps into more
rows as needed (ready for up to 50 refs for Seedance 2.5)
- Cards support drag-to-reorder (grab/grabbing cursors), hover remove,
name tooltips (Image 1, Video 2, …), upload progress spinners on
individual cards, and click-to-preview fullscreen
- A circular **+** overlaps the deck: hovering shows an add menu grouped
by type (Image / Video / Audio) with compact limits (e.g. `2/9`, `1/3 ·
8/15s`); clicking it opens a file picker that accepts any supported
media and sorts files by type automatically

### Video modes
- Keyframe mode: Start/End frame band replaced with compact "First frame
⇄ Last frame" cards with a swap button
- Reference mode renamed to **Omni reference** and is now the default;
images, videos, and audio all stack into one mixed deck

### Generate button
- Replaced the "Generate" pill with a circular arrow-up button, credit
cost shown beside it

### Under the hood
- New shared deck components live in `@storyteller/ui-promptbox` and are
used by both apps (previously the two promptbox implementations were
fully separate)
- New `GenerateIconButton` in `@storyteller/ui-button`; the old
`GenerateButton` stays for mobile
- Webapp mobile, audio/object/world pages, and the 2D/3D/Edit
promptboxes keep their existing UI
2026-07-14 21:08:15 +07:00
BFlat dbe38a588f Webapp: swap folders and unfoldered sidebar links (#1788) 2026-07-14 20:51:18 +07:00
Tanish Manku 58e2d08c76 Frontend: Endpoint migration and cleanup (#1787) 2026-07-13 22:44:41 -07:00
Tanish Manku f991eafd01 Frontend: mesh/splat classes and handling (#1786) 2026-07-13 22:42:41 -07:00
Tanish Manku 283b4ab04f Frontend: remove dead API client code targeting unregistered endpoints (#1785)
Deletes dormant legacy API surface (zero live callers, verified against
the full backend route table):

- VoiceDesignerApi, VoiceConversionApi, AnalyticsApi
(/v1/analytics/log_session was never registered) - both @storyteller/api
and the Tauri app's ApiManager copies, plus their tests
- PromptsApi/EngineApi: /v1/image_studio/prompt,
/v1/image_studio/session_jobs, /v1/jobs/session/{token} methods (routes
do not exist)
- UserBookmarksApi: broken-path list methods (/list/{user} and
/list/{et}/{etok}; backend shapes are /list/user/{u} and
/list/entity/{et}/{etok}) and the FakeYou-era scoped enums that only
they consumed
- Dashboard UsersApi: RequestPasswordReset/ResetPassword pointing at
removed /v1/request_password_reset + /v1/reset_password (live pages use
PasswordResetApi -> /v1/password_reset/* correctly)

Fixes WeightsApi cover-image path to the registered route:
/v1/weights/weight/{token}/cover_image (both copies).
2026-07-13 21:56:11 -07:00
Brandon Thomas 0003cad179 seedream5 (#1784) 2026-07-13 11:10:21 -04:00
295 changed files with 20795 additions and 3728 deletions
+5
View File
@@ -1,3 +1,8 @@
# Shell scripts and dotenv config files are executed/parsed on Linux (WSL2, CI)
# even when the repo is cloned on Windows — they must stay LF in the worktree.
*.sh text eol=lf
*.env text eol=lf
# Ignore certain code for Github language statistics
# https://github.com/github/linguist/blob/master/docs/overrides.md
old_code/trumpet-trumped.com/dictionary/cmudict-0.7b linguist-vendored
+3
View File
@@ -104,3 +104,6 @@ windows/sqlite/sqlite3.lib
crates/desktop/src-tauri/*.dll
windows_dlls/
# Portable local dev stack (script/bootstrap) - MySQL/Redis binaries, data, logs
.devstack/
+1 -1
View File
@@ -28,7 +28,7 @@ Text-to-image is great, but artists *need control*. It's important to know what
| **3D Image Compositing** | ![Video](https://github.com/user-attachments/assets/f93a616f-571d-474e-bcc0-53736de7303d) Use images (backdrops, foreground elements, props, etc.) in scenes with depth and blend them naturally together. Just a couple of images usually leads to great compositions. |
| **2D Image Compositing** | ![Video](https://github.com/user-attachments/assets/d6f99391-e496-4c62-9e37-29734ba5f899) Use images, background removal, layers, and simple drawing tools to precisely compose a scene. |
| **Image to 3D Mesh** | ![Video](https://github.com/user-attachments/assets/600a405c-e360-48c1-9b42-6e657ae6243b) It's almost impossible to lay out complicated objects or block complicated scenes; turning images into 3D helps position elements exactingly and intentionally. |
| **Character Posing** | ![Video](https://github.com/user-attachments/assets/52a8e983-7c8f-42d2-be8b-25296ab9ed57) You can dynamically pose your characters to achieve the precise character, scene, and camera blocking before calling "action". |
| **Character Posing** | ![Video](https://raw.githubusercontent.com/storytold/github-media/22a27373707b13f48fc56405f1f89ca143c43d5d/character-posing.webp) You can dynamically pose your characters to achieve the precise character, scene, and camera blocking before calling "action". |
| **Scene Blocking w/ Kit Bashing** | ![Video](https://github.com/user-attachments/assets/eef025ac-0346-4a46-a023-d48e23629eb5) Use 3D asset kits to precisely block out your scene: get the correct angles, object positions, and rich depth layering you can't with text prompting. |
| **Character Identity Transfer** | ![Video](https://github.com/user-attachments/assets/629119ee-8c76-4a83-9827-8c6c995a3ec1) Use mannequins as simple 3D ControlNets for posing any character. |
| **Background Removal** | ![Video](https://github.com/user-attachments/assets/90c65057-5531-404f-83af-b34e66e24ec1) Remove backgrounds from images to make them useful in 2D or 3D compositing. They can be props, layers, or backdrops. |
+293
View File
@@ -0,0 +1,293 @@
# Local Dev Stack — Requirements & Bootstrap
How to run `storyteller-web` on your own machine against a local MySQL and
Redis — no shared dev database, no Docker, no admin rights. This is for **work
on the backend itself**.
> **Working on the frontend?** You probably want
> `frontend/apps/fake-storyteller-web` instead — an in-memory stand-in for this
> API that needs no database, no build, and no setup. Everything below is
> unnecessary for webapp work. Come back here when you need the real server.
These scripts install and supervise infrastructure. They change no application
behaviour, and nothing in them is compiled into any binary: the backend they
start is the same one that ships. Integrations they cannot provide locally
(object storage, generation providers, email) get placeholder credentials, so
the endpoints that use them fail when exercised. Swap in a real dev credential
for whichever one you're actually working on.
The automated path on **Windows** (primary platform, native — no WSL, no
Docker, no admin rights for the stack itself):
```powershell
.\script\bootstrap\windows\bootstrap_dev_stack.ps1 # one-time setup, idempotent, re-run anytime
.\script\bootstrap\windows\run_backend_dev.ps1 # terminal 1: API on http://localhost:12345
.\script\bootstrap\windows\seed_demo_user.ps1 # terminal 2: create demo login (localdev1 / localdev1pass)
$env:VITE_DEV_API_HOST = "http://localhost:12345" # terminal 2: aim the webapp at your backend
cd frontend; npx nx dev artcraft-webapp # webapp on http://localhost:4201
.\script\bootstrap\windows\dev_stack_doctor.ps1 # health check, any time
.\script\bootstrap\windows\stop_dev_services.ps1 # stop MySQL/Redis when done
```
The equivalent on **Linux** (Ubuntu 22.04+ — native, WSL2, or CI):
```bash
./script/bootstrap/bootstrap_dev_stack.sh # one-time setup, idempotent, re-run anytime
./script/bootstrap/run_backend_dev.sh # terminal 1: API on http://localhost:12345
./script/bootstrap/seed_demo_user.sh # terminal 2: create demo login (localdev1 / localdev1pass)
cd frontend && VITE_DEV_API_HOST=http://localhost:12345 npx nx dev artcraft-webapp # terminal 2: webapp on :4201
./script/bootstrap/dev_stack_doctor.sh # health check, any time
```
This document records the underlying requirements so the scripts aren't a
black box, and corrects some stale claims in older docs.
## Supported platforms
| Platform | Status |
|---------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Windows 10/11 | Primary. `script/bootstrap/windows/*.ps1` run a **portable stack**: MySQL 8.4 (official zip) and Redis live under the gitignored `.devstack/` and run as plain user processes — no admin, no services, no WSL, no Docker. Build tools (cmake, perl, nasm, llvm) are offered via winget |
| Ubuntu 22.04+ | Supported by `script/bootstrap/*.sh` (native, WSL2, or CI runner/container); services via apt |
| macOS | Not covered by the scripts; follow `_docs/dev_setup_server.md` manually |
### Windows specifics
- Everything stateful lives in `.devstack/` (gitignored): `mysql/` (extracted
zip), `mysql-data/`, `redis/`, `logs/`, `pids/`, `downloads/` (cached zips),
`my.ini`. Delete the directory to fully reset; re-run bootstrap to rebuild.
- Pinned services: MySQL **8.4.11** from `cdn.mysql.com` (the 8.x series is
required; only some patch versions exist on the CDN) and Redis **5.0.14.1**
(tporadowski build — old but sufficient for the backend's keepalive/TTL/
rate-limit usage; override `REDIS_ZIP_URL` to use Memurai/Garnet instead).
- The portable MySQL root user has **no password** (`--initialize-insecure`) —
local dev only, bound to 127.0.0.1.
- diesel-cli is compiled against the portable MySQL's own client library
(`MYSQLCLIENT_LIB_DIR`/`MYSQLCLIENT_VERSION`); `libmysql.dll` must be on
`PATH` to *run* it — the scripts handle both.
- Compiling the backend needs cmake, Strawberry Perl, NASM, and LLVM/libclang
(the `wreq` HTTP client builds BoringSSL, and `aws-lc-sys`/`ring` need
NASM/clang on MSVC). The bootstrap checks and offers winget installs.
- The generated secrets file additionally sets `TEMP_DIR` (the upload temp-dir
helper defaults to `/tmp`, which doesn't exist on Windows).
## Requirements matrix
| Requirement | Version / detail | Needed for |
|-------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------|
| MySQL server | 8.x series (9.x unsupported by tooling) | Backend boot (eager connect) + migrations |
| Redis server | any recent | Backend boot (r2d2 pool connects eagerly) |
| Rust (stable) | no pinned MSRV; Docker builds use 1.93.0 | Building `storyteller-web`, diesel_cli |
| diesel_cli | `cargo install diesel_cli --no-default-features --features mysql,sqlite` | Running the 281 migrations |
| Node.js | 20+ (enforced by `script/common/frontend_preflight.sh`) | Frontend (Nx 21 / Vite 6) |
| npm | ships with Node; **pnpm must not be used** | Frontend install (`frontend/package-lock.json`) |
| System libs (apt) | build-essential, cmake, libssl-dev, libclang-dev, libfontconfig1-dev, fontconfig, perl, pkg-config, ffmpeg, libmysqlclient-dev, libsqlite3-dev, curl, jq | Compiling the workspace + diesel_cli; seeding scripts |
| Elasticsearch | optional (7.x) | Only 4 search endpoints; server boots without it |
| Docker | **not required** — no docker-compose exists | — |
Database identity (used consistently by docs, `.env`, and defaults):
database `storyteller`, user `storyteller`, password `password`, host
`localhost`.
## Backend: what `storyteller-web` needs to boot
Binary: `storyteller-web` in `crates/service/web/storyteller_web`
(binaries: `storyteller-web`, `docs` — Swagger UI on :8989, `docs-cli` — emits
`api.json`). Binds `0.0.0.0:12345` (`BIND_ADDRESS`), plain HTTP.
**Run it from the repo root** — the config search path and `includes/` paths
are cwd-relative.
### Config loading (corrected facts)
- Mode is selected by the `SERVER_ENVIRONMENT` **process env var only**
(default: Development). Setting it inside an env file does nothing — it is
read before any file loads.
- In Development mode, files load from
`crates/service/web/storyteller_web/config/` in this order:
`storyteller-web.common.env` → `storyteller-web.development.env` →
`storyteller-web.development-secrets.env` (gitignored; the bootstrap
generates it).
- **Precedence is first-value-wins** (dotenv never overrides an existing key):
real process env > common.env > development.env > development-secrets.env.
The secrets file therefore cannot override anything already set in
`development.env`.
- The repo-root `.env` is **NOT loaded by the server** (it passes
`ignore_legacy_dot_env_file: true`). Root `.env`'s `DATABASE_URL` feeds
diesel-cli/sqlx-cli only; the server reads **`MYSQL_URL`** (default
`mysql://storyteller:password@localhost/storyteller`).
### Hard requirements at boot
| Requirement | Behavior if missing |
|-------------------------------------------------------------------|------------------------------------------------------------------------------------------------------------------------------------|
| MySQL reachable via `MYSQL_URL` | Boot fails (sqlx pool connects eagerly; the DB must exist and be migrated — health-check and ban-poller threads query immediately) |
| Redis reachable via `REDIS_0_URL` (default `redis://localhost/0`) | Boot fails (r2d2 pool connects eagerly) |
| ~30 `get_env_string_required` vars present | Boot fails on the first absent var |
Of the required-present vars, 19 are supplied by **no** checked-in file and
live in the generated secrets file: `ACCESS_KEY`, `SECRET_KEY`, `REGION_NAME`,
`W2L_PRIVATE_DOWNLOAD_BUCKET_NAME`, `W2L_PUBLIC_DOWNLOAD_BUCKET_NAME`,
`RESEND_API_KEY`, `FAL_API_KEY`, `GMICLOUD_API_KEY`, `GROK_API_KEY`,
`BEEBLE_API_KEY`, `OPENAI_API_KEY`, `WORLDLABS_API_KEY`,
`SEEDANCE2PRO_COOKIES`, `SEEDANCE2PRO_WHITELIST_COOKIES`,
`SEEDANCE2PRO_BYTEPLUS_ULTRA_COOKIES`, `STRIPE_FAKEYOU_ACCOUNT_ID`,
`STRIPE_ARTCRAFT_ACCOUNT_ID`, `STRIPE_ARTCRAFT_SECRET_KEY`,
`STRIPE_ARTCRAFT_SECRET_WEBHOOK_KEY`.
**Placeholder values are safe at boot**: none of these services is contacted
at boot (clients are constructed offline; Elasticsearch, S3/R2, Stripe,
Resend, and Google-cert fetches are all lazy). But two earlier claims here
were wrong and are corrected:
- **The omni_gen endpoints call providers synchronously in the HTTP handler**
(`/v1/omni_gen/generate/{image,video}` — the path the webapp uses). With
placeholder keys the provider call fails and the submit returns HTTP 500
*after* billing the wallet (the image pipeline has no refund; video refunds
only Kinovi). Only the legacy polling providers (GmiCloud, Grok, Kinovi
orders, WorldLabs) are worker-driven.
- **Uploads fail hard with placeholder S3 creds** — every media upload endpoint
uploads to the bucket *before* inserting the DB row, and `rust-s3` is built
with `fail-on-err`.
### What does and doesn't work locally
| Area | Works with placeholder credentials? |
|----------------------------------------------|-------------------------------------|
| Login, sessions, account, profile | Yes |
| Wallets, credits, billing reads | Yes |
| Folders, tags, prompts, characters, projects | Yes |
| Media listing (rows) | Yes — but the images 404, see below |
| Media uploads | No — needs real R2/S3 credentials |
| Image/video/mesh/splat generation | No — needs a real provider key |
| Transactional email | No — needs a real Resend key |
| Elasticsearch-backed search | No — needs a local ES + reindex |
The rows-but-no-images case is the one that surprises people: `media_files`
rows carry a bucket path, and the URL the API builds from it points at the
shared R2 bucket, which does not have your local objects. Put real dev R2
credentials in the secrets file if you need media to render, or use
`fake-storyteller-web`, which serves its own bytes.
### Auth on localhost
- `POST /v1/login` takes `{"username_or_email": "...", "password": "..."}`;
success returns `{"success": true, "signed_session": "<jwt>"}` **and** a
`session` cookie.
- With the checked-in dev config (`COOKIE_DOMAIN=.jungle.horse`), the session
cookie is issued non-Secure and host-only (the Domain attribute is never
set), so plain-HTTP localhost login works as-is. The frontend additionally
stores `signed_session` in localStorage and sends it as a `session` header.
- Dev-mode CORS allows any `localhost`/`127.0.0.1` origin on any port, with
credentials.
- Health endpoint for scripts/CI: `GET /_status`.
### Build
- Compile with `SQLX_OFFLINE=true` (uses the checked-in `.sqlx/` cache; 483
queries). Without it, the sqlx macros open a **live DB connection at compile
time** using `DATABASE_URL` from the root `.env`. `SQLX_OFFLINE` has no
runtime effect — ignore older notes saying not to set it when running.
- First build takes several minutes; the crate has ~370 pre-existing warnings.
## Database: migrations and seed data
- 281 Diesel migrations in `_database/sql/migrations/` (`diesel.toml` at repo
root points there). Run from the repo root: `diesel migration run`.
`"Encountered unknown type for Mysql: enum"` warnings are harmless.
`migrations_squashed/` is documentation only — not executable.
- Seed data: `_database/sql/seed/sql/system_roles.sql` (roles `user`, `mod`,
`admin`) and `user_badges.sql`. **The `user` role is mandatory** — account
creation hardcodes `user_role_slug='user'`. The legacy wrapper
`bootstrap_inserts_roles_etc.sh` must run with cwd `_database/sql/` and is
not idempotent; the bootstrap script applies the same SQL files with
idempotency guards instead.
- Demo user: created through **`POST /v1/create_account`** (no captcha, no
invite code, no email-verification gate; password just needs 6+ chars). This
exercises the real signup path instead of hand-rolled SQL. Gotcha: usernames
like `demo`, `admin`, `test`, `dev`, `artcraft` are reserved
(`includes/binary_includes/usernames/reserved_usernames.txt`) — the default
demo user is `localdev1`. The Rust seeder
(`cargo run --bin dev-database-seed`) exists but hard-requires a
`.env-secrets` file and also seeds model weights, so the HTTP path is
preferred.
- Demo credits: signup creates **no wallet**, and the webapp gates the
generate button on wallet balance — so `seed_demo_user` also seeds an
`artcraft`-namespace wallet with `DEMO_CREDITS` (default 100,000) banked
credits, plus matching ledger rows. Idempotent: re-runs top the balance
back up to `DEMO_CREDITS` but never lower it. (Raw SQL, because the only
real credit-granting paths are Stripe webhooks and moderator tooling.)
- Demo gallery: **not seeded**. Media rows without reachable bytes render as
broken tiles, so seeding them locally would be worse than an empty library.
Use `fake-storyteller-web` for a populated one.
## Frontend: `artcraft-webapp`
- Nx workspace at repo-root `frontend/` (npm workspaces; `npm install` from
`frontend/`). The FontAwesome Pro registry token is committed in
`frontend/.npmrc` — no manual auth step.
- Serve: `npx nx dev artcraft-webapp` → `http://localhost:4201`. No lib
pre-build is needed in dev (libs resolve to TS source via the
`"development"` export condition); production `nx build` does need lib
builds.
- **By default the dev webapp targets production.** `src/main.tsx` points the
API host at the Vite dev origin, and Vite proxies `/v1` to
`https://api.storyteller.ai`. The adjacent `setDevelopment()` call — which
would point it at `localhost:12345` — is commented out. To aim the webapp at
your local backend, set `VITE_DEV_API_HOST=http://localhost:12345` before
`nx dev` (that override ships with the `fake-storyteller-web` change; until
it lands, uncomment `setDevelopment()` locally instead).
- Optional: `VITE_GOOGLE_CLIENT_ID` in `frontend/apps/artcraft-webapp/.env`
enables the Google SSO button; email/password login needs nothing.
- Other dev ports if you run more apps: website 4200, dashboard 4202 (separate
pnpm app, excluded from Nx), video-info-website 4250, genhub 4300, Tauri
frontend 5173.
## What still touches production (known, deliberate seams)
The bootstrap changes no runtime behaviour, so these remain open. Each is
closed by supplying a real credential, not by a code path:
| Seam | Today |
|---------------------------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Media/CDN URLs in API responses | The compiled-in `cdn_link.rs` constants point at the public `pub-….r2.dev` bucket, so a locally-created row's media URL resolves to an object that isn't there |
| Media uploads | 500 with placeholder credentials; supply real dev R2/S3 keys to exercise them |
| Generation | 500 with placeholder keys, *after* the wallet is billed; supply a real key for the provider under test |
| Vite `/v1` proxy in webapp | Targets `https://api.storyteller.ai`, so a dev webapp with no host override reads and writes production |
| Elasticsearch-backed search | `search_featured` / `search_session` media files, `weights/search`, legacy `tts/search` fail without a local ES + `elasticsearch-cli` reindex (optional) |
## CI usage
The same bootstrap works on a fresh Ubuntu runner/container:
```bash
export MYSQL_ROOT_PASSWORD=root # GitHub Actions' preinstalled MySQL uses password auth
./script/bootstrap/bootstrap_dev_stack.sh --yes
./script/bootstrap/run_backend_dev.sh &
./script/bootstrap/seed_demo_user.sh # waits for /_status
./script/bootstrap/dev_stack_doctor.sh
```
`--yes` answers all prompts; `--skip-frontend` / `--skip-rust-build` /
`--skip-packages` slice the work for focused jobs.
## Verification checklist
- `./script/bootstrap/dev_stack_doctor.sh` exits 0 with no FAILs.
- `POST /v1/login` with the demo user returns `success: true` plus a
`signed_session` (done automatically by `seed_demo_user.sh`).
- `GET /_status` returns 200, and MySQL has the full migration set applied.
- With `VITE_DEV_API_HOST` pointed at `localhost:12345`, the webapp logs in as
`localdev1` and shows its credits in the top bar. The library is empty and
generation 500s — both expected, see the table above.
## Corrections to older docs
- `crates/service/web/storyteller_web/CLAUDE.md`: the repo-root `.env` is
*not* auto-loaded by the server, and `SQLX_OFFLINE=true` is fine (and
recommended) when running — it only affects compile-time query checking.
- `_docs/dev_setup_server.md`: `./script/dev_mysql_connect.sh` doesn't exist —
it's `./script/dev_database_connect.sh`.
- `diesel.toml`'s comment about needing an unreleased diesel CLI is stale —
any current `diesel_cli` works.
- `TESTING_INFRA_HANDOFF.md` (untracked): the server env var is `MYSQL_URL`,
not `DATABASE_URL`; migration count is 281; crate path is
`crates/service/web/storyteller_web`.
@@ -43,6 +43,8 @@ pub enum RouterImageModel {
Seedream5Lite,
#[serde(rename = "seedream_5p0_pro")]
Seedream5p0Pro,
#[serde(rename = "seedream_5p0_pro_u")]
Seedream5p0ProUltra,
#[serde(rename = "qwen_edit_2511_angles")]
QwenEdit2511Angles,
#[serde(rename = "flux_2_lora_angles")]
@@ -80,4 +82,9 @@ mod tests {
fn seedream_5p0_pro_serializes_to_seedream_5p0_pro() {
assert_round_trip(RouterImageModel::Seedream5p0Pro, "seedream_5p0_pro");
}
#[test]
fn seedream_5p0_pro_ultra_serializes_to_seedream_5p0_pro_u() {
assert_round_trip(RouterImageModel::Seedream5p0ProUltra, "seedream_5p0_pro_u");
}
}
@@ -27,6 +27,7 @@ use crate::generate::generate_image::providers::artcraft::seedream_4::build::bui
use crate::generate::generate_image::providers::artcraft::seedream_4p5::build::build_artcraft_seedream_4p5;
use crate::generate::generate_image::providers::artcraft::seedream_5_lite::build::build_artcraft_seedream_5_lite;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro::build::build_artcraft_seedream_5p0_pro;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro_ultra::build::build_artcraft_seedream_5p0_pro_ultra;
use crate::generate::generate_image::providers::fal::flux_1_dev::build::build_fal_flux_1_dev;
use crate::generate::generate_image::providers::fal::flux_1_schnell::build::build_fal_flux_1_schnell;
use crate::generate::generate_image::providers::fal::flux_2_lora_angles::build::build_fal_flux_2_lora_angles;
@@ -122,6 +123,7 @@ impl GenerateImageRequestBuilder {
(RouterProvider::Artcraft, RouterImageModel::Seedream4p5) => build_artcraft_seedream_4p5(self),
(RouterProvider::Artcraft, RouterImageModel::Seedream5Lite) => build_artcraft_seedream_5_lite(self),
(RouterProvider::Artcraft, RouterImageModel::Seedream5p0Pro) => build_artcraft_seedream_5p0_pro(self),
(RouterProvider::Artcraft, RouterImageModel::Seedream5p0ProUltra) => build_artcraft_seedream_5p0_pro_ultra(self),
(RouterProvider::Artcraft, RouterImageModel::QwenEdit2511Angles) => build_artcraft_qwen_edit_2511_angles(self),
(RouterProvider::Artcraft, RouterImageModel::Flux2LoraAngles) => build_artcraft_flux_2_lora_angles(self),
(RouterProvider::Artcraft, RouterImageModel::Midjourney7) => build_artcraft_midjourney_7(self),
@@ -149,6 +151,7 @@ impl GenerateImageRequestBuilder {
(RouterProvider::Seedance2Pro, RouterImageModel::Midjourney7Niji) => build_kinovi_midjourney_7_niji(self),
(RouterProvider::Seedance2Pro, RouterImageModel::Midjourney8) => build_kinovi_midjourney_8(self),
(RouterProvider::Seedance2Pro, RouterImageModel::Seedream5p0Pro) => build_kinovi_seedream_5p0_pro(self),
(RouterProvider::Seedance2Pro, RouterImageModel::Seedream5p0ProUltra) => build_kinovi_seedream_5p0_pro(self),
_ => self.unsupported_provider_and_model(),
}
@@ -41,6 +41,8 @@ use crate::generate::generate_image::providers::artcraft::seedream_5_lite::cost:
use crate::generate::generate_image::providers::artcraft::seedream_5_lite::request::ArtcraftSeedream5LiteRequestState;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro::cost::ArtcraftSeedream5p0ProCostState;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro::request::ArtcraftSeedream5p0ProRequestState;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro_ultra::cost::ArtcraftSeedream5p0ProUltraCostState;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro_ultra::request::ArtcraftSeedream5p0ProUltraRequestState;
use crate::generate::generate_image::providers::fal::flux_1_dev::cost::FalFlux1DevCostState;
use crate::generate::generate_image::providers::fal::flux_1_dev::request::FalFlux1DevRequestState;
use crate::generate::generate_image::providers::fal::flux_1_schnell::cost::FalFlux1SchnellCostState;
@@ -97,6 +99,7 @@ pub enum ImageGenerationRequest {
ArtcraftSeedream4p5(ArtcraftSeedream4p5RequestState),
ArtcraftSeedream5Lite(ArtcraftSeedream5LiteRequestState),
ArtcraftSeedream5p0Pro(ArtcraftSeedream5p0ProRequestState),
ArtcraftSeedream5p0ProUltra(ArtcraftSeedream5p0ProUltraRequestState),
ArtcraftQwenEdit2511Angles(ArtcraftQwenEdit2511AnglesRequestState),
ArtcraftFlux2LoraAngles(ArtcraftFlux2LoraAnglesRequestState),
ArtcraftMidjourney7(ArtcraftMidjourney7RequestState),
@@ -144,6 +147,7 @@ impl ImageGenerationRequest {
Self::ArtcraftSeedream4p5(_) => RouterProvider::Artcraft,
Self::ArtcraftSeedream5Lite(_) => RouterProvider::Artcraft,
Self::ArtcraftSeedream5p0Pro(_) => RouterProvider::Artcraft,
Self::ArtcraftSeedream5p0ProUltra(_) => RouterProvider::Artcraft,
Self::ArtcraftQwenEdit2511Angles(_) => RouterProvider::Artcraft,
Self::ArtcraftFlux2LoraAngles(_) => RouterProvider::Artcraft,
Self::ArtcraftMidjourney7(_) => RouterProvider::Artcraft,
@@ -218,6 +222,9 @@ impl ImageGenerationRequest {
Self::ArtcraftSeedream5p0Pro(request) => {
Ok(ArtcraftSeedream5p0ProCostState::from_request(request).estimate_cost())
}
Self::ArtcraftSeedream5p0ProUltra(request) => {
Ok(ArtcraftSeedream5p0ProUltraCostState::from_request(request).estimate_cost())
}
Self::ArtcraftQwenEdit2511Angles(request) => {
Ok(ArtcraftQwenEdit2511AnglesCostState::from_request(request).estimate_cost())
}
@@ -317,6 +324,10 @@ impl ImageGenerationRequest {
let artcraft_client = client.get_artcraft_client_ref()?;
request.send(artcraft_client).await
}
Self::ArtcraftSeedream5p0ProUltra(request) => {
let artcraft_client = client.get_artcraft_client_ref()?;
request.send(artcraft_client).await
}
Self::ArtcraftQwenEdit2511Angles(request) => {
let artcraft_client = client.get_artcraft_client_ref()?;
request.send(artcraft_client).await
@@ -20,3 +20,4 @@ pub mod seedream_4;
pub mod seedream_4p5;
pub mod seedream_5_lite;
pub mod seedream_5p0_pro;
pub mod seedream_5p0_pro_ultra;
@@ -0,0 +1,17 @@
use enums::common::generation::common_image_model::CommonImageModel as CommonImageModelEnum;
use crate::errors::artcraft_router_error::ArtcraftRouterError;
use crate::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
use crate::generate::generate_image::image_generation_draft_or_request::ImageGenerationDraftOrRequest;
use crate::generate::generate_image::image_generation_request::ImageGenerationRequest;
use crate::generate::generate_image::providers::artcraft::build_common::build_artcraft_omni_image_request;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro_ultra::request::ArtcraftSeedream5p0ProUltraRequestState;
pub fn build_artcraft_seedream_5p0_pro_ultra(
builder: GenerateImageRequestBuilder,
) -> Result<ImageGenerationDraftOrRequest, ArtcraftRouterError> {
let request = build_artcraft_omni_image_request(builder, CommonImageModelEnum::Seedream5p0ProUltra)?;
Ok(ImageGenerationDraftOrRequest::Request(
ImageGenerationRequest::ArtcraftSeedream5p0ProUltra(ArtcraftSeedream5p0ProUltraRequestState { request }),
))
}
@@ -0,0 +1,208 @@
use enums::common::generation::common_resolution::CommonResolution as CommonResolutionEnum;
use crate::generate::generate_image::image_generation_cost_estimate::ImageGenerationCostEstimate;
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro_ultra::request::ArtcraftSeedream5p0ProUltraRequestState;
/// Customer-facing price per image at 1k resolution (the default), in USD cents.
const USD_CENTS_PER_IMAGE_1K: u64 = 18;
/// Customer-facing price per image at 2k resolution, in USD cents.
const USD_CENTS_PER_IMAGE_2K: u64 = 30;
/// Cost state for Artcraft Seedream 5.0 Pro Ultra. Pricing is per image and
/// resolution-dependent:
///
/// 1K (default) → 18¢, 2K → 30¢. 3K/4K fall back to 2K pricing; 0.5K and
/// legacy video resolutions (480p/720p/1080p) fall back to 1K.
#[derive(Clone, Debug)]
pub struct ArtcraftSeedream5p0ProUltraCostState {
pub resolution: Option<CommonResolutionEnum>,
pub num_images: u16,
}
impl ArtcraftSeedream5p0ProUltraCostState {
pub fn from_request(request: &ArtcraftSeedream5p0ProUltraRequestState) -> Self {
Self {
resolution: request.request.resolution,
num_images: request.request.image_batch_count.unwrap_or(1),
}
}
pub fn estimate_cost(&self) -> ImageGenerationCostEstimate {
let cost_per_image: u64 = cost_per_image_in_cents(self.resolution);
let cost_in_usd_cents = cost_per_image * self.num_images as u64;
ImageGenerationCostEstimate {
cost_in_credits: Some(cost_in_usd_cents),
cost_in_usd_cents: Some(cost_in_usd_cents),
is_free: false,
is_unlimited: false,
is_rate_limited: false,
has_watermark: false,
failures_are_refunded: None,
}
}
}
fn cost_per_image_in_cents(resolution: Option<CommonResolutionEnum>) -> u64 {
use CommonResolutionEnum as R;
match resolution {
None
| Some(R::HalfK)
| Some(R::OneK)
| Some(R::FourEightyP)
| Some(R::SevenTwentyP)
| Some(R::TenEightyP) => USD_CENTS_PER_IMAGE_1K,
Some(R::TwoK) | Some(R::ThreeK) | Some(R::FourK) => USD_CENTS_PER_IMAGE_2K,
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::api::router_image_model::RouterImageModel;
use crate::api::router_provider::RouterProvider;
use crate::api::router_resolution::RouterResolution;
use crate::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
use crate::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
fn estimate(resolution: Option<RouterResolution>, image_batch_count: u16) -> ImageGenerationCostEstimate {
let builder = GenerateImageRequestBuilder {
model: RouterImageModel::Seedream5p0ProUltra,
provider: RouterProvider::Artcraft,
prompt: Some("an anime girl riding a dinosaur".to_string()),
image_inputs: None,
resolution,
aspect_ratio: None,
quality: None,
image_batch_count: Some(image_batch_count),
horizontal_angle: None,
vertical_angle: None,
zoom: None,
request_mismatch_mitigation_strategy: RequestMismatchMitigationStrategy::ErrorOut,
generation_mode_mismatch_strategy: None,
idempotency_token: None,
};
builder.build2().unwrap().estimate_cost().unwrap()
}
fn cost_cents(resolution: Option<RouterResolution>, image_batch_count: u16) -> u64 {
estimate(resolution, image_batch_count).cost_in_usd_cents.unwrap()
}
// ── Default / 1K (18¢) ──
#[test]
fn default_resolution_one_image_is_18c() { assert_eq!(cost_cents(None, 1), 18); }
#[test]
fn one_k_one_image_is_18c() { assert_eq!(cost_cents(Some(RouterResolution::OneK), 1), 18); }
#[test]
fn one_k_four_images_is_72c() { assert_eq!(cost_cents(Some(RouterResolution::OneK), 4), 72); }
// ── 2K (30¢) ──
#[test]
fn two_k_one_image_is_30c() { assert_eq!(cost_cents(Some(RouterResolution::TwoK), 1), 30); }
#[test]
fn two_k_four_images_is_120c() { assert_eq!(cost_cents(Some(RouterResolution::TwoK), 4), 120); }
// ── Fallbacks ──
#[test]
fn four_k_falls_back_to_two_k_pricing() {
assert_eq!(
ArtcraftSeedream5p0ProUltraCostState {
resolution: Some(CommonResolutionEnum::FourK),
num_images: 1,
}.estimate_cost().cost_in_usd_cents,
Some(30),
);
}
#[test]
fn half_k_falls_back_to_one_k_pricing() {
assert_eq!(
ArtcraftSeedream5p0ProUltraCostState {
resolution: Some(CommonResolutionEnum::HalfK),
num_images: 1,
}.estimate_cost().cost_in_usd_cents,
Some(18),
);
}
// ── Credits track USD cents exactly (1 credit = 1 cent) ──
#[test]
fn credits_equal_usd_cents() {
for batch in 1..=4u16 {
let one_k = estimate(Some(RouterResolution::OneK), batch);
assert_eq!(one_k.cost_in_credits, one_k.cost_in_usd_cents, "1k batch {}", batch);
let two_k = estimate(Some(RouterResolution::TwoK), batch);
assert_eq!(two_k.cost_in_credits, two_k.cost_in_usd_cents, "2k batch {}", batch);
}
}
// ── The price must exceed the base Seedream 5.0 Pro price ──
#[test]
fn price_exceeds_base_seedream_5p0_pro_price() {
use crate::generate::generate_image::providers::artcraft::seedream_5p0_pro::cost::ArtcraftSeedream5p0ProCostState;
for resolution in [None, Some(CommonResolutionEnum::OneK), Some(CommonResolutionEnum::TwoK)] {
for num_images in [1u16, 4] {
let ultra_cents = ArtcraftSeedream5p0ProUltraCostState { resolution, num_images }
.estimate_cost().cost_in_usd_cents.unwrap();
let base_cents = ArtcraftSeedream5p0ProCostState { resolution, num_images }
.estimate_cost().cost_in_usd_cents.unwrap();
assert!(
ultra_cents > base_cents,
"ultra price {}¢ must exceed base price {}¢ ({:?} batch {})",
ultra_cents, base_cents, resolution, num_images,
);
}
}
}
// ── The Artcraft price must cover the Kinovi provider cost ──
#[test]
fn price_covers_kinovi_cost() {
use seedance2pro_client::generate::image::generate_seedream_5p0_pro::{
KinoviSeedream5p0ProBatchCount, KinoviSeedream5p0ProResolution,
};
use crate::generate::generate_image::providers::kinovi::seedream_5p0_pro::cost::KinoviSeedream5p0ProCostState;
let cases = [
(RouterResolution::OneK, KinoviSeedream5p0ProResolution::OneK, 1u16, KinoviSeedream5p0ProBatchCount::One),
(RouterResolution::OneK, KinoviSeedream5p0ProResolution::OneK, 4u16, KinoviSeedream5p0ProBatchCount::Four),
(RouterResolution::TwoK, KinoviSeedream5p0ProResolution::TwoK, 1u16, KinoviSeedream5p0ProBatchCount::One),
(RouterResolution::TwoK, KinoviSeedream5p0ProResolution::TwoK, 4u16, KinoviSeedream5p0ProBatchCount::Four),
];
for (router_res, kinovi_res, batch, kinovi_batch) in cases {
let artcraft_cents = cost_cents(Some(router_res), batch);
let kinovi_cents = KinoviSeedream5p0ProCostState {
resolution: kinovi_res,
batch_count: kinovi_batch,
}.estimate_cost().cost_in_usd_cents.unwrap();
assert!(
artcraft_cents > kinovi_cents,
"artcraft price {}¢ must exceed kinovi cost {}¢ ({:?} batch {})",
artcraft_cents, kinovi_cents, router_res, batch,
);
}
}
// ── Flags ──
#[test]
fn cost_flags_are_correct() {
let cost = ArtcraftSeedream5p0ProUltraCostState { resolution: None, num_images: 1 }.estimate_cost();
assert!(!cost.is_free);
assert!(!cost.is_unlimited);
assert!(!cost.is_rate_limited);
assert!(!cost.has_watermark);
assert_eq!(cost.failures_are_refunded, None);
}
}
@@ -0,0 +1,3 @@
pub mod build;
pub mod cost;
pub mod request;
@@ -0,0 +1,17 @@
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_image_cost_and_generate_request::OmniGenImageCostAndGenerateRequest;
use crate::client::router_artcraft_client::RouterArtcraftClient;
use crate::errors::artcraft_router_error::ArtcraftRouterError;
use crate::generate::generate_image::generate_image_response::GenerateImageResponse;
use crate::generate::generate_image::providers::artcraft::request_common::send_artcraft_omni_image_request;
#[derive(Clone, Debug)]
pub struct ArtcraftSeedream5p0ProUltraRequestState {
pub request: OmniGenImageCostAndGenerateRequest,
}
impl ArtcraftSeedream5p0ProUltraRequestState {
pub async fn send(&self, client: &RouterArtcraftClient) -> Result<GenerateImageResponse, ArtcraftRouterError> {
send_artcraft_omni_image_request(&self.request, client).await
}
}
@@ -0,0 +1,421 @@
use chrono::{DateTime, Utc};
use log::info;
use serde::Deserialize;
use wreq::Client;
use wreq_util::Emulation;
use crate::creds::seedance2pro_session::Seedance2ProSession;
use crate::error::seedance2pro_client_error::Seedance2ProClientError;
use crate::error::seedance2pro_error::Seedance2ProError;
use crate::error::seedance2pro_generic_api_error::Seedance2ProGenericApiError;
use crate::requests::kinovi_host::{KinoviHost, resolve_host};
use crate::utils::common_headers::FIREFOX_USER_AGENT;
use crate::utils::number_coercion::de_u64_int_or_float;
/// Page size the kinovi.ai billing dashboard uses.
pub const DEFAULT_BILLING_PAGE_LIMIT: u32 = 20;
// --- Args & response ---
pub struct GetBillingPaymentsHistoryArgs<'a> {
pub session: &'a Seedance2ProSession,
/// Page size. The web dashboard uses [`DEFAULT_BILLING_PAGE_LIMIT`].
pub limit: u32,
/// Zero-based row offset. Pagination is offset-based: advance by the
/// number of payments actually returned until a page comes back empty.
pub offset: u64,
/// Override the default host (kinovi.ai).
pub host_override: Option<KinoviHost>,
}
pub struct BillingPaymentsHistoryPage {
/// Payments on this page, newest first.
pub payments: Vec<BillingPayment>,
/// Cursor of the row after this page (`None` on the last page). The
/// dashboard paginates by limit/offset; this is informational.
pub maybe_next_cursor: Option<String>,
/// Total payment rows on the account, per the server.
pub total: u64,
}
// --- Public types ---
/// One payment on the Kinovi billing dashboard (a credit-package purchase).
#[derive(Debug, Clone)]
pub struct BillingPayment {
pub id: String,
/// Payment amount in US dollars (the API returns dollars: `2159`, `99.99`).
pub amount_usd: f64,
pub status: BillingPaymentStatus,
pub payment_type: BillingPaymentType,
/// Kinovi credits granted by this payment (e.g. 525,000).
pub credits_earned: u64,
/// e.g. "Credits Package"
pub product_name: String,
pub transaction_id: String,
pub payment_id: String,
pub product_id: String,
/// ISO 8601 creation timestamp (e.g. `"2026-07-13T20:40:49.405Z"`).
pub created_at: String,
/// Parsed `created_at`. `None` if the raw string could not be parsed.
pub created_at_utc: Option<DateTime<Utc>>,
pub updated_at: String,
}
/// Payment status. Only `PAID` has been observed; unrecognised values are
/// preserved in `Unknown` so future statuses (refunds, chargebacks, pending)
/// don't break deserialization.
#[derive(Debug, Clone, PartialEq)]
pub enum BillingPaymentStatus {
Paid,
Unknown(String),
}
impl BillingPaymentStatus {
fn from_str(s: &str) -> Self {
match s {
"PAID" => Self::Paid,
other => Self::Unknown(other.to_string()),
}
}
}
/// Payment type. Only `ONE_TIME` has been observed.
#[derive(Debug, Clone, PartialEq)]
pub enum BillingPaymentType {
OneTime,
Unknown(String),
}
impl BillingPaymentType {
fn from_str(s: &str) -> Self {
match s {
"ONE_TIME" => Self::OneTime,
other => Self::Unknown(other.to_string()),
}
}
}
// --- Implementation ---
pub async fn get_billing_payments_history(
args: GetBillingPaymentsHistoryArgs<'_>,
) -> Result<BillingPaymentsHistoryPage, Seedance2ProError> {
let host = resolve_host(args.host_override.as_ref());
let base_url = host.api_base_url();
let url = format!("{}/api/trpc/billing.getPayments", base_url);
info!("Fetching billing payments (limit: {}, offset: {})...", args.limit, args.offset);
let input_json = build_input_json(args.limit, args.offset);
let client = Client::builder()
.emulation(Emulation::Firefox143)
.build()
.map_err(|err| Seedance2ProClientError::WreqClientError(err))?;
let cookie = args.session.cookies.as_str();
let referer = format!("{}/app/dashboard/billing", base_url);
let request = client.get(&url)
.query(&[("batch", "1"), ("input", input_json.as_str())])
.header("User-Agent", FIREFOX_USER_AGENT)
.header("Accept", "*/*")
.header("Accept-Language", "en-US,en;q=0.9")
.header("Accept-Encoding", "gzip, deflate, br, zstd")
.header("Referer", &referer)
.header("content-type", "application/json")
.header("x-trpc-source", "client")
.header("Connection", "keep-alive")
.header("Cookie", cookie)
.header("Sec-Fetch-Dest", "empty")
.header("Sec-Fetch-Mode", "cors")
.header("Sec-Fetch-Site", "same-origin")
.header("Priority", "u=4")
.header("TE", "trailers")
.build()
.map_err(|err| Seedance2ProClientError::WreqClientError(err))?;
let response = client.execute(request)
.await
.map_err(|err| Seedance2ProGenericApiError::WreqError(err))?;
let status = response.status();
let response_body = response.text()
.await
.map_err(|err| Seedance2ProGenericApiError::WreqError(err))?;
info!("Billing payments response status: {}", status);
if !status.is_success() {
return Err(Seedance2ProGenericApiError::UncategorizedBadResponseWithStatusAndBody {
status_code: status,
body: response_body,
}.into());
}
parse_response_body(&response_body)
}
fn parse_response_body(response_body: &str) -> Result<BillingPaymentsHistoryPage, Seedance2ProError> {
let batch_response: Vec<BatchResponseItem> = serde_json::from_str(response_body)
.map_err(|err| Seedance2ProGenericApiError::SerdeResponseParseErrorWithBody(err, response_body.to_string()))?;
let json = batch_response
.into_iter()
.next()
.ok_or_else(|| Seedance2ProGenericApiError::UnexpectedResponseShape {
explanation: "Empty batch response array".to_string(),
raw_body: response_body.to_string(),
})?
.result
.data
.json;
let payments = json.payments
.into_iter()
.map(|p| {
let created_at_utc = DateTime::parse_from_rfc3339(&p.created_at)
.map(|dt| dt.with_timezone(&Utc))
.ok();
BillingPayment {
id: p.id,
amount_usd: p.amount,
status: BillingPaymentStatus::from_str(&p.payment_status),
payment_type: BillingPaymentType::from_str(&p.payment_type),
credits_earned: p.credits_earned,
product_name: p.product_name,
transaction_id: p.transaction_id,
payment_id: p.payment_id,
product_id: p.product_id,
created_at: p.created_at,
created_at_utc,
updated_at: p.updated_at,
}
})
.collect();
Ok(BillingPaymentsHistoryPage {
payments,
maybe_next_cursor: json.next_cursor,
total: json.total,
})
}
/// Builds the tRPC `input` JSON for the billing.getPayments endpoint.
fn build_input_json(limit: u32, offset: u64) -> String {
format!(r#"{{"0":{{"json":{{"limit":{limit},"offset":{offset}}}}}}}"#)
}
// --- Raw response types ---
#[derive(Deserialize, Debug)]
struct BatchResponseItem {
result: RawResult,
}
#[derive(Deserialize, Debug)]
struct RawResult {
data: RawData,
}
#[derive(Deserialize, Debug)]
struct RawData {
json: RawJson,
}
#[derive(Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct RawJson {
payments: Vec<RawPayment>,
#[serde(default)]
next_cursor: Option<String>,
#[serde(deserialize_with = "de_u64_int_or_float")]
total: u64,
}
#[derive(Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct RawPayment {
id: String,
amount: f64,
payment_status: String,
#[serde(default)]
transaction_id: String,
#[serde(default)]
payment_id: String,
#[serde(default)]
product_id: String,
#[serde(rename = "type")]
payment_type: String,
#[serde(deserialize_with = "de_u64_int_or_float")]
credits_earned: u64,
#[serde(default)]
product_name: String,
created_at: String,
#[serde(default)]
updated_at: String,
}
#[cfg(test)]
mod tests {
use super::*;
use crate::creds::seedance2pro_session::Seedance2ProSession;
use crate::test_utils::get_test_cookies::get_test_cookies;
use crate::test_utils::setup_test_logging::setup_test_logging;
use errors::AnyhowResult;
use log::LevelFilter;
mod offline_parsing {
use super::*;
/// Trimmed verbatim from a captured billing.getPayments response
/// (external/requests/sites/kinovi.ai/2026-07-13-billing/).
const CAPTURED_PAGE: &str = r#"[{"result":{"data":{"json":{"payments":[
{"id":"31722","amount":2159,"paymentStatus":"PAID","transactionId":"","paymentId":"","createdAt":"2026-07-13T20:40:49.405Z","updatedAt":"2026-07-13T20:40:49.405Z","productId":"","type":"ONE_TIME","creditsEarned":525000,"productName":"Credits Package"},
{"id":"11491","amount":99.99,"paymentStatus":"PAID","transactionId":"202603141400154949180","paymentId":"202603141400154949180","createdAt":"2026-03-14T14:00:15.782Z","updatedAt":"2026-03-14T14:01:02.630Z","productId":"","type":"ONE_TIME","creditsEarned":22000,"productName":"Credits Package"}
],"nextCursor":"29564","total":997}}}}]"#;
#[test]
fn parses_captured_page() {
let page = parse_response_body(CAPTURED_PAGE).expect("parse");
assert_eq!(page.payments.len(), 2);
assert_eq!(page.maybe_next_cursor.as_deref(), Some("29564"));
assert_eq!(page.total, 997);
let first = &page.payments[0];
assert_eq!(first.id, "31722");
assert_eq!(first.amount_usd, 2159.0);
assert_eq!(first.status, BillingPaymentStatus::Paid);
assert_eq!(first.payment_type, BillingPaymentType::OneTime);
assert_eq!(first.credits_earned, 525_000);
assert!(first.created_at_utc.is_some());
let second = &page.payments[1];
assert_eq!(second.amount_usd, 99.99);
assert_eq!(second.transaction_id, "202603141400154949180");
}
#[test]
fn last_page_has_no_next_cursor() {
let body = r#"[{"result":{"data":{"json":{"payments":[],"nextCursor":null,"total":997}}}}]"#;
let page = parse_response_body(body).expect("parse");
assert!(page.payments.is_empty());
assert_eq!(page.maybe_next_cursor, None);
}
#[test]
fn unknown_status_and_type_are_preserved() {
let body = r#"[{"result":{"data":{"json":{"payments":[
{"id":"1","amount":5,"paymentStatus":"REFUNDED","type":"SUBSCRIPTION","creditsEarned":0,"createdAt":"2026-07-13T20:40:49.405Z"}
],"nextCursor":null,"total":1}}}}]"#;
let page = parse_response_body(body).expect("parse");
assert_eq!(page.payments[0].status, BillingPaymentStatus::Unknown("REFUNDED".to_string()));
assert_eq!(page.payments[0].payment_type, BillingPaymentType::Unknown("SUBSCRIPTION".to_string()));
}
}
mod live {
use super::*;
#[tokio::test]
#[ignore] // manual test — requires real cookies
async fn live_fetch_first_payments_page() -> AnyhowResult<()> {
setup_test_logging(LevelFilter::Info);
let session = test_session()?;
let page = get_billing_payments_history(GetBillingPaymentsHistoryArgs {
session: &session,
limit: DEFAULT_BILLING_PAGE_LIMIT,
offset: 0,
host_override: None,
}).await?;
println!("payments: {}, total: {}, next_cursor: {:?}",
page.payments.len(), page.total, page.maybe_next_cursor);
for payment in &page.payments {
println!(" {} | {} | ${:.2} | {} credits | {:?} | {:?}",
payment.id, payment.created_at, payment.amount_usd,
payment.credits_earned, payment.status, payment.payment_type);
}
assert!(!page.payments.is_empty());
Ok(())
}
/// Pages through the ENTIRE payments history with throttling and prints
/// lifetime totals. ~1,000 rows / 50 pages as of 2026-07.
/// cargo test -p seedance2pro_client live_paginate_entire_payments_history -- --ignored --nocapture
#[tokio::test]
#[ignore] // manual test — requires real cookies; fetches every page
async fn live_paginate_entire_payments_history() -> AnyhowResult<()> {
setup_test_logging(LevelFilter::Info);
let session = test_session()?;
const PAGE_DELAY_MS: u64 = 250;
const MAX_PAGES: usize = 2_000; // runaway guard
let mut offset: u64 = 0;
let mut pages = 0usize;
let mut total_rows = 0u64;
let mut total_amount_usd = 0f64;
let mut total_credits = 0u64;
let mut last_reported_total = 0u64;
for _ in 0..MAX_PAGES {
let page = get_billing_payments_history(GetBillingPaymentsHistoryArgs {
session: &session,
limit: DEFAULT_BILLING_PAGE_LIMIT,
offset,
host_override: None,
}).await?;
pages += 1;
last_reported_total = page.total;
if page.payments.is_empty() {
break;
}
for payment in &page.payments {
total_rows += 1;
total_amount_usd += payment.amount_usd;
total_credits += payment.credits_earned;
}
println!("page {} (offset {}): {} payments, running total ${:.2}",
pages, offset, page.payments.len(), total_amount_usd);
// Advance by what was actually returned so a server-side limit cap
// can never skip rows.
offset += page.payments.len() as u64;
tokio::time::sleep(std::time::Duration::from_millis(PAGE_DELAY_MS)).await;
}
println!("── payments history totals ──");
println!("pages: {}", pages);
println!("payments: {} (server reports total {})", total_rows, last_reported_total);
println!("lifetime spend: ${:.2}", total_amount_usd);
println!("lifetime credits: {}", total_credits);
assert!(total_rows > 0);
Ok(())
}
fn test_session() -> AnyhowResult<Seedance2ProSession> {
let cookies = get_test_cookies()?;
Ok(Seedance2ProSession::from_cookies_string(cookies))
}
}
}
@@ -0,0 +1,415 @@
use chrono::{DateTime, Utc};
use log::info;
use serde::Deserialize;
use wreq::Client;
use wreq_util::Emulation;
use crate::creds::seedance2pro_session::Seedance2ProSession;
use crate::error::seedance2pro_client_error::Seedance2ProClientError;
use crate::error::seedance2pro_error::Seedance2ProError;
use crate::error::seedance2pro_generic_api_error::Seedance2ProGenericApiError;
use crate::requests::kinovi_host::{KinoviHost, resolve_host};
use crate::utils::common_headers::FIREFOX_USER_AGENT;
use crate::utils::number_coercion::{de_i64_int_or_float, de_u64_int_or_float};
/// Page size the kinovi.ai billing dashboard uses.
pub const DEFAULT_CREDITS_PAGE_LIMIT: u32 = 20;
// --- Args & response ---
pub struct GetCreditsHistoryArgs<'a> {
pub session: &'a Seedance2ProSession,
/// Page size. The web dashboard uses [`DEFAULT_CREDITS_PAGE_LIMIT`].
pub limit: u32,
/// Zero-based row offset. Pagination is offset-based: advance by the
/// number of entries actually returned until a page comes back empty.
/// NB: the ledger grows while you page (offsets shift under live traffic),
/// so dedupe by entry `id` when sweeping the full history.
pub offset: u64,
/// Override the default host (kinovi.ai).
pub host_override: Option<KinoviHost>,
}
pub struct CreditsHistoryPage {
/// Ledger entries on this page, newest first.
pub entries: Vec<CreditHistoryEntry>,
/// Total ledger rows on the account, per the server.
pub total: u64,
}
// --- Public types ---
/// One row of the Kinovi credits ledger: a signed credit delta attributed to
/// a generation order (consumption) or a grant (purchase, registration bonus).
#[derive(Debug, Clone)]
pub struct CreditHistoryEntry {
pub id: u64,
pub entry_type: CreditHistoryEntryType,
/// Signed credit delta: negative for consumption, positive for grants.
pub credit_delta: i64,
/// The generation order this entry bills, when applicable (`ord_...`).
/// `None` for grants (purchases, registration bonuses).
pub maybe_order_id: Option<String>,
/// e.g. "Seedance 2.0 Video", "Credit Package - xl (25000 credits)"
pub product_name: String,
/// ISO 8601 creation timestamp (e.g. `"2026-07-14T00:31:12.820Z"`).
pub created_at: String,
/// Parsed `created_at`. `None` if the raw string could not be parsed.
pub created_at_utc: Option<DateTime<Utc>>,
}
/// Ledger entry type. Observed values are mapped; anything new (refunds,
/// failure credits, expirations, ...) lands in `Unknown` so deserialization
/// never breaks.
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum CreditHistoryEntryType {
/// Credits consumed by a successful generation (negative delta).
ConsumptionSuccess,
/// Credits granted by a package purchase (positive delta).
PurchasedCredits,
/// Sign-up bonus credits (positive delta).
NewUserRegistration,
Unknown(String),
}
impl CreditHistoryEntryType {
pub fn as_str(&self) -> &str {
match self {
Self::ConsumptionSuccess => "CONSUMPTION_SUCCESS",
Self::PurchasedCredits => "PURCHASED_CREDITS",
Self::NewUserRegistration => "NEW_USER_REGISTRATION",
Self::Unknown(other) => other.as_str(),
}
}
fn from_str(s: &str) -> Self {
match s {
"CONSUMPTION_SUCCESS" => Self::ConsumptionSuccess,
"PURCHASED_CREDITS" => Self::PurchasedCredits,
"NEW_USER_REGISTRATION" => Self::NewUserRegistration,
other => Self::Unknown(other.to_string()),
}
}
}
// --- Implementation ---
pub async fn get_credits_history(
args: GetCreditsHistoryArgs<'_>,
) -> Result<CreditsHistoryPage, Seedance2ProError> {
let host = resolve_host(args.host_override.as_ref());
let base_url = host.api_base_url();
let url = format!("{}/api/trpc/credits.getCreditHistory", base_url);
info!("Fetching credits history (limit: {}, offset: {})...", args.limit, args.offset);
let input_json = build_input_json(args.limit, args.offset);
let client = Client::builder()
.emulation(Emulation::Firefox143)
.build()
.map_err(|err| Seedance2ProClientError::WreqClientError(err))?;
let cookie = args.session.cookies.as_str();
let referer = format!("{}/app/dashboard/billing", base_url);
let request = client.get(&url)
.query(&[("batch", "1"), ("input", input_json.as_str())])
.header("User-Agent", FIREFOX_USER_AGENT)
.header("Accept", "*/*")
.header("Accept-Language", "en-US,en;q=0.9")
.header("Accept-Encoding", "gzip, deflate, br, zstd")
.header("Referer", &referer)
.header("content-type", "application/json")
.header("x-trpc-source", "client")
.header("Connection", "keep-alive")
.header("Cookie", cookie)
.header("Sec-Fetch-Dest", "empty")
.header("Sec-Fetch-Mode", "cors")
.header("Sec-Fetch-Site", "same-origin")
.header("Priority", "u=4")
.header("TE", "trailers")
.build()
.map_err(|err| Seedance2ProClientError::WreqClientError(err))?;
let response = client.execute(request)
.await
.map_err(|err| Seedance2ProGenericApiError::WreqError(err))?;
let status = response.status();
let response_body = response.text()
.await
.map_err(|err| Seedance2ProGenericApiError::WreqError(err))?;
info!("Credits history response status: {}", status);
if !status.is_success() {
return Err(Seedance2ProGenericApiError::UncategorizedBadResponseWithStatusAndBody {
status_code: status,
body: response_body,
}.into());
}
parse_response_body(&response_body)
}
fn parse_response_body(response_body: &str) -> Result<CreditsHistoryPage, Seedance2ProError> {
let batch_response: Vec<BatchResponseItem> = serde_json::from_str(response_body)
.map_err(|err| Seedance2ProGenericApiError::SerdeResponseParseErrorWithBody(err, response_body.to_string()))?;
let json = batch_response
.into_iter()
.next()
.ok_or_else(|| Seedance2ProGenericApiError::UnexpectedResponseShape {
explanation: "Empty batch response array".to_string(),
raw_body: response_body.to_string(),
})?
.result
.data
.json;
let entries = json.credits
.into_iter()
.map(|entry| {
let created_at_utc = DateTime::parse_from_rfc3339(&entry.created_at)
.map(|dt| dt.with_timezone(&Utc))
.ok();
CreditHistoryEntry {
id: entry.id,
entry_type: CreditHistoryEntryType::from_str(&entry.entry_type),
credit_delta: entry.credit,
maybe_order_id: entry.order_id,
product_name: entry.product_name,
created_at: entry.created_at,
created_at_utc,
}
})
.collect();
Ok(CreditsHistoryPage {
entries,
total: json.total,
})
}
/// Builds the tRPC `input` JSON for the credits.getCreditHistory endpoint.
fn build_input_json(limit: u32, offset: u64) -> String {
format!(r#"{{"0":{{"json":{{"limit":{limit},"offset":{offset}}}}}}}"#)
}
// --- Raw response types ---
#[derive(Deserialize, Debug)]
struct BatchResponseItem {
result: RawResult,
}
#[derive(Deserialize, Debug)]
struct RawResult {
data: RawData,
}
#[derive(Deserialize, Debug)]
struct RawData {
json: RawJson,
}
#[derive(Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct RawJson {
credits: Vec<RawCreditEntry>,
#[serde(deserialize_with = "de_u64_int_or_float")]
total: u64,
}
#[derive(Deserialize, Debug)]
#[serde(rename_all = "camelCase")]
struct RawCreditEntry {
#[serde(deserialize_with = "de_u64_int_or_float")]
id: u64,
#[serde(rename = "type")]
entry_type: String,
#[serde(deserialize_with = "de_i64_int_or_float")]
credit: i64,
#[serde(default)]
order_id: Option<String>,
created_at: String,
#[serde(default)]
product_name: String,
}
#[cfg(test)]
mod tests {
use super::*;
use crate::creds::seedance2pro_session::Seedance2ProSession;
use crate::test_utils::get_test_cookies::get_test_cookies;
use crate::test_utils::setup_test_logging::setup_test_logging;
use errors::AnyhowResult;
use log::LevelFilter;
mod offline_parsing {
use super::*;
/// Trimmed verbatim from a captured credits.getCreditHistory response
/// (external/requests/sites/kinovi.ai/2026-07-13-billing/).
const CAPTURED_PAGE: &str = r#"[{"result":{"data":{"json":{"credits":[
{"id":2750659,"type":"CONSUMPTION_SUCCESS","credit":-454,"orderId":"ord_mff2cjk2feczmzqweizcdal2","createdAt":"2026-07-14T00:31:12.820Z","productName":"Seedance 2.0 Video"},
{"id":591799,"type":"PURCHASED_CREDITS","credit":25000,"orderId":null,"createdAt":"2026-02-19T08:16:58.524Z","productName":"Credit Package - xl (25000 credits)"},
{"id":563856,"type":"NEW_USER_REGISTRATION","credit":200,"orderId":null,"createdAt":"2026-02-18T01:02:52.747Z","productName":"unknown"}
],"total":304617}}}}]"#;
#[test]
fn parses_captured_page() {
let page = parse_response_body(CAPTURED_PAGE).expect("parse");
assert_eq!(page.entries.len(), 3);
assert_eq!(page.total, 304_617);
let consumption = &page.entries[0];
assert_eq!(consumption.id, 2_750_659);
assert_eq!(consumption.entry_type, CreditHistoryEntryType::ConsumptionSuccess);
assert_eq!(consumption.credit_delta, -454);
assert_eq!(consumption.maybe_order_id.as_deref(), Some("ord_mff2cjk2feczmzqweizcdal2"));
assert!(consumption.created_at_utc.is_some());
let purchase = &page.entries[1];
assert_eq!(purchase.entry_type, CreditHistoryEntryType::PurchasedCredits);
assert_eq!(purchase.credit_delta, 25_000);
assert_eq!(purchase.maybe_order_id, None);
let registration = &page.entries[2];
assert_eq!(registration.entry_type, CreditHistoryEntryType::NewUserRegistration);
assert_eq!(registration.credit_delta, 200);
}
#[test]
fn unknown_entry_type_is_preserved() {
let body = r#"[{"result":{"data":{"json":{"credits":[
{"id":1,"type":"CONSUMPTION_REFUND","credit":454,"orderId":"ord_x","createdAt":"2026-07-14T00:31:12.820Z","productName":"Seedance 2.0 Video"}
],"total":1}}}}]"#;
let page = parse_response_body(body).expect("parse");
assert_eq!(
page.entries[0].entry_type,
CreditHistoryEntryType::Unknown("CONSUMPTION_REFUND".to_string()),
);
assert_eq!(page.entries[0].credit_delta, 454);
}
#[test]
fn entry_type_round_trips_as_str() {
for raw in ["CONSUMPTION_SUCCESS", "PURCHASED_CREDITS", "NEW_USER_REGISTRATION", "SOMETHING_NEW"] {
assert_eq!(CreditHistoryEntryType::from_str(raw).as_str(), raw);
}
}
}
mod live {
use super::*;
use std::collections::HashMap;
use std::collections::HashSet;
#[tokio::test]
#[ignore] // manual test — requires real cookies
async fn live_fetch_first_credits_page() -> AnyhowResult<()> {
setup_test_logging(LevelFilter::Info);
let session = test_session()?;
let page = get_credits_history(GetCreditsHistoryArgs {
session: &session,
limit: DEFAULT_CREDITS_PAGE_LIMIT,
offset: 0,
host_override: None,
}).await?;
println!("entries: {}, total: {}", page.entries.len(), page.total);
for entry in &page.entries {
println!(" {} | {} | {:?} | {} | {:?}",
entry.id, entry.created_at, entry.entry_type, entry.credit_delta,
entry.maybe_order_id);
}
assert!(!page.entries.is_empty());
Ok(())
}
/// Pages through the ENTIRE credits ledger with throttling and prints
/// per-type tallies (this is where any refund-like entry types will
/// surface). ~300k rows as of 2026-07 — expect a long run; the ledger
/// also grows while paging, so entries are deduped by id.
/// cargo test -p seedance2pro_client live_paginate_entire_credits_history -- --ignored --nocapture
#[tokio::test]
#[ignore] // manual test — requires real cookies; fetches every page
async fn live_paginate_entire_credits_history() -> AnyhowResult<()> {
setup_test_logging(LevelFilter::Info);
let session = test_session()?;
const PAGE_LIMIT: u32 = 100;
const PAGE_DELAY_MS: u64 = 250;
const MAX_PAGES: usize = 50_000; // runaway guard
let mut offset: u64 = 0;
let mut pages = 0usize;
let mut seen_ids: HashSet<u64> = HashSet::new();
let mut type_counts: HashMap<CreditHistoryEntryType, (u64, i64)> = HashMap::new();
let mut last_reported_total = 0u64;
for _ in 0..MAX_PAGES {
let page = get_credits_history(GetCreditsHistoryArgs {
session: &session,
limit: PAGE_LIMIT,
offset,
host_override: None,
}).await?;
pages += 1;
last_reported_total = page.total;
if page.entries.is_empty() {
break;
}
for entry in &page.entries {
if !seen_ids.insert(entry.id) {
continue; // offset shifted under live traffic; already counted
}
let tally = type_counts.entry(entry.entry_type.clone()).or_insert((0, 0));
tally.0 += 1;
tally.1 += entry.credit_delta;
}
if pages % 25 == 0 {
println!("page {} (offset {}): {} unique entries so far (server total {})",
pages, offset, seen_ids.len(), page.total);
}
// Advance by what was actually returned so a server-side limit cap
// can never skip rows.
offset += page.entries.len() as u64;
tokio::time::sleep(std::time::Duration::from_millis(PAGE_DELAY_MS)).await;
}
println!("── credits ledger totals ──");
println!("pages: {}", pages);
println!("unique entries: {} (server reports total {})", seen_ids.len(), last_reported_total);
let mut types: Vec<_> = type_counts.iter().collect();
types.sort_by_key(|(_, (count, _))| std::cmp::Reverse(*count));
for (entry_type, (count, delta_sum)) in types {
println!(" {:<28} n={:<8} credit_sum={}", entry_type.as_str(), count, delta_sum);
}
assert!(!seen_ids.is_empty());
Ok(())
}
fn test_session() -> AnyhowResult<Seedance2ProSession> {
let cookies = get_test_cookies()?;
Ok(Seedance2ProSession::from_cookies_string(cookies))
}
}
}
@@ -0,0 +1,2 @@
pub mod get_billing_payments_history;
pub mod get_credits_history;
@@ -2,6 +2,7 @@
pub (crate) mod test_utils;
// User API
pub mod billing;
pub mod cost;
pub mod creds;
pub mod error;
@@ -19,7 +19,9 @@ pub fn setup_test_logging(level: LevelFilter) {
println!("Log level: {:?}", env::var(RUST_LOG));
Builder::new()
// try_init: several #[ignore] live tests may run in one process, and only
// the first can install the logger.
let _ = Builder::new()
.is_test(true)
.format(|buf, record| {
writeln!(buf,
@@ -31,7 +33,7 @@ pub fn setup_test_logging(level: LevelFilter) {
})
.filter(None, level)
.filter_level(level)
.init();
.try_init();
trace!("Test trace log");
debug!("Test debug log");
@@ -28,6 +28,20 @@ fn coerce_to_u32(number: &Number) -> u32 {
coerce_to_u64(number).min(u32::MAX as u64) as u32
}
/// Coerce a parsed JSON number to `i64`. Exact integers pass through; floats
/// are rounded; non-finite values become `0`.
fn coerce_to_i64(number: &Number) -> i64 {
if let Some(signed) = number.as_i64() {
return signed;
}
if let Some(float) = number.as_f64() {
if float.is_finite() {
return float.round() as i64;
}
}
0
}
/// `#[serde(deserialize_with = "...")]` for a required `u64` field that may
/// arrive as an integer or a float.
pub fn de_u64_int_or_float<'de, D>(deserializer: D) -> Result<u64, D::Error>
@@ -38,6 +52,16 @@ where
Ok(coerce_to_u64(&number))
}
/// `#[serde(deserialize_with = "...")]` for a required `i64` field that may
/// arrive as an integer or a float (e.g. signed credit deltas).
pub fn de_i64_int_or_float<'de, D>(deserializer: D) -> Result<i64, D::Error>
where
D: Deserializer<'de>,
{
let number = Number::deserialize(deserializer)?;
Ok(coerce_to_i64(&number))
}
/// `#[serde(deserialize_with = "...")]` for an optional `u64` field (handles
/// `null` / missing as `None`).
pub fn de_opt_u64_int_or_float<'de, D>(deserializer: D) -> Result<Option<u64>, D::Error>
@@ -90,6 +114,12 @@ mod tests {
value: Option<u32>,
}
#[derive(Deserialize)]
struct I64Holder {
#[serde(deserialize_with = "de_i64_int_or_float")]
value: i64,
}
fn parse_u32(json: &str) -> u32 {
serde_json::from_str::<U32Holder>(json).unwrap().value
}
@@ -98,6 +128,10 @@ mod tests {
serde_json::from_str::<U64Holder>(json).unwrap().value
}
fn parse_i64(json: &str) -> i64 {
serde_json::from_str::<I64Holder>(json).unwrap().value
}
#[test]
fn u32_accepts_integer() {
assert_eq!(parse_u32(r#"{"value": 568}"#), 568);
@@ -126,6 +160,13 @@ mod tests {
assert_eq!(parse_u64(r#"{"value": -5.5}"#), 0);
}
#[test]
fn i64_accepts_negative_integers_and_floats() {
assert_eq!(parse_i64(r#"{"value": -454}"#), -454);
assert_eq!(parse_i64(r#"{"value": -454.6}"#), -455);
assert_eq!(parse_i64(r#"{"value": 25000}"#), 25000);
}
#[test]
fn optional_handles_null_missing_and_values() {
assert_eq!(serde_json::from_str::<OptU32Holder>(r#"{"value": null}"#).unwrap().value, None);
@@ -7,6 +7,10 @@ use super::subcommands;
/// All canonical subcommand names for this module.
/// Used by the underscore-insensitive arg normalizer.
pub const SUBCOMMAND_NAMES: &[&str] = &[
"account_info",
"audit_credits",
"audit_orders",
"audit_payments",
"failed_job_histogram",
"find_job",
"generate_video",
@@ -15,6 +19,21 @@ pub const SUBCOMMAND_NAMES: &[&str] = &[
#[derive(Subcommand)]
#[command(rename_all = "snake_case")]
pub enum Seedance2proCommand {
/// Print the account's current credit balances as one CSV line
AccountInfo(subcommands::account_info::AccountInfoArgs),
/// Dump the account's credits ledger (credits.getCreditHistory) to CSV
/// back to a start date, for refund-entry auditing
AuditCredits(subcommands::audit_credits::AuditCreditsArgs),
/// Dump Kinovi's per-order billing records (totalCredits) to CSV for a
/// date window, to audit whether failed orders are charged
AuditOrders(subcommands::audit_orders::AuditOrdersArgs),
/// Dump the account's billing payments history (credit-package
/// purchases) to CSV, for reconciling refills against invoices
AuditPayments(subcommands::audit_payments::AuditPaymentsArgs),
/// Find a job by its order ID across all pages
FindJob(subcommands::find_job::FindJobArgs),
@@ -32,6 +51,10 @@ pub async fn run(command: Seedance2proCommand) -> anyhow::Result<()> {
let state = Seedance2ProState { cookies };
match command {
Seedance2proCommand::AccountInfo(args) => subcommands::account_info::run(&state, args).await,
Seedance2proCommand::AuditCredits(args) => subcommands::audit_credits::run(&state, args).await,
Seedance2proCommand::AuditOrders(args) => subcommands::audit_orders::run(&state, args).await,
Seedance2proCommand::AuditPayments(args) => subcommands::audit_payments::run(&state, args).await,
Seedance2proCommand::FindJob(args) => subcommands::find_job::run(&state, args).await,
Seedance2proCommand::FailedJobHistogram => subcommands::failed_job_histogram::run(&state).await,
Seedance2proCommand::GenerateVideo(args) => subcommands::generate_video::run(&state, args).await,
@@ -0,0 +1,46 @@
use anyhow::anyhow;
use clap::Args;
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use seedance2pro_client::requests::get_user_auth_details::get_user_auth_details::{
get_user_auth_details, GetUserAuthDetailsArgs,
};
use super::super::state::Seedance2ProState;
/// Print the account's current credit balances (auth.user endpoint) as a
/// single CSV line: account,email,credits,available_credits.
#[derive(Args)]
pub struct AccountInfoArgs {
/// Read session cookies from this env var instead of SEEDANCE2PRO_COOKIES.
#[arg(long)]
pub cookies_env: Option<String>,
/// Account label for the output line. Defaults to the cookies env var name.
#[arg(long)]
pub account: Option<String>,
}
pub async fn run(state: &Seedance2ProState, args: AccountInfoArgs) -> anyhow::Result<()> {
let (cookies, account_label) = match &args.cookies_env {
Some(var) => (
easyenv::get_env_string_required(var)
.map_err(|err| anyhow!("Missing {} env var: {:?}", var, err))?,
args.account.clone().unwrap_or_else(|| var.clone()),
),
None => (
state.cookies.clone(),
args.account.clone().unwrap_or_else(|| "SEEDANCE2PRO_COOKIES".to_string()),
),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
let details = get_user_auth_details(GetUserAuthDetailsArgs {
session: &session,
host_override: None,
}).await
.map_err(|err| anyhow!("Error fetching auth details: {:?}", err))?;
println!("{},{},{},{}",
account_label, details.email, details.credits, details.available_credits);
Ok(())
}
@@ -0,0 +1,167 @@
use std::collections::HashMap;
use std::fs::OpenOptions;
use std::io::Write;
use std::time::Duration;
use anyhow::anyhow;
use clap::Args;
use log::info;
use seedance2pro_client::billing::get_credits_history::{
get_credits_history, CreditHistoryEntry, GetCreditsHistoryArgs,
};
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use super::super::state::Seedance2ProState;
/// Dump the account's credits ledger (credits.getCreditHistory) to CSV,
/// newest-first, back to a start date. The ledger is offset-paginated and
/// grows while paging (new rows push older rows to higher offsets), which
/// causes re-reads rather than gaps - consumers should dedupe by entry id.
#[derive(Args)]
pub struct AuditCreditsArgs {
/// Output CSV path. Appends if the file exists.
#[arg(long)]
pub out: String,
/// Stop paging once a whole page is older than this UTC date (YYYY-MM-DD).
#[arg(long)]
pub start_date: Option<String>,
/// Page size to request (the server may cap it; paging advances by rows
/// actually returned, so a cap cannot skip rows).
#[arg(long, default_value_t = 100)]
pub limit: u32,
/// Maximum pages to fetch this run.
#[arg(long, default_value_t = 10_000)]
pub max_pages: usize,
/// Resume from a row offset (e.g. after an interrupted run).
#[arg(long, default_value_t = 0)]
pub offset: u64,
/// Delay between page fetches, in milliseconds.
#[arg(long, default_value_t = 250)]
pub delay_ms: u64,
/// Read session cookies from this env var instead of SEEDANCE2PRO_COOKIES.
#[arg(long)]
pub cookies_env: Option<String>,
/// Account label written into each CSV row. Defaults to the cookies env
/// var name.
#[arg(long)]
pub account: Option<String>,
}
pub async fn run(state: &Seedance2ProState, args: AuditCreditsArgs) -> anyhow::Result<()> {
let (cookies, account_label) = match &args.cookies_env {
Some(var) => (
easyenv::get_env_string_required(var)
.map_err(|err| anyhow!("Missing {} env var: {:?}", var, err))?,
args.account.clone().unwrap_or_else(|| var.clone()),
),
None => (
state.cookies.clone(),
args.account.clone().unwrap_or_else(|| "SEEDANCE2PRO_COOKIES".to_string()),
),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
let file_is_new = std::fs::metadata(&args.out).map(|m| m.len() == 0).unwrap_or(true);
let mut out = OpenOptions::new().create(true).append(true).open(&args.out)
.map_err(|err| anyhow!("Cannot open {}: {}", args.out, err))?;
if file_is_new {
writeln!(out, "account,entry_id,created_at,entry_type,credit_delta,order_id,product_name")?;
}
let mut offset = args.offset;
let mut rows = 0u64;
let mut type_tallies: HashMap<String, (u64, i64)> = HashMap::new();
let mut last_reported_total = 0u64;
for page_number in 1..=args.max_pages {
let page = get_credits_history(GetCreditsHistoryArgs {
session: &session,
limit: args.limit,
offset,
host_override: None,
}).await
.map_err(|err| anyhow!(
"Error fetching credits page {} (resume with --offset {}): {:?}",
page_number, offset, err))?;
last_reported_total = page.total;
if page.entries.is_empty() {
info!("Page {} returned no entries; reached the end of the ledger.", page_number);
break;
}
let newest_day = page.entries.first().map(|e| day_of(e)).unwrap_or_default();
let oldest_day = page.entries.last().map(|e| day_of(e)).unwrap_or_default();
for entry in &page.entries {
write_entry_row(&mut out, &account_label, entry)?;
rows += 1;
let tally = type_tallies.entry(entry.entry_type.as_str().to_string()).or_insert((0, 0));
tally.0 += 1;
tally.1 += entry.credit_delta;
}
out.flush()?;
if page_number % 25 == 0 || page_number == 1 {
info!(
"Page {} (offset {}): {} entries ({} .. {}), written {} (server total {})",
page_number, offset, page.entries.len(), newest_day, oldest_day, rows, page.total,
);
}
// Newest-first: once a whole page predates the window, everything
// after it does too.
if args.start_date.as_deref().is_some_and(|start| oldest_day.as_str() < start) {
info!("Reached entries older than {}; done.", args.start_date.as_deref().unwrap());
break;
}
offset += page.entries.len() as u64;
tokio::time::sleep(Duration::from_millis(args.delay_ms)).await;
}
println!("── audit_credits summary [{}] ──", account_label);
println!("entries written: {} (server reports ledger total {}; duplicates possible", rows, last_reported_total);
println!(" under live growth - dedupe by entry_id downstream)");
let mut tallies: Vec<_> = type_tallies.iter().collect();
tallies.sort_by_key(|(_, (count, _))| std::cmp::Reverse(*count));
for (entry_type, (count, delta_sum)) in tallies {
println!(" {:<28} n={:<9} credit_sum={}", entry_type, count, delta_sum);
}
Ok(())
}
/// UTC day prefix ("YYYY-MM-DD"); ISO strings compare lexicographically.
fn day_of(entry: &CreditHistoryEntry) -> String {
entry.created_at.get(..10).unwrap_or("").to_string()
}
fn write_entry_row(out: &mut impl Write, account: &str, entry: &CreditHistoryEntry) -> anyhow::Result<()> {
writeln!(
out,
"{},{},{},{},{},{},{}",
account,
entry.id,
entry.created_at,
entry.entry_type.as_str(),
entry.credit_delta,
entry.maybe_order_id.as_deref().unwrap_or(""),
csv_quote(&entry.product_name),
)?;
Ok(())
}
/// Minimal CSV quoting for the one free-text field.
fn csv_quote(field: &str) -> String {
if field.is_empty() {
return String::new();
}
format!("\"{}\"", field.replace('"', "\"\"").replace('\n', " "))
}
@@ -0,0 +1,243 @@
use std::fs::OpenOptions;
use std::io::Write;
use std::time::Duration;
use anyhow::anyhow;
use clap::Args;
use log::{info, warn};
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use seedance2pro_client::requests::poll_orders::poll_orders::{
poll_orders, OrderStatus, PollOrdersArgs, TaskStatus,
};
use super::super::state::Seedance2ProState;
/// Dump Kinovi's own per-order billing records (`totalCredits`) to CSV for a
/// date window, by paging the getOrders API newest-to-oldest. Used to audit
/// whether failed orders carry charges.
#[derive(Args)]
pub struct AuditOrdersArgs {
/// Output CSV path. Appends if the file exists (resume-friendly).
#[arg(long)]
pub out: String,
/// Stop paging once a whole page is older than this UTC date (YYYY-MM-DD).
/// Without it, paging stops at --max_pages.
#[arg(long)]
pub start_date: Option<String>,
/// Skip (do not write) orders on days after this UTC date (YYYY-MM-DD,
/// inclusive). Pages still have to be fetched to reach older orders.
#[arg(long)]
pub end_date: Option<String>,
/// Maximum pages to fetch this run (30 orders per page).
#[arg(long, default_value_t = 200)]
pub max_pages: usize,
/// Resume from a cursor logged by a previous run.
#[arg(long)]
pub cursor: Option<u64>,
/// Delay between page fetches, in milliseconds.
#[arg(long, default_value_t = 250)]
pub delay_ms: u64,
/// Read session cookies from this env var instead of SEEDANCE2PRO_COOKIES
/// (e.g. SEEDANCE2PRO_VOLCENGINE_COOKIES / SEEDANCE2PRO_BYTEPLUS_COOKIES /
/// SEEDANCE2PRO_BYTEPLUS_ULTRA_COOKIES to audit each Kinovi account).
#[arg(long)]
pub cookies_env: Option<String>,
/// Account label written into each CSV row. Defaults to the cookies env
/// var name (or "SEEDANCE2PRO_COOKIES").
#[arg(long)]
pub account: Option<String>,
}
pub async fn run(state: &Seedance2ProState, args: AuditOrdersArgs) -> anyhow::Result<()> {
let (cookies, account_label) = match &args.cookies_env {
Some(var) => (
easyenv::get_env_string_required(var)
.map_err(|err| anyhow!("Missing {} env var: {:?}", var, err))?,
args.account.clone().unwrap_or_else(|| var.clone()),
),
None => (
state.cookies.clone(),
args.account.clone().unwrap_or_else(|| "SEEDANCE2PRO_COOKIES".to_string()),
),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
let file_is_new = std::fs::metadata(&args.out).map(|m| m.len() == 0).unwrap_or(true);
let mut out = OpenOptions::new().create(true).append(true).open(&args.out)
.map_err(|err| anyhow!("Cannot open {}: {}", args.out, err))?;
if file_is_new {
writeln!(out, "account,order_id,created_at,task_status,media_type,total_credits,result_count,fail_reason")?;
}
let mut cursor = args.cursor;
let mut stats = AuditStats::default();
for page in 1..=args.max_pages {
let result = poll_orders(PollOrdersArgs {
session: &session,
cursor,
host_override: None,
}).await
.map_err(|err| anyhow!(
"Error polling page {} (resume with --cursor {:?}): {:?}", page, cursor, err))?;
if result.orders.is_empty() {
info!("Page {} returned no orders; stopping.", page);
break;
}
let newest = order_day(result.orders.first().unwrap());
let oldest = order_day(result.orders.last().unwrap());
for order in &result.orders {
let day = order_day(order);
if args.end_date.as_deref().is_some_and(|end| day > end) {
stats.skipped_after_window += 1;
continue;
}
if args.start_date.as_deref().is_some_and(|start| day < start) {
stats.skipped_before_window += 1;
continue;
}
write_order_row(&mut out, &account_label, order)?;
stats.record(order);
}
out.flush()?;
info!(
"Page {}: {} orders ({} .. {}), written so far: {}, next cursor: {:?}",
page, result.orders.len(), newest, oldest, stats.written, result.next_cursor,
);
// The API pages newest-to-oldest: once the whole page predates the
// window, everything after it will too.
if args.start_date.as_deref().is_some_and(|start| oldest < start) {
info!("Reached orders older than {}; done.", args.start_date.as_deref().unwrap());
cursor = result.next_cursor;
break;
}
cursor = result.next_cursor;
if cursor.is_none() {
info!("No further pages; reached the end of the account's history.");
break;
}
tokio::time::sleep(Duration::from_millis(args.delay_ms)).await;
}
stats.print_summary(&account_label, cursor);
Ok(())
}
#[derive(Default)]
struct AuditStats {
written: u64,
skipped_after_window: u64,
skipped_before_window: u64,
completed: u64,
completed_credits: u64,
completed_missing_credits: u64,
failed: u64,
failed_credits: u64,
failed_with_nonzero_credits: u64,
in_flight: u64,
in_flight_credits: u64,
}
impl AuditStats {
fn record(&mut self, order: &OrderStatus) {
self.written += 1;
let credits = u64::from(order.total_credits.unwrap_or(0));
match order.task_status {
TaskStatus::Completed => {
self.completed += 1;
self.completed_credits += credits;
if order.total_credits.is_none() {
self.completed_missing_credits += 1;
}
}
TaskStatus::Failed => {
self.failed += 1;
self.failed_credits += credits;
if credits > 0 {
self.failed_with_nonzero_credits += 1;
}
}
_ => {
self.in_flight += 1;
self.in_flight_credits += credits;
}
}
}
fn print_summary(&self, account: &str, resume_cursor: Option<u64>) {
// Final results are intentionally println: they ARE the tool's output.
println!("── audit_orders summary [{}] ──", account);
println!("orders written: {}", self.written);
println!(" completed: {} ({} credits; {} missing totalCredits)",
self.completed, self.completed_credits, self.completed_missing_credits);
println!(" failed: {} ({} credits; {} orders with credits > 0)",
self.failed, self.failed_credits, self.failed_with_nonzero_credits);
println!(" pending/processing: {} ({} credits)", self.in_flight, self.in_flight_credits);
println!("skipped (newer than window): {}", self.skipped_after_window);
println!("skipped (older than window): {}", self.skipped_before_window);
match resume_cursor {
Some(c) => println!("resume cursor: {}", c),
None => println!("resume cursor: (none - history exhausted or window done)"),
}
if self.failed_with_nonzero_credits > 0 {
warn!(
"{} FAILED orders carry non-zero totalCredits ({} credits total) — \
Kinovi's own records say failures were charged.",
self.failed_with_nonzero_credits, self.failed_credits,
);
}
}
}
/// UTC day prefix ("YYYY-MM-DD") of the order's ISO 8601 created_at.
/// ISO strings compare lexicographically, so no date parsing is needed.
fn order_day(order: &OrderStatus) -> &str {
order.created_at.get(..10).unwrap_or("")
}
fn write_order_row(out: &mut impl Write, account: &str, order: &OrderStatus) -> anyhow::Result<()> {
let media_type = order.media_type.as_ref()
.map(|m| format!("{:?}", m))
.unwrap_or_default();
let total_credits = order.total_credits
.map(|c| c.to_string())
.unwrap_or_default();
let fail_reason = order.fail_reason.as_ref()
.map(|fr| fr.reason.as_str())
.unwrap_or("");
writeln!(
out,
"{},{},{},{:?},{},{},{},{}",
account,
order.order_id,
order.created_at,
order.task_status,
media_type,
total_credits,
order.results.len(),
csv_quote(fail_reason),
)?;
Ok(())
}
/// Minimal CSV quoting for the one free-text field.
fn csv_quote(field: &str) -> String {
if field.is_empty() {
return String::new();
}
format!("\"{}\"", field.replace('"', "\"\"").replace('\n', " "))
}
@@ -0,0 +1,131 @@
use std::fs::OpenOptions;
use std::io::Write;
use std::time::Duration;
use anyhow::anyhow;
use clap::Args;
use log::info;
use seedance2pro_client::billing::get_billing_payments_history::{
get_billing_payments_history, BillingPayment, GetBillingPaymentsHistoryArgs,
DEFAULT_BILLING_PAGE_LIMIT,
};
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use super::super::state::Seedance2ProState;
/// Dump the account's full billing payments history (credit-package
/// purchases) to CSV, for reconciling refill charges against invoices.
#[derive(Args)]
pub struct AuditPaymentsArgs {
/// Output CSV path. Appends if the file exists.
#[arg(long)]
pub out: String,
/// Maximum pages to fetch (20 payments per page).
#[arg(long, default_value_t = 500)]
pub max_pages: usize,
/// Delay between page fetches, in milliseconds.
#[arg(long, default_value_t = 250)]
pub delay_ms: u64,
/// Read session cookies from this env var instead of SEEDANCE2PRO_COOKIES.
#[arg(long)]
pub cookies_env: Option<String>,
/// Account label written into each CSV row. Defaults to the cookies env
/// var name.
#[arg(long)]
pub account: Option<String>,
}
pub async fn run(state: &Seedance2ProState, args: AuditPaymentsArgs) -> anyhow::Result<()> {
let (cookies, account_label) = match &args.cookies_env {
Some(var) => (
easyenv::get_env_string_required(var)
.map_err(|err| anyhow!("Missing {} env var: {:?}", var, err))?,
args.account.clone().unwrap_or_else(|| var.clone()),
),
None => (
state.cookies.clone(),
args.account.clone().unwrap_or_else(|| "SEEDANCE2PRO_COOKIES".to_string()),
),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
let file_is_new = std::fs::metadata(&args.out).map(|m| m.len() == 0).unwrap_or(true);
let mut out = OpenOptions::new().create(true).append(true).open(&args.out)
.map_err(|err| anyhow!("Cannot open {}: {}", args.out, err))?;
if file_is_new {
writeln!(out, "account,payment_id,created_at,amount_usd,credits_earned,status,payment_type,product_name,transaction_id")?;
}
let mut offset: u64 = 0;
let mut rows = 0u64;
let mut total_amount = 0f64;
let mut total_credits = 0u64;
let mut last_reported_total = 0u64;
for page_number in 1..=args.max_pages {
let page = get_billing_payments_history(GetBillingPaymentsHistoryArgs {
session: &session,
limit: DEFAULT_BILLING_PAGE_LIMIT,
offset,
host_override: None,
}).await
.map_err(|err| anyhow!(
"Error fetching payments page {} (offset {}): {:?}", page_number, offset, err))?;
last_reported_total = page.total;
if page.payments.is_empty() {
break;
}
for payment in &page.payments {
write_payment_row(&mut out, &account_label, payment)?;
rows += 1;
total_amount += payment.amount_usd;
total_credits += payment.credits_earned;
}
out.flush()?;
info!(
"Page {} (offset {}): {} payments; running ${:.2} / {} credits (server total {})",
page_number, offset, page.payments.len(), total_amount, total_credits, page.total,
);
offset += page.payments.len() as u64;
tokio::time::sleep(Duration::from_millis(args.delay_ms)).await;
}
println!("── audit_payments summary [{}] ──", account_label);
println!("payments written: {} (server reports total {})", rows, last_reported_total);
println!("lifetime charged: ${:.2}", total_amount);
println!("lifetime credits: {}", total_credits);
Ok(())
}
fn write_payment_row(out: &mut impl Write, account: &str, payment: &BillingPayment) -> anyhow::Result<()> {
writeln!(
out,
"{},{},{},{},{},{:?},{:?},{},{}",
account,
payment.id,
payment.created_at,
payment.amount_usd,
payment.credits_earned,
payment.status,
payment.payment_type,
csv_quote(&payment.product_name),
payment.transaction_id,
)?;
Ok(())
}
/// Minimal CSV quoting for the one free-text field.
fn csv_quote(field: &str) -> String {
if field.is_empty() {
return String::new();
}
format!("\"{}\"", field.replace('"', "\"\"").replace('\n', " "))
}
@@ -1,3 +1,7 @@
pub mod account_info;
pub mod audit_credits;
pub mod audit_orders;
pub mod audit_payments;
pub mod failed_job_histogram;
pub mod find_job;
pub mod generate_video;
@@ -55,6 +55,7 @@ fn image_model_to_common_model_type(model: RouterImageModel) -> Option<CommonMod
RouterImageModel::Seedream4p5 => Some(CommonModelType::Seedream4p5),
RouterImageModel::Seedream5Lite => Some(CommonModelType::Seedream5Lite),
RouterImageModel::Seedream5p0Pro => Some(CommonModelType::Seedream5p0Pro),
RouterImageModel::Seedream5p0ProUltra => Some(CommonModelType::Seedream5p0ProUltra),
RouterImageModel::QwenEdit2511Angles => Some(CommonModelType::QwenEdit2511Angles),
RouterImageModel::Flux2LoraAngles => Some(CommonModelType::Flux2LoraAngles),
RouterImageModel::GrokImagineImage => Some(CommonModelType::GrokImagineImage),
@@ -83,7 +83,8 @@ pub async fn insert_generic_inference_job_for_seedance2pro_queue_with_apriori_jo
Some(CommonModelType::Midjourney7) |
Some(CommonModelType::Midjourney7Niji) |
Some(CommonModelType::Midjourney8) |
Some(CommonModelType::Seedream5p0Pro) => true,
Some(CommonModelType::Seedream5p0Pro) |
Some(CommonModelType::Seedream5p0ProUltra) => true,
_ => false,
};
@@ -111,7 +112,12 @@ pub async fn insert_generic_inference_job_for_seedance2pro_queue_with_apriori_jo
InferenceJobExternalThirdParty::Seedance2ProAlt,
InferenceJobProductCategory::Seedance2ProVideoAlt,
),
(KinoviVersion::BytePlusUltra, _) => (
(KinoviVersion::BytePlusUltra, true) => (
InferenceJobType::Seedance2ProBytePlusUltraQueue,
InferenceJobExternalThirdParty::Seedance2ProBytePlusUltra,
InferenceJobProductCategory::Seedance2ProImageBytePlusUltra,
),
(KinoviVersion::BytePlusUltra, false) => (
InferenceJobType::Seedance2ProBytePlusUltraQueue,
InferenceJobExternalThirdParty::Seedance2ProBytePlusUltra,
InferenceJobProductCategory::Seedance2ProVideoBytePlusUltra,
@@ -72,6 +72,10 @@ pub enum InferenceJobProductCategory {
#[serde(rename = "seedance2pro_video_bpu")]
Seedance2ProVideoBytePlusUltra,
/// Seedance 2 Pro / Kinovi (BytePlus Ultra account): image generation.
#[serde(rename = "seedance2pro_image_bpu")]
Seedance2ProImageBytePlusUltra,
// =============== WORLD LABS ===============
/// World Labs: Gaussian Splat Generation
@@ -183,6 +187,7 @@ impl InferenceJobProductCategory {
Self::Seedance2ProAudio => "seedance2pro_audio",
Self::Seedance2ProVideoAlt => "seedance2pro_video_alt",
Self::Seedance2ProVideoBytePlusUltra => "seedance2pro_video_bpu",
Self::Seedance2ProImageBytePlusUltra => "seedance2pro_image_bpu",
Self::WorldlabsSplat => "worldlabs_splat",
Self::TtsGptSoVits => "tts_gpt_so_vits",
Self::TtsStyleTts2 => "tts_style_tts2",
@@ -226,6 +231,7 @@ impl InferenceJobProductCategory {
"seedance2pro_audio" => Ok(Self::Seedance2ProAudio),
"seedance2pro_video_alt" => Ok(Self::Seedance2ProVideoAlt),
"seedance2pro_video_bpu" => Ok(Self::Seedance2ProVideoBytePlusUltra),
"seedance2pro_image_bpu" => Ok(Self::Seedance2ProImageBytePlusUltra),
"worldlabs_splat" => Ok(Self::WorldlabsSplat),
"tts_gpt_so_vits" => Ok(Self::TtsGptSoVits),
"tts_style_tts2" => Ok(Self::TtsStyleTts2),
@@ -272,6 +278,7 @@ impl InferenceJobProductCategory {
Self::Seedance2ProAudio,
Self::Seedance2ProVideoAlt,
Self::Seedance2ProVideoBytePlusUltra,
Self::Seedance2ProImageBytePlusUltra,
Self::WorldlabsSplat,
Self::TtsGptSoVits,
Self::TtsStyleTts2,
@@ -324,6 +331,7 @@ mod tests {
assert_serialization(InferenceJobProductCategory::Seedance2ProAudio, "seedance2pro_audio");
assert_serialization(InferenceJobProductCategory::Seedance2ProVideoAlt, "seedance2pro_video_alt");
assert_serialization(InferenceJobProductCategory::Seedance2ProVideoBytePlusUltra, "seedance2pro_video_bpu");
assert_serialization(InferenceJobProductCategory::Seedance2ProImageBytePlusUltra, "seedance2pro_image_bpu");
assert_serialization(InferenceJobProductCategory::WorldlabsSplat, "worldlabs_splat");
assert_serialization(InferenceJobProductCategory::TtsGptSoVits, "tts_gpt_so_vits");
assert_serialization(InferenceJobProductCategory::TtsStyleTts2, "tts_style_tts2");
@@ -366,6 +374,7 @@ mod tests {
assert_eq!(InferenceJobProductCategory::Seedance2ProAudio.to_str(), "seedance2pro_audio");
assert_eq!(InferenceJobProductCategory::Seedance2ProVideoAlt.to_str(), "seedance2pro_video_alt");
assert_eq!(InferenceJobProductCategory::Seedance2ProVideoBytePlusUltra.to_str(), "seedance2pro_video_bpu");
assert_eq!(InferenceJobProductCategory::Seedance2ProImageBytePlusUltra.to_str(), "seedance2pro_image_bpu");
assert_eq!(InferenceJobProductCategory::WorldlabsSplat.to_str(), "worldlabs_splat");
assert_eq!(InferenceJobProductCategory::TtsGptSoVits.to_str(), "tts_gpt_so_vits");
assert_eq!(InferenceJobProductCategory::TtsStyleTts2.to_str(), "tts_style_tts2");
@@ -407,6 +416,7 @@ mod tests {
assert_eq!(InferenceJobProductCategory::from_str("seedance2pro_audio").unwrap(), InferenceJobProductCategory::Seedance2ProAudio);
assert_eq!(InferenceJobProductCategory::from_str("seedance2pro_video_alt").unwrap(), InferenceJobProductCategory::Seedance2ProVideoAlt);
assert_eq!(InferenceJobProductCategory::from_str("seedance2pro_video_bpu").unwrap(), InferenceJobProductCategory::Seedance2ProVideoBytePlusUltra);
assert_eq!(InferenceJobProductCategory::from_str("seedance2pro_image_bpu").unwrap(), InferenceJobProductCategory::Seedance2ProImageBytePlusUltra);
assert_eq!(InferenceJobProductCategory::from_str("worldlabs_splat").unwrap(), InferenceJobProductCategory::WorldlabsSplat);
assert_eq!(InferenceJobProductCategory::from_str("tts_gpt_so_vits").unwrap(), InferenceJobProductCategory::TtsGptSoVits);
assert_eq!(InferenceJobProductCategory::from_str("tts_style_tts2").unwrap(), InferenceJobProductCategory::TtsStyleTts2);
@@ -434,7 +444,7 @@ mod tests {
#[test]
fn all_variants() {
// Static check
const EXPECTED_COUNT : usize = 38;
const EXPECTED_COUNT : usize = 39;
assert_eq!(InferenceJobProductCategory::all_variants().len(), EXPECTED_COUNT);
assert_eq!(InferenceJobProductCategory::iter().len(), EXPECTED_COUNT);
@@ -129,6 +129,8 @@ pub enum InferenceModelType {
Seedream5Lite,
#[serde(rename = "seedream_5p0_pro")]
Seedream5p0Pro,
#[serde(rename = "seedream_5p0_pro_u")]
Seedream5p0ProUltra,
#[serde(rename = "midjourney")]
Midjourney,
#[serde(rename = "midjourney_v6")]
@@ -337,6 +339,7 @@ impl InferenceModelType {
Self::Seedream4p5 => "seedream_4p5",
Self::Seedream5Lite => "seedream_5_lite",
Self::Seedream5p0Pro => "seedream_5p0_pro",
Self::Seedream5p0ProUltra => "seedream_5p0_pro_u",
Self::Midjourney => "midjourney",
Self::MidjourneyV6 => "midjourney_v6",
Self::MidjourneyV6p1 => "midjourney_v6p1",
@@ -466,6 +469,7 @@ impl InferenceModelType {
"seedream_4p5" => Ok(Self::Seedream4p5),
"seedream_5_lite" => Ok(Self::Seedream5Lite),
"seedream_5p0_pro" => Ok(Self::Seedream5p0Pro),
"seedream_5p0_pro_u" => Ok(Self::Seedream5p0ProUltra),
"midjourney" => Ok(Self::Midjourney),
"midjourney_v6" => Ok(Self::MidjourneyV6),
"midjourney_v6p1" => Ok(Self::MidjourneyV6p1),
@@ -599,6 +603,7 @@ impl InferenceModelType {
Self::Seedream4p5,
Self::Seedream5Lite,
Self::Seedream5p0Pro,
Self::Seedream5p0ProUltra,
Self::Midjourney,
Self::MidjourneyV6,
Self::MidjourneyV6p1,
@@ -715,6 +720,7 @@ impl InferenceModelType {
CommonModelType::Seedream4p5 => Self::Seedream4p5,
CommonModelType::Seedream5Lite => Self::Seedream5Lite,
CommonModelType::Seedream5p0Pro => Self::Seedream5p0Pro,
CommonModelType::Seedream5p0ProUltra => Self::Seedream5p0ProUltra,
CommonModelType::Midjourney => Self::Midjourney,
CommonModelType::MidjourneyV6 => Self::MidjourneyV6,
CommonModelType::MidjourneyV6p1 => Self::MidjourneyV6p1,
@@ -49,6 +49,8 @@ pub enum CommonImageModel {
Seedream5Lite,
#[serde(rename = "seedream_5p0_pro")]
Seedream5p0Pro,
#[serde(rename = "seedream_5p0_pro_u")]
Seedream5p0ProUltra,
#[serde(rename = "qwen_edit_2511_angles")]
QwenEdit2511Angles,
#[serde(rename = "flux_2_lora_angles")]
@@ -78,6 +80,7 @@ impl CommonImageModel {
Self::Seedream4p5 => CommonModelType::Seedream4p5,
Self::Seedream5Lite => CommonModelType::Seedream5Lite,
Self::Seedream5p0Pro => CommonModelType::Seedream5p0Pro,
Self::Seedream5p0ProUltra => CommonModelType::Seedream5p0ProUltra,
Self::QwenEdit2511Angles => CommonModelType::QwenEdit2511Angles,
Self::Flux2LoraAngles => CommonModelType::Flux2LoraAngles,
}
@@ -115,6 +118,7 @@ mod tests {
assert_serialization(CommonImageModel::Seedream4p5, "seedream_4p5");
assert_serialization(CommonImageModel::Seedream5Lite, "seedream_5_lite");
assert_serialization(CommonImageModel::Seedream5p0Pro, "seedream_5p0_pro");
assert_serialization(CommonImageModel::Seedream5p0ProUltra, "seedream_5p0_pro_u");
assert_serialization(CommonImageModel::QwenEdit2511Angles, "qwen_edit_2511_angles");
assert_serialization(CommonImageModel::Flux2LoraAngles, "flux_2_lora_angles");
}
@@ -141,6 +145,7 @@ mod tests {
(CommonImageModel::Seedream4p5, CommonModelType::Seedream4p5),
(CommonImageModel::Seedream5Lite, CommonModelType::Seedream5Lite),
(CommonImageModel::Seedream5p0Pro, CommonModelType::Seedream5p0Pro),
(CommonImageModel::Seedream5p0ProUltra, CommonModelType::Seedream5p0ProUltra),
(CommonImageModel::QwenEdit2511Angles, CommonModelType::QwenEdit2511Angles),
(CommonImageModel::Flux2LoraAngles, CommonModelType::Flux2LoraAngles),
];
@@ -169,7 +174,7 @@ mod tests {
#[test]
fn variant_count() {
use strum::IntoEnumIterator;
assert_eq!(CommonImageModel::iter().len(), 21);
assert_eq!(CommonImageModel::iter().len(), 22);
}
#[test]
@@ -85,6 +85,9 @@ pub enum CommonModelType {
/// Seedream 5.0 Pro as routed via kinovi
#[serde(rename = "seedream_5p0_pro")]
Seedream5p0Pro,
/// Seedream 5.0 Pro as routed via the kinovi ultra account
#[serde(rename = "seedream_5p0_pro_u")]
Seedream5p0ProUltra,
/// Midjourney without distinguishing a model type or version
#[serde(rename = "midjourney")]
@@ -291,6 +294,7 @@ impl CommonModelType {
Self::Seedream4p5 => "seedream_4p5",
Self::Seedream5Lite => "seedream_5_lite",
Self::Seedream5p0Pro => "seedream_5p0_pro",
Self::Seedream5p0ProUltra => "seedream_5p0_pro_u",
Self::Midjourney => "midjourney",
Self::MidjourneyV6 => "midjourney_v6",
Self::MidjourneyV6p1 => "midjourney_v6p1",
@@ -403,6 +407,7 @@ impl CommonModelType {
"seedream_4p5" => Ok(Self::Seedream4p5),
"seedream_5_lite" => Ok(Self::Seedream5Lite),
"seedream_5p0_pro" => Ok(Self::Seedream5p0Pro),
"seedream_5p0_pro_u" => Ok(Self::Seedream5p0ProUltra),
"midjourney" => Ok(Self::Midjourney),
"midjourney_v6" => Ok(Self::MidjourneyV6),
"midjourney_v6p1" => Ok(Self::MidjourneyV6p1),
@@ -519,6 +524,7 @@ impl CommonModelType {
Self::Seedream4p5,
Self::Seedream5Lite,
Self::Seedream5p0Pro,
Self::Seedream5p0ProUltra,
Self::Midjourney,
Self::MidjourneyV6,
Self::MidjourneyV6p1,
@@ -632,6 +638,7 @@ impl CommonModelType {
Self::Seedream4p5 => CommonModelClass::Image,
Self::Seedream5Lite => CommonModelClass::Image,
Self::Seedream5p0Pro => CommonModelClass::Image,
Self::Seedream5p0ProUltra => CommonModelClass::Image,
Self::Midjourney => CommonModelClass::Image,
Self::MidjourneyV6 => CommonModelClass::Image,
Self::MidjourneyV6p1 => CommonModelClass::Image,
@@ -755,6 +762,7 @@ mod tests {
assert_serialization(CommonModelType::Seedream4p5, "seedream_4p5");
assert_serialization(CommonModelType::Seedream5Lite, "seedream_5_lite");
assert_serialization(CommonModelType::Seedream5p0Pro, "seedream_5p0_pro");
assert_serialization(CommonModelType::Seedream5p0ProUltra, "seedream_5p0_pro_u");
assert_serialization(CommonModelType::Midjourney, "midjourney");
assert_serialization(CommonModelType::MidjourneyV6, "midjourney_v6");
assert_serialization(CommonModelType::MidjourneyV6p1, "midjourney_v6p1");
@@ -861,6 +869,7 @@ mod tests {
assert_eq!(CommonModelType::Seedream4p5.to_str(), "seedream_4p5");
assert_eq!(CommonModelType::Seedream5Lite.to_str(), "seedream_5_lite");
assert_eq!(CommonModelType::Seedream5p0Pro.to_str(), "seedream_5p0_pro");
assert_eq!(CommonModelType::Seedream5p0ProUltra.to_str(), "seedream_5p0_pro_u");
assert_eq!(CommonModelType::Midjourney.to_str(), "midjourney");
assert_eq!(CommonModelType::MidjourneyV6.to_str(), "midjourney_v6");
assert_eq!(CommonModelType::MidjourneyV6p1.to_str(), "midjourney_v6p1");
@@ -971,6 +980,7 @@ mod tests {
assert_eq!(CommonModelType::from_str("seedream_4p5").unwrap(), CommonModelType::Seedream4p5);
assert_eq!(CommonModelType::from_str("seedream_5_lite").unwrap(), CommonModelType::Seedream5Lite);
assert_eq!(CommonModelType::from_str("seedream_5p0_pro").unwrap(), CommonModelType::Seedream5p0Pro);
assert_eq!(CommonModelType::from_str("seedream_5p0_pro_u").unwrap(), CommonModelType::Seedream5p0ProUltra);
assert_eq!(CommonModelType::from_str("midjourney").unwrap(), CommonModelType::Midjourney);
assert_eq!(CommonModelType::from_str("midjourney_v6").unwrap(), CommonModelType::MidjourneyV6);
assert_eq!(CommonModelType::from_str("midjourney_v6p1").unwrap(), CommonModelType::MidjourneyV6p1);
@@ -1052,7 +1062,7 @@ mod tests {
#[test]
fn all_variants() {
let mut variants = CommonModelType::all_variants();
assert_eq!(variants.len(), 97);
assert_eq!(variants.len(), 98);
// Image models
assert_eq!(variants.pop_first(), Some(CommonModelType::Flux1Dev));
assert_eq!(variants.pop_first(), Some(CommonModelType::Flux1Schnell));
@@ -1080,6 +1090,7 @@ mod tests {
assert_eq!(variants.pop_first(), Some(CommonModelType::Seedream4p5));
assert_eq!(variants.pop_first(), Some(CommonModelType::Seedream5Lite));
assert_eq!(variants.pop_first(), Some(CommonModelType::Seedream5p0Pro));
assert_eq!(variants.pop_first(), Some(CommonModelType::Seedream5p0ProUltra));
assert_eq!(variants.pop_first(), Some(CommonModelType::Midjourney));
assert_eq!(variants.pop_first(), Some(CommonModelType::MidjourneyV6));
assert_eq!(variants.pop_first(), Some(CommonModelType::MidjourneyV6p1));
@@ -470,6 +470,37 @@ fn build_omni_gen_image_models() -> Vec<OmniGenImageModelDetails> {
..Default::default()
});
models.push(OmniGenImageModelDetails {
model: CommonImageModel::Seedream5p0ProUltra,
model_creator: Some(ModelCreator::Bytedance),
full_name: Some("Seedream 5.0 Pro Ultra".to_string()),
text_prompt_supported: Some(true),
image_refs_supported: Some(true),
image_refs_max: Some(14),
aspect_ratio_options: Some(vec![
CommonAspectRatio::Auto,
CommonAspectRatio::WideTwentyOneByNine,
CommonAspectRatio::WideSixteenByNine,
CommonAspectRatio::WideThreeByTwo,
CommonAspectRatio::WideFourByThree,
CommonAspectRatio::Square,
CommonAspectRatio::TallThreeByFour,
CommonAspectRatio::TallTwoByThree,
CommonAspectRatio::TallNineBySixteen,
]),
aspect_ratio_default: Some(CommonAspectRatio::Square),
aspect_ratio_default_when_editing: Some(CommonAspectRatio::Auto),
resolution_options: Some(vec![
CommonResolution::OneK,
CommonResolution::TwoK,
]),
resolution_default: Some(CommonResolution::TwoK),
batch_size_min: Some(1),
batch_size_max: Some(8),
batch_size_default: Some(1),
..Default::default()
});
models
}
@@ -519,6 +550,10 @@ fn build_omni_gen_image_model_providers() -> Vec<OmniGenImageModelProviderDetail
model: CommonImageModel::Seedream5p0Pro,
overrides: None,
},
OmniGenImageProviderModelDetails {
model: CommonImageModel::Seedream5p0ProUltra,
overrides: None,
},
OmniGenImageProviderModelDetails {
model: CommonImageModel::NanoBanana,
overrides: None,
@@ -95,6 +95,7 @@ use artcraft_api_defs::moderation::debug_logs::debug_log_entry::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_all_debug_logs::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_token::*;
use artcraft_api_defs::moderation::debug_logs::moderation_list_debug_logs_for_user::*;
use crate::http_server::endpoints::moderation::dashboards::moderator_list_databox_dashboards_handler::*;
use crate::http_server::endpoints::moderation::jobs::moderation_get_job_by_token_handler::*;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_edit_user_feature_flags_handler::*;
use crate::http_server::endpoints::moderation::user_feature_flags::moderator_list_all_available_user_feature_flags_handler::*;
@@ -640,6 +641,7 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
crate::http_server::endpoints::omni_api::job_status::batch_get_job_status_handler::omni_api_batch_get_job_status_handler,
// Moderation
crate::http_server::endpoints::moderation::alerts::moderation_send_alert_handler::moderation_send_alert_handler,
crate::http_server::endpoints::moderation::dashboards::moderator_list_databox_dashboards_handler::moderator_list_databox_dashboards_handler,
crate::http_server::endpoints::moderation::info::moderator_token_info_handler::moderator_get_token_info_handler,
crate::http_server::endpoints::moderation::user::moderator_list_subscribing_users_by_signup_date::moderator_list_subscribing_users_by_signup_date_handler,
crate::http_server::endpoints::moderation::user::moderator_list_users_by_signup_date::moderator_list_users_by_signup_date_handler,
@@ -724,6 +726,8 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
crate::http_server::endpoints::image_studio::update_gpt_image_job_status_handler::update_gpt_image_job_status_handler,
),
components(schemas(
DataboxDashboardEntry,
ListDataboxDashboardsResponse,
ModeratorGetUserSpendSummaryResponse,
UserSpendSummaryView,
ModeratorReengagementListResponse,
@@ -10,7 +10,6 @@ use http_server_common::request::get_request_ip::get_request_ip;
use mysql_queries::queries::api_keys::insert_api_key::{insert_api_key, InsertApiKeyArgs};
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::user_lookup::user_session::session_utils::lookup::user_session_feature_flags::UserSessionFeatureFlags;
use crate::http_server::user_lookup::user_session::require_user_session::require_user_session;
use crate::state::server_state::ServerState;
use crate::util::generate_api_key::generate_api_key;
@@ -30,7 +29,6 @@ const MAX_DESCRIPTION_LEN: usize = 512;
(status = 200, body = CreateApiKeySuccessResponse),
(status = 400, body = CommonWebError),
(status = 401, body = CommonWebError),
(status = 403, body = CommonWebError),
(status = 500, body = CommonWebError),
),
)]
@@ -46,14 +44,6 @@ pub async fn create_api_key_handler(
let user_session = require_user_session(&http_request, &server_state.session_checker, &mut *conn).await?;
// API key creation is gated behind the `api_key` feature flag.
let feature_flags = UserSessionFeatureFlags::new(user_session.maybe_feature_flags.as_deref());
if !feature_flags.can_create_api_key() {
warn!("User {:?} attempted to create an API key without the api_key feature flag", user_session.user_token.as_str());
return Err(CommonWebError::Forbidden);
}
let name = request.name.trim().to_string();
let maybe_description = request.maybe_description.as_ref().map(|s| s.trim().to_string());
@@ -142,6 +142,7 @@ fn map_image_model(model: CommonImageModel) -> Result<RouterImageModel, HandlerE
CommonImageModel::Seedream4p5 => RouterImageModel::Seedream4p5,
CommonImageModel::Seedream5Lite => RouterImageModel::Seedream5Lite,
CommonImageModel::Seedream5p0Pro => RouterImageModel::Seedream5p0Pro,
CommonImageModel::Seedream5p0ProUltra => RouterImageModel::Seedream5p0ProUltra,
CommonImageModel::GrokImagineImage => RouterImageModel::GrokImagineImage,
CommonImageModel::GrokImagineImageQuality => RouterImageModel::GrokImagineImageQuality,
CommonImageModel::QwenEdit2511Angles => RouterImageModel::QwenEdit2511Angles,
@@ -0,0 +1 @@
pub mod moderator_list_databox_dashboards_handler;
@@ -0,0 +1,67 @@
use std::sync::Arc;
use actix_web::web::Json;
use actix_web::{web, HttpRequest};
use serde::Serialize;
use utoipa::ToSchema;
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::user_lookup::user_session::require_moderator::require_moderator;
use crate::state::server_state::ServerState;
#[derive(Serialize, ToSchema)]
pub struct ListDataboxDashboardsResponse {
pub success: bool,
/// Configured Databox datawalls, in display order.
pub databoards: Vec<DataboxDashboardEntry>,
}
#[derive(Serialize, ToSchema)]
pub struct DataboxDashboardEntry {
/// Human-readable dashboard name.
pub name: String,
/// Databox datawall id.
pub id: String,
}
/// List the Databox metrics dashboards configured for the admin dashboard.
/// Dashboards whose env vars aren't set are omitted.
#[utoipa::path(
get,
tag = "Moderation",
path = "/v1/moderation/dashboards/databox",
responses(
(status = 200, description = "Success", body = ListDataboxDashboardsResponse),
(status = 401, description = "Not authorized", body = CommonWebError),
(status = 500, description = "Server error", body = CommonWebError),
),
)]
pub async fn moderator_list_databox_dashboards_handler(
http_request: HttpRequest,
server_state: web::Data<Arc<ServerState>>,
) -> Result<Json<ListDataboxDashboardsResponse>, CommonWebError> {
require_moderator(&http_request, &server_state.session_checker, &server_state.mysql_pool).await?;
let databox = &server_state.dashboards.databox;
let databoards = [
("DAUs", databox.daus_id.as_ref()),
("Daily Generations", databox.daily_generations_id.as_ref()),
]
.into_iter()
.filter_map(|(name, maybe_id)| {
maybe_id.map(|id| DataboxDashboardEntry {
name: name.to_string(),
id: id.clone(),
})
})
.collect();
Ok(Json(ListDataboxDashboardsResponse {
success: true,
databoards,
}))
}
@@ -1,4 +1,5 @@
pub mod alerts;
pub mod dashboards;
pub mod debug_logs;
pub mod info;
pub mod ip_bans;
@@ -13,6 +13,7 @@ use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use enums::by_table::prompt_context_items::prompt_context_semantic_type::PromptContextSemanticType;
use enums::by_table::prompts::prompt_type::PromptType;
use enums::common::generation::common_generation_mode::CommonGenerationMode;
use enums::common::generation::common_image_model::CommonImageModel;
use enums::common::generation::common_model_type::CommonModelType;
use enums::common::generation_provider::GenerationProvider;
use enums::common::platform_type::PlatformType;
@@ -37,6 +38,7 @@ use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::ins
use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::shared_job_args::SharedJobArgs;
use crate::http_server::endpoints::omni_api::generate::image::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
use crate::http_server::endpoints::omni_api::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
use crate::http_server::user_lookup::user_session::session_utils::lookup::user_session_feature_flags::UserSessionFeatureFlags;
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
@@ -135,6 +137,14 @@ pub async fn omni_api_image_generate_handler(
let router_builder = hydrate_to_router_request(&request)?;
// ==================== PIPELINE DISPATCH ==================== //
let kinovi_account = match request.model {
Some(CommonImageModel::Seedream5p0ProUltra) => KinoviAccount::BytePlusUltra,
// Everything else goes through Volcengine
_ => KinoviAccount::Volcengine,
};
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
@@ -170,6 +180,7 @@ pub async fn omni_api_image_generate_handler(
server_state: &server_state,
user_token,
resolved_media: &resolved_media,
kinovi_account,
debug_log_context: &debug_log_context,
mysql_connection,
}).await;
@@ -289,10 +300,11 @@ pub async fn omni_api_image_generate_handler(
GenerateImageResponse::Seedance2Pro(payload) => {
info!("Inserting seedance2pro image job(s) with token: {:?}", pipeline_result.apriori_job_token);
// The image-side omni pipeline always dispatches Midjourney via the
// Volcengine Kinovi account today. If we ever route to BytePlus /
// BytePlus Ultra here, mirror the video-side `kinovi_account` knob.
let kinovi_version = KinoviVersion::Volcengine;
let kinovi_version = match kinovi_account {
KinoviAccount::Volcengine => KinoviVersion::Volcengine,
KinoviAccount::BytePlus => KinoviVersion::BytePlus,
KinoviAccount::BytePlusUltra => KinoviVersion::BytePlusUltra,
};
let result = insert_seedance2pro_jobs(InsertSeedance2proJobsArgs {
primary_order_id: &payload.order_id,
@@ -21,6 +21,7 @@ use crate::http_server::endpoints::generate::common::generation_debug_logs::{
insert_provider_request_debug_log, provider_request_debug_log_type, GenerationDebugLogContext,
};
use crate::http_server::endpoints::omni_api::generate::image::pipeline_result::ImagePipelineResult;
use crate::http_server::endpoints::omni_api::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
use crate::util::lookup::lookup_media_files_as_cdn_url_list_and_map::MediaFilesAsCdnUrlListAndMap;
@@ -30,6 +31,7 @@ pub struct RunPipelineV2Args<'a> {
pub server_state: &'a ServerState,
pub user_token: &'a UserToken,
pub resolved_media: &'a MediaFilesAsCdnUrlListAndMap,
pub kinovi_account: KinoviAccount,
pub debug_log_context: &'a GenerationDebugLogContext<'a>,
/// The handler's open connection. The pipeline uses it for its remaining
/// pre-request DB writes (billing, outbound-request debug log) and releases
@@ -49,6 +51,7 @@ pub async fn run_pipeline_v2(
server_state,
user_token,
resolved_media,
kinovi_account,
debug_log_context,
mut mysql_connection,
} = args;
@@ -97,7 +100,7 @@ pub async fn run_pipeline_v2(
// across the external generation call below.
drop(mysql_connection);
let response = finalize_and_generate(draft_or_request, server_state, resolved_media).await?;
let response = finalize_and_generate(draft_or_request, server_state, resolved_media, kinovi_account).await?;
Ok(ImagePipelineResult {
apriori_job_token,
@@ -118,15 +121,17 @@ fn build_execution_request(
})
}
/// Route each image model to the provider that fulfils it. Midjourney is
/// served via the Seedance2Pro/Kinovi (Volcengine) backend; everything else
/// flows through Fal.
/// Route each image model to the provider that fulfils it. Midjourney and
/// Seedream 5.0 Pro (both variants) are served via the Seedance2Pro/Kinovi
/// backend; everything else flows through Fal. Which Kinovi account fulfils
/// a Kinovi request is the handler's `kinovi_account` choice.
fn provider_for_model(model: RouterImageModel) -> RouterProvider {
match model {
RouterImageModel::Midjourney7
| RouterImageModel::Midjourney7Niji
| RouterImageModel::Midjourney8
| RouterImageModel::Seedream5p0Pro => RouterProvider::Seedance2Pro,
| RouterImageModel::Seedream5p0Pro
| RouterImageModel::Seedream5p0ProUltra => RouterProvider::Seedance2Pro,
_ => RouterProvider::Fal,
}
}
@@ -172,9 +177,10 @@ async fn finalize_and_generate(
draft_or_request: ImageGenerationDraftOrRequest,
server_state: &ServerState,
resolved_media: &MediaFilesAsCdnUrlListAndMap,
kinovi_account: KinoviAccount,
) -> Result<GenerateImageResponse, CommonWebError> {
let provider = draft_or_request.get_provider();
let client = build_router_client(provider, server_state)?;
let client = build_router_client(provider, server_state, kinovi_account)?;
// Inline the draft finalization so the draft context can borrow `client`.
let request: ImageGenerationRequest = match draft_or_request {
@@ -204,6 +210,7 @@ async fn finalize_and_generate(
fn build_router_client(
provider: RouterProvider,
server_state: &ServerState,
kinovi_account: KinoviAccount,
) -> Result<RouterClient, CommonWebError> {
match provider {
RouterProvider::Fal => {
@@ -214,11 +221,12 @@ fn build_router_client(
Ok(RouterClient::Fal(fal_client))
},
RouterProvider::Seedance2Pro => {
// Midjourney image generation always uses the Volcengine account.
// Per-request account selection (like the video flow's
// `KinoviAccount` enum) isn't needed here yet — if it ever is, plumb
// a parallel knob through `RunPipelineV2Args`.
let cookies = server_state.inference_providers.seedance2pro.cookies_volcengine.clone();
let seedance2pro = &server_state.inference_providers.seedance2pro;
let cookies = match kinovi_account {
KinoviAccount::Volcengine => seedance2pro.cookies_volcengine.clone(),
KinoviAccount::BytePlus => seedance2pro.cookies_byteplus.clone(),
KinoviAccount::BytePlusUltra => seedance2pro.cookies_byteplus_ultra.clone(),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
Ok(RouterClient::Seedance2Pro(RouterSeedance2ProClient::new(session)))
},
@@ -9,7 +9,7 @@ use artcraft_router::client::router_seedance2pro_client::RouterSeedance2ProClien
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::shared_utils::kinovi_account::KinoviAccount;
use crate::state::server_state::ServerState;
pub fn build_router_client(
@@ -4,4 +4,3 @@ pub mod helpers;
pub mod ingest_url_inputs;
pub mod pipeline_v2;
pub mod insert_db_job;
pub mod kinovi_account;
@@ -43,7 +43,7 @@ use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::ins
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_grok_api_job::{insert_grok_api_job, InsertGrokApiJobArgs};
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::shared_job_args::SharedJobArgs;
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::generate::video::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
use crate::http_server::endpoints::omni_api::shared_utils::video::validate_video_request::validate_video_request;
use crate::http_server::user_lookup::user_session::session_utils::lookup::user_session_feature_flags::UserSessionFeatureFlags;
@@ -23,7 +23,7 @@ use crate::http_server::endpoints::omni_api::generate::video::helpers::build_rou
use crate::http_server::endpoints::omni_api::generate::video::helpers::pipeline_result::PipelineResult;
use crate::http_server::endpoints::omni_api::generate::video::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
use mysql_queries::queries::generic_inference::common::job_cost_estimates::JobCostEstimates;
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_api::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
@@ -1,4 +1,5 @@
pub mod check_url_inputs;
pub mod kinovi_account;
pub mod map_seedance2pro_router_error;
pub mod media_uploads;
pub mod video;
@@ -36,7 +36,7 @@ use crate::http_server::endpoints::omni_gen::generate::audio::insert_db_job::ins
use crate::http_server::endpoints::omni_gen::generate::audio::insert_db_job::insert_seedance2pro_job::{insert_seedance2pro_job, InsertSeedance2proJobArgs};
use crate::http_server::endpoints::omni_gen::generate::audio::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
use crate::http_server::endpoints::omni_gen::generate::video::insert_db_job::shared_job_args::SharedJobArgs;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::audio::validate_audio_request::validate_audio_request;
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
use crate::http_server::web_utils::get_request_platform_type::get_request_platform_type;
@@ -22,7 +22,7 @@ use crate::http_server::endpoints::omni_gen::generate::audio::helpers::pipeline_
use crate::http_server::endpoints::omni_gen::generate::audio::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::bill_wallet::bill_wallet;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::build_router_client::build_router_client;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
@@ -12,6 +12,7 @@ use enums::by_table::debug_logs::debug_log_type::DebugLogType;
use enums::by_table::prompt_context_items::prompt_context_semantic_type::PromptContextSemanticType;
use enums::by_table::prompts::prompt_type::PromptType;
use enums::common::generation::common_generation_mode::CommonGenerationMode;
use enums::common::generation::common_image_model::CommonImageModel;
use enums::common::generation::common_model_type::CommonModelType;
use enums::common::generation_provider::GenerationProvider;
use http_server_common::request::get_request_ip::get_request_ip;
@@ -34,6 +35,7 @@ use crate::http_server::endpoints::omni_gen::generate::image::insert_db_job::ins
use crate::http_server::endpoints::omni_gen::generate::image::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
use crate::http_server::endpoints::omni_gen::generate::image::insert_db_job::shared_job_args::SharedJobArgs;
use crate::http_server::endpoints::omni_gen::generate::image::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::user_lookup::user_session::session_utils::lookup::user_session_feature_flags::UserSessionFeatureFlags;
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
use crate::http_server::web_utils::get_request_platform_type::get_request_platform_type;
@@ -130,6 +132,14 @@ pub async fn omni_gen_image_generate_handler(
let router_builder = hydrate_to_router_request(&request)?;
// ==================== PIPELINE DISPATCH ==================== //
let kinovi_account = match request.model {
Some(CommonImageModel::Seedream5p0ProUltra) => KinoviAccount::BytePlusUltra,
// Everything else goes through Volcengine
_ => KinoviAccount::Volcengine,
};
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
let ip_address = get_request_ip(&http_request);
@@ -168,6 +178,7 @@ pub async fn omni_gen_image_generate_handler(
server_state: &server_state,
user_token,
resolved_media: &resolved_media,
kinovi_account,
debug_log_context: &debug_log_context,
mysql_connection,
}).await;
@@ -286,10 +297,11 @@ pub async fn omni_gen_image_generate_handler(
GenerateImageResponse::Seedance2Pro(payload) => {
info!("Inserting seedance2pro image job(s) with token: {:?}", pipeline_result.apriori_job_token);
// The image-side omni pipeline always dispatches Midjourney via the
// Volcengine Kinovi account today. If we ever route to BytePlus /
// BytePlus Ultra here, mirror the video-side `kinovi_account` knob.
let kinovi_version = KinoviVersion::Volcengine;
let kinovi_version = match kinovi_account {
KinoviAccount::Volcengine => KinoviVersion::Volcengine,
KinoviAccount::BytePlus => KinoviVersion::BytePlus,
KinoviAccount::BytePlusUltra => KinoviVersion::BytePlusUltra,
};
let result = insert_seedance2pro_jobs(InsertSeedance2proJobsArgs {
primary_order_id: &payload.order_id,
@@ -21,6 +21,7 @@ use crate::http_server::endpoints::generate::common::generation_debug_logs::{
insert_provider_request_debug_log, provider_request_debug_log_type, GenerationDebugLogContext,
};
use crate::http_server::endpoints::omni_gen::generate::image::pipeline_result::ImagePipelineResult;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
use crate::util::lookup::lookup_media_files_as_cdn_url_list_and_map::MediaFilesAsCdnUrlListAndMap;
@@ -30,6 +31,7 @@ pub struct RunPipelineV2Args<'a> {
pub server_state: &'a ServerState,
pub user_token: &'a UserToken,
pub resolved_media: &'a MediaFilesAsCdnUrlListAndMap,
pub kinovi_account: KinoviAccount,
pub debug_log_context: &'a GenerationDebugLogContext<'a>,
/// The handler's open connection. The pipeline uses it for its remaining
/// pre-request DB writes (billing, outbound-request debug log) and releases
@@ -49,6 +51,7 @@ pub async fn run_pipeline_v2(
server_state,
user_token,
resolved_media,
kinovi_account,
debug_log_context,
mut mysql_connection,
} = args;
@@ -97,7 +100,7 @@ pub async fn run_pipeline_v2(
// across the external generation call below.
drop(mysql_connection);
let response = finalize_and_generate(draft_or_request, server_state, resolved_media).await?;
let response = finalize_and_generate(draft_or_request, server_state, resolved_media, kinovi_account).await?;
Ok(ImagePipelineResult {
apriori_job_token,
@@ -119,14 +122,16 @@ fn build_execution_request(
}
/// Route each image model to the provider that fulfils it. Midjourney and
/// Seedream 5.0 Pro are served via the Seedance2Pro/Kinovi (Volcengine)
/// backend; everything else flows through Fal.
/// Seedream 5.0 Pro (both variants) are served via the Seedance2Pro/Kinovi
/// backend; everything else flows through Fal. Which Kinovi account fulfils
/// a Kinovi request is the handler's `kinovi_account` choice.
fn provider_for_model(model: RouterImageModel) -> RouterProvider {
match model {
RouterImageModel::Midjourney7
| RouterImageModel::Midjourney7Niji
| RouterImageModel::Midjourney8
| RouterImageModel::Seedream5p0Pro => RouterProvider::Seedance2Pro,
| RouterImageModel::Seedream5p0Pro
| RouterImageModel::Seedream5p0ProUltra => RouterProvider::Seedance2Pro,
_ => RouterProvider::Fal,
}
}
@@ -172,9 +177,10 @@ async fn finalize_and_generate(
draft_or_request: ImageGenerationDraftOrRequest,
server_state: &ServerState,
resolved_media: &MediaFilesAsCdnUrlListAndMap,
kinovi_account: KinoviAccount,
) -> Result<GenerateImageResponse, CommonWebError> {
let provider = draft_or_request.get_provider();
let client = build_router_client(provider, server_state)?;
let client = build_router_client(provider, server_state, kinovi_account)?;
// Inline the draft finalization so the draft context can borrow `client`.
let request: ImageGenerationRequest = match draft_or_request {
@@ -204,6 +210,7 @@ async fn finalize_and_generate(
fn build_router_client(
provider: RouterProvider,
server_state: &ServerState,
kinovi_account: KinoviAccount,
) -> Result<RouterClient, CommonWebError> {
match provider {
RouterProvider::Fal => {
@@ -214,11 +221,12 @@ fn build_router_client(
Ok(RouterClient::Fal(fal_client))
},
RouterProvider::Seedance2Pro => {
// Midjourney image generation always uses the Volcengine account.
// Per-request account selection (like the video flow's
// `KinoviAccount` enum) isn't needed here yet — if it ever is, plumb
// a parallel knob through `RunPipelineV2Args`.
let cookies = server_state.inference_providers.seedance2pro.cookies_volcengine.clone();
let seedance2pro = &server_state.inference_providers.seedance2pro;
let cookies = match kinovi_account {
KinoviAccount::Volcengine => seedance2pro.cookies_volcengine.clone(),
KinoviAccount::BytePlus => seedance2pro.cookies_byteplus.clone(),
KinoviAccount::BytePlusUltra => seedance2pro.cookies_byteplus_ultra.clone(),
};
let session = Seedance2ProSession::from_cookies_string(cookies);
Ok(RouterClient::Seedance2Pro(RouterSeedance2ProClient::new(session)))
},
@@ -21,7 +21,7 @@ use crate::http_server::endpoints::omni_gen::generate::mesh::helpers::pipeline_r
use crate::http_server::endpoints::omni_gen::generate::mesh::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::bill_wallet::bill_wallet;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::build_router_client::build_router_client;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
@@ -22,7 +22,7 @@ use crate::http_server::endpoints::omni_gen::generate::splat::helpers::pipeline_
use crate::http_server::endpoints::omni_gen::generate::splat::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::bill_wallet::bill_wallet;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::build_router_client::build_router_client;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::splat::map_worldlabs_router_error::map_worldlabs_router_error;
use crate::state::server_state::ServerState;
@@ -11,7 +11,7 @@ use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use worldlabs_api_client::credentials::world_labs_api_creds::WorldLabsApiCreds;
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::state::server_state::ServerState;
pub fn build_router_client(
@@ -2,4 +2,3 @@ pub mod omni_gen_video_generate_handler;
pub mod helpers;
pub mod pipeline_v2;
pub mod insert_db_job;
pub mod kinovi_account;
@@ -40,7 +40,7 @@ use crate::http_server::endpoints::omni_gen::generate::video::insert_db_job::ins
use crate::http_server::endpoints::omni_gen::generate::video::insert_db_job::insert_grok_api_job::{insert_grok_api_job, InsertGrokApiJobArgs};
use crate::http_server::endpoints::omni_gen::generate::video::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
use crate::http_server::endpoints::omni_gen::generate::video::insert_db_job::shared_job_args::SharedJobArgs;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::generate::video::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
use crate::http_server::endpoints::omni_gen::shared_utils::video::validate_video_request::validate_video_request;
use crate::http_server::user_lookup::user_session::session_utils::lookup::user_session_feature_flags::UserSessionFeatureFlags;
@@ -23,7 +23,7 @@ use crate::http_server::endpoints::omni_gen::generate::video::helpers::build_rou
use crate::http_server::endpoints::omni_gen::generate::video::helpers::pipeline_result::PipelineResult;
use crate::http_server::endpoints::omni_gen::generate::video::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
use mysql_queries::queries::generic_inference::common::job_cost_estimates::JobCostEstimates;
use crate::http_server::endpoints::omni_gen::generate::video::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::kinovi_account::KinoviAccount;
use crate::http_server::endpoints::omni_gen::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
use crate::state::server_state::ServerState;
@@ -1,4 +1,5 @@
pub mod audio;
pub mod kinovi_account;
pub mod map_router_cost_error;
pub mod map_seedance2pro_router_error;
pub mod mesh;
@@ -35,6 +35,7 @@ use crate::http_server::endpoints::moderation::user::moderator_list_users_by_sig
use crate::http_server::endpoints::moderation::user::moderator_user_lookup_by_stripe_customer_id_handler::moderator_user_lookup_by_stripe_customer_id_handler;
use crate::http_server::endpoints::moderation::user::moderator_user_lookup_handler::moderator_user_lookup_handler;
use crate::http_server::endpoints::moderation::alerts::moderation_send_alert_handler::moderation_send_alert_handler;
use crate::http_server::endpoints::moderation::dashboards::moderator_list_databox_dashboards_handler::moderator_list_databox_dashboards_handler;
use crate::http_server::endpoints::moderation::staff_audit_logs::moderator_list_staff_audit_logs_handler::moderator_list_staff_audit_logs_handler;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_all_debug_logs_handler::moderation_list_all_debug_logs_handler;
use crate::http_server::endpoints::moderation::debug_logs::moderation_list_debug_logs_for_token_handler::moderation_list_debug_logs_for_token_handler;
@@ -106,6 +107,12 @@ pub fn add_moderator_routes<T, B> (app: App<T>) -> App<T>
.route(web::head().to(|| HttpResponse::Ok()))
)
)
.service(web::scope("/dashboards")
.service(web::resource("/databox")
.route(web::get().to(moderator_list_databox_dashboards_handler))
.route(web::head().to(|| HttpResponse::Ok()))
)
)
.service(web::scope("/debug_logs")
.service(web::resource("/list/{token}")
.route(web::get().to(moderation_list_debug_logs_for_token_handler))
@@ -101,10 +101,6 @@ impl UserSessionFeatureFlags {
pub fn can_use_referrals_program(&self) -> bool {
self.feature_flags.contains(&UserFeatureFlag::ReferralsProgram)
}
pub fn can_create_api_key(&self) -> bool {
self.feature_flags.contains(&UserFeatureFlag::ApiKey)
}
}
#[cfg(test)]
@@ -34,6 +34,7 @@ use crate::startup::setup_bans::{
load_static_container_ip_bans, load_troll_user_token_bans,
};
use crate::startup::setup_inference_providers::setup_inference_providers;
use crate::startup::setup_dashboards::setup_dashboards;
use crate::startup::setup_seedance_video_bucket::setup_seedance_video_bucket;
use crate::startup::setup_static_feature_flags::setup_static_feature_flags;
use crate::startup::setup_stripe_artcraft::setup_stripe_artcraft;
@@ -262,6 +263,7 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
},
temp_dir_creator: ScopedTempDirCreator::auto_setup(),
google_sign_in_cert: GoogleSignInCert::new(),
dashboards: setup_dashboards(),
};
Ok(SetupResult {
@@ -1,5 +1,6 @@
pub mod build_dependencies;
pub mod setup_bans;
pub mod setup_dashboards;
pub mod setup_metrics;
pub mod setup_pager;
pub mod setup_seedance_video_bucket;
@@ -0,0 +1,38 @@
//! Optional setup for the admin dashboard's embedded metrics dashboards.
//!
//! Reads the `DATABOX_DASHBOARD_*` env vars. Each is independent and
//! optional — a missing var means that dashboard is simply not offered to
//! the admin dashboard. Never blocks startup.
use std::env;
use log::info;
use crate::state::server_state::{Dashboards, DataboxDashboards};
/// Build the dashboard configuration from the environment.
pub fn setup_dashboards() -> Dashboards {
let daus_id = read_non_empty_env("DATABOX_DASHBOARD_DAUS");
let daily_generations_id = read_non_empty_env("DATABOX_DASHBOARD_DAILY_GENERATIONS");
info!(
"Databox dashboards configured: daus={}, daily_generations={}",
daus_id.is_some(),
daily_generations_id.is_some(),
);
Dashboards {
databox: DataboxDashboards {
daus_id,
daily_generations_id,
},
}
}
/// Read an env var, treating unset or whitespace-only as absent.
fn read_non_empty_env(name: &str) -> Option<String> {
match env::var(name) {
Ok(value) if !value.trim().is_empty() => Some(value.trim().to_string()),
_ => None,
}
}
@@ -114,6 +114,10 @@ pub struct ServerState {
pub google_sign_in_cert: GoogleSignInCert,
pub temp_dir_creator: ScopedTempDirCreator,
/// Embedded metrics dashboards for the admin dashboard, configured from
/// optional env vars at startup.
pub dashboards: Dashboards,
}
#[derive(Clone)]
@@ -276,3 +280,17 @@ pub struct TrollBans {
pub struct ResendData {
pub api_key: String,
}
/// Embedded metrics dashboards surfaced to admins/mods.
#[derive(Clone)]
pub struct Dashboards {
pub databox: DataboxDashboards,
}
/// Databox datawall IDs. Each is `None` when its env var isn't configured,
/// in which case the dashboard is simply omitted from the moderation API.
#[derive(Clone)]
pub struct DataboxDashboards {
pub daus_id: Option<String>,
pub daily_generations_id: Option<String>,
}
@@ -116,8 +116,12 @@ export function getMediaThumbnailUrl(
}
}
// 3D/dimensional: use cover image URL
if (item.media_class === "dimensional") {
// 3D (mesh/splat, plus legacy pre-split "dimensional"): use cover image URL
if (
item.media_class === "mesh" ||
item.media_class === "splat" ||
item.media_class === "dimensional"
) {
if ("cover_image" in item && item.cover_image) {
if (item.cover_image.maybe_cover_image_public_bucket_url) {
return item.cover_image.maybe_cover_image_public_bucket_url;
@@ -57,6 +57,12 @@ export interface UserReferralListItem {
maybe_referral_code_token: string | null;
}
/** A Databox datawall configured server-side for the admin home page. */
export interface DataboxDashboard {
name: string;
id: string;
}
/** Which table the customer id was found in. */
export type ModeratorStripeCustomerIdSource = "customer_link" | "subscription";
@@ -475,6 +481,25 @@ export class ModerationApi extends ApiManager {
}));
}
// Databox metrics dashboards configured server-side, in display order.
// Dashboards without a configured id are omitted by the server.
public async ListDataboxDashboards(): Promise<
ApiResponse<DataboxDashboard[]>
> {
const endpoint = `${this.getApiSchemeAndHost()}/v1/moderation/dashboards/databox`;
return await this.get<{
success: boolean;
databoards: DataboxDashboard[];
error_message?: string;
}>({ endpoint })
.then((response) => ({
success: response.success,
data: response.databoards || [],
errorMessage: response.error_message,
}))
.catch((err) => ({ success: false, errorMessage: err.message }));
}
public async SendAlert(params: {
title?: string | null;
description?: string | null;
@@ -169,60 +169,6 @@ export class UsersApi extends ApiManager {
}));
}
public async RequestPasswordReset({
email,
}: {
email: string;
}): Promise<ApiResponse<null>> {
const endpoint = `${this.getApiSchemeAndHost()}/v1/request_password_reset`;
return await this.post<
{ email: string },
{ success: boolean; error_message?: string }
>({
endpoint,
body: { email },
})
.then((response) => ({
success: response.success,
errorMessage: response.error_message,
}))
.catch((err) => ({
success: false,
errorMessage: err.message,
}));
}
public async ResetPassword({
token,
password,
passwordConfirmation,
}: {
token: string;
password: string;
passwordConfirmation: string;
}): Promise<ApiResponse<null>> {
const endpoint = `${this.getApiSchemeAndHost()}/v1/reset_password`;
return await this.post<
{ token: string; password: string; password_confirmation: string },
{ success: boolean; error_message?: string }
>({
endpoint,
body: {
token,
password,
password_confirmation: passwordConfirmation,
},
})
.then((response) => ({
success: response.success,
errorMessage: response.error_message,
}))
.catch((err) => ({
success: false,
errorMessage: err.message,
}));
}
public async ChangePassword({
password,
passwordConfirmation,
@@ -28,6 +28,8 @@ interface MediaCardProps {
const classIcons: Record<string, typeof IconPhoto> = {
video: IconVideo,
dimensional: IconCube,
mesh: IconCube,
splat: IconCube,
};
function formatDuration(ms: number): string {
@@ -41,11 +41,15 @@ const classLabels: Record<string, string> = {
image: "Image",
video: "Video",
dimensional: "3D",
mesh: "3D Object",
splat: "3D World",
};
const classIcons: Record<string, typeof IconPhoto> = {
video: IconVideo,
dimensional: IconCube,
mesh: IconCube,
splat: IconCube,
};
function formatModelType(model: string): string {
@@ -1,78 +1,31 @@
import { Bar, BarChart, CartesianGrid, XAxis, YAxis } from "recharts";
import {
ChartContainer,
ChartTooltip,
ChartTooltipContent,
type ChartConfig,
} from "@/components/ui/chart";
import { Card, CardContent, CardHeader, CardTitle } from "@/components/ui/card";
import { IconChartBar, IconDashboard, IconWorld } from "@tabler/icons-react";
import { useEffect, useState } from "react";
import { IconDashboard } from "@tabler/icons-react";
import { ModerationApi, type DataboxDashboard } from "@/api/ModerationApi";
import { Skeleton } from "@/components/ui/skeleton";
import { usePageTitle } from "@/hooks/usePageTitle";
const activeUsersData = [
{ date: "02/05", activeUsers: 812 },
{ date: "02/06", activeUsers: 945 },
{ date: "02/07", activeUsers: 1023 },
{ date: "02/08", activeUsers: 876 },
{ date: "02/09", activeUsers: 654 },
{ date: "02/10", activeUsers: 598 },
{ date: "02/11", activeUsers: 901 },
{ date: "02/12", activeUsers: 1102 },
{ date: "02/13", activeUsers: 1045 },
{ date: "02/14", activeUsers: 978 },
{ date: "02/15", activeUsers: 1134 },
{ date: "02/16", activeUsers: 756 },
{ date: "02/17", activeUsers: 689 },
{ date: "02/18", activeUsers: 1023 },
{ date: "02/19", activeUsers: 1187 },
{ date: "02/20", activeUsers: 1056 },
{ date: "02/21", activeUsers: 998 },
{ date: "02/22", activeUsers: 1210 },
{ date: "02/23", activeUsers: 834 },
{ date: "02/24", activeUsers: 712 },
{ date: "02/25", activeUsers: 967 },
{ date: "02/26", activeUsers: 1089 },
{ date: "02/27", activeUsers: 1145 },
{ date: "02/28", activeUsers: 1034 },
{ date: "03/01", activeUsers: 1267 },
{ date: "03/02", activeUsers: 890 },
{ date: "03/03", activeUsers: 756 },
{ date: "03/04", activeUsers: 1123 },
{ date: "03/05", activeUsers: 1198 },
{ date: "03/06", activeUsers: 704 },
];
const countryData = [
{ country: "United States", activeUsers: 704 },
{ country: "India", activeUsers: 105 },
{ country: "Canada", activeUsers: 100 },
{ country: "China", activeUsers: 74 },
{ country: "Brazil", activeUsers: 59 },
{ country: "Argentina", activeUsers: 57 },
{ country: "Australia", activeUsers: 51 },
];
const activeUsersConfig = {
activeUsers: {
label: "Active Users",
color: "hsl(221, 83%, 53%)",
},
} satisfies ChartConfig;
const countryConfig = {
activeUsers: {
label: "Active Users",
color: "hsl(142, 71%, 45%)",
},
} satisfies ChartConfig;
const totalActiveUsers = activeUsersData.reduce(
(sum, d) => sum + d.activeUsers,
0,
);
function databoxEmbedUrl(id: string): string {
return `https://app.databox.com/datawall/${encodeURIComponent(id)}?i`;
}
export function DashboardHome() {
usePageTitle("Dashboard");
const [databoards, setDataboards] = useState<DataboxDashboard[]>([]);
const [isLoading, setIsLoading] = useState(true);
useEffect(() => {
let cancelled = false;
const api = new ModerationApi();
api.ListDataboxDashboards().then((resp) => {
if (cancelled) return;
setDataboards(resp.success && resp.data ? resp.data : []);
setIsLoading(false);
});
return () => {
cancelled = true;
};
}, []);
return (
<div className="flex flex-col gap-6">
<div className="flex flex-col gap-2">
@@ -83,87 +36,27 @@ export function DashboardHome() {
<p className="text-muted-foreground">Welcome to the admin dashboard.</p>
</div>
<div className="grid gap-4 lg:grid-cols-5">
{/* Active Users Over Time */}
<Card className="lg:col-span-3">
<CardHeader className="flex flex-row items-center justify-between pb-2">
<div className="flex items-center gap-2">
<IconChartBar className="size-4 text-muted-foreground" />
<CardTitle className="text-sm font-medium">
Active Users (DUMMY CONTENT RIGHT NOW)
</CardTitle>
</div>
<span className="text-2xl font-bold tabular-nums">
{totalActiveUsers.toLocaleString()}
</span>
</CardHeader>
<CardContent>
<ChartContainer
config={activeUsersConfig}
className="aspect-auto h-[300px] w-full"
{isLoading ? (
<Skeleton className="w-full rounded-xl" style={{ paddingTop: "63%" }} />
) : (
databoards.map((databoard) => (
<div key={databoard.id} className="flex flex-col gap-2">
<h2 className="text-lg font-semibold">{databoard.name}</h2>
<div
className="relative w-full overflow-hidden rounded-xl border"
style={{ paddingTop: "63%" }}
>
<BarChart data={activeUsersData}>
<CartesianGrid vertical={false} />
<XAxis
dataKey="date"
tickLine={false}
axisLine={false}
tickMargin={8}
interval="preserveStartEnd"
/>
<YAxis tickLine={false} axisLine={false} tickMargin={8} />
<ChartTooltip
content={<ChartTooltipContent hideLabel={false} />}
/>
<Bar
dataKey="activeUsers"
fill="var(--color-activeUsers)"
radius={[4, 4, 0, 0]}
/>
</BarChart>
</ChartContainer>
</CardContent>
</Card>
{/* Users by Country */}
<Card className="lg:col-span-2">
<CardHeader className="pb-2">
<div className="flex items-center gap-2">
<IconWorld className="size-4 text-muted-foreground" />
<CardTitle className="text-sm font-medium">
Users by Country (DUMMY CONTENT RIGHT NOW)
</CardTitle>
<iframe
src={databoxEmbedUrl(databoard.id)}
title={databoard.name}
className="absolute inset-0 h-full w-full"
frameBorder="0"
allowFullScreen
/>
</div>
</CardHeader>
<CardContent>
<ChartContainer
config={countryConfig}
className="aspect-auto h-[300px] w-full"
>
<BarChart data={countryData} layout="vertical">
<CartesianGrid horizontal={false} />
<XAxis type="number" tickLine={false} axisLine={false} />
<YAxis
type="category"
dataKey="country"
tickLine={false}
axisLine={false}
tickMargin={8}
width={100}
/>
<ChartTooltip
content={<ChartTooltipContent hideLabel={false} />}
/>
<Bar
dataKey="activeUsers"
fill="var(--color-activeUsers)"
radius={[0, 4, 4, 0]}
/>
</BarChart>
</ChartContainer>
</CardContent>
</Card>
</div>
</div>
))
)}
</div>
);
}
@@ -26,7 +26,8 @@ const MEDIA_CLASS_OPTIONS = [
{ value: "", label: "All" },
{ value: "image", label: "Image" },
{ value: "video", label: "Video" },
{ value: "dimensional", label: "3D" },
{ value: "mesh", label: "Meshes" },
{ value: "splat", label: "Splats" },
] as const;
const PAGE_SIZE = 24;
@@ -74,7 +75,8 @@ export function ExploreMedia() {
const resp = await api.ListMediaFiles({
page_size: PAGE_SIZE,
cursor,
filter_media_classes: filterMediaClasses || "image,video,dimensional",
filter_media_classes:
filterMediaClasses || "image,video,mesh,splat,dimensional",
filter_media_type: filterMediaType || undefined,
include_user_uploads: includeUserUploads || undefined,
});
@@ -46,7 +46,7 @@ export function UserCreations() {
const resp = await mediaApi.ListUserMediaFiles(username, {
page_index: pageIndex,
page_size: PAGE_SIZE,
filter_media_classes: ["image", "video", "dimensional"],
filter_media_classes: ["image", "video", "mesh", "splat", "dimensional"],
});
if (cancelledRef.current) return;
@@ -84,7 +84,7 @@ export function UserCreations() {
mediaApi.ListUserMediaFiles(username, {
page_index: 0,
page_size: PAGE_SIZE,
filter_media_classes: ["image", "video", "dimensional"],
filter_media_classes: ["image", "video", "mesh", "splat", "dimensional"],
}),
]);
@@ -174,7 +174,7 @@ export function UserProfile() {
modApi.UserLookup(username),
mediaApi.ListUserMediaFiles(username, {
page_size: 24,
filter_media_classes: ["image", "video", "dimensional"],
filter_media_classes: ["image", "video", "mesh", "splat", "dimensional"],
}),
]);
@@ -3,6 +3,7 @@
"version": "0.0.1",
"private": true,
"dependencies": {
"jszip": "^3.10.1",
"react-aria": "^3.48.0",
"react-masonry-css": "^1.0.16"
}
+13 -7
View File
@@ -18,6 +18,7 @@ import CreateAudio from "../pages/create-audio";
import CreateObject from "../pages/create-object";
import CreateWorld from "../pages/create-world";
import CreateVFX from "../pages/create-vfx";
import FrameExtractor from "../pages/frame-extractor";
import PageScene from "../pages/pagescene";
import PageDraw from "../pages/pagedraw";
import VideoEditorPage from "../pages/video-editor";
@@ -160,15 +161,11 @@ export function App() {
<Route path="/create-object" element={<CreateObject />} />
<Route path="/create-world" element={<CreateWorld />} />
<Route path="/background-change" element={<CreateVFX />} />
<Route path="/frame-extractor" element={<FrameExtractor />} />
<Route path="/moodboard" element={<MoodboardPage />} />
<Route path="/edit-3d" element={<PageScene />} />
<Route path="/edit-3d/:sceneToken" element={<PageScene />} />
<Route path="/edit-image" element={<PageDraw />} />
<Route path="/video-editor" element={<VideoEditorPage />} />
<Route
path="/video-editor/:projectId"
element={<VideoEditorPage />}
/>
<Route path="/support" element={<Support />} />
<Route path="/pricing" element={<Pricing />} />
{/* Welcome is public so it stays reachable right after signup
@@ -179,12 +176,21 @@ export function App() {
{/* Protected — sign-in required (user-owned content / billing flows) */}
<Route element={<RequireAuth />}>
{/* Video projects persist server-side, so the whole editor is
sign-in gated: logged-out visits bounce to /login?from=...
and return here after auth. */}
<Route path="/video-editor" element={<VideoEditorPage />} />
<Route
path="/video-editor/:projectId"
element={<VideoEditorPage />}
/>
<Route path="/media" element={<Media />} />
<Route path="/media/:id" element={<Media />} />
<Route path="/library" element={<Library />} />
<Route path="/library/folders" element={<Library />} />
{/* `:slug` is a media-class filter (images/videos/meshes) OR a
folder token (prefixed `folder_`); the page disambiguates. */}
{/* `:slug` is a media-class filter (images/videos/meshes), a
folder token (prefixed `folder_`), the static `tags` tab, or
a tag token (prefixed `tag_`); the page disambiguates. */}
<Route path="/library/:slug" element={<Library />} />
<Route path="/referrals" element={<Referrals />} />
<Route path="/onboarding" element={<Onboarding />} />
@@ -7,7 +7,11 @@ import { TabSelector } from "@storyteller/ui-tab-selector";
import { TruchetPattern } from "@storyteller/ui-vfx";
import Seo from "../../components/seo";
import { useIsMobile } from "../ui/use-mobile";
import { GalleryViewToggle } from "./GalleryViewToggle";
import {
GalleryAutoplayToggle,
GallerySelectToggle,
GalleryViewToggle,
} from "./GalleryViewToggle";
import {
MobileCreateTabsProvider,
type MobileCreateTab,
@@ -23,6 +27,12 @@ interface CreateMediaPageShellProps {
authChecked: boolean;
// Content
hasContent: boolean;
// Video page only: playing/still toggle for video preview thumbnails,
// rendered next to the grid/list toggle on the mobile History tab.
showAutoplayToggle?: boolean;
// Multi-select + batch download toggle, rendered in the same mobile
// History-tab cluster (the desktop toggle lives in the TopBar).
showSelectToggle?: boolean;
emptyStateTitle: string;
emptyStateSubtitle: string;
// Optional CTA rendered under the empty-state subtitle (e.g. a signup button
@@ -54,6 +64,8 @@ export function CreateMediaPageShell({
description,
authChecked,
hasContent,
showAutoplayToggle = false,
showSelectToggle = false,
emptyStateTitle,
emptyStateSubtitle,
emptyStateCta,
@@ -92,18 +104,26 @@ export function CreateMediaPageShell({
<div className="flex h-full w-full flex-col bg-[#101014] text-white">
<Seo title={title} description={description} />
<div className="relative flex items-center justify-center border-b border-ui-panel-border px-3 py-2">
{/* Flex (not absolute) keeps the toggle cluster from overlapping
the tabs — the tabs stay centered via the equal flex-1 sides and
just shift left if the cluster outgrows its half. */}
<div className="flex items-center border-b border-ui-panel-border px-3 py-2">
<div className="flex-1" />
<TabSelector
tabs={MOBILE_TABS}
activeTab={mobileTab}
onTabChange={(id) => setMobileTab(id as MobileCreateTab)}
className="w-auto"
className="w-auto shrink-0"
/>
{mobileTab === "history" && hasContent && (
<div className="absolute right-3 top-1/2 -translate-y-1/2">
<GalleryViewToggle />
</div>
)}
<div className="flex flex-1 items-center justify-end gap-1.5">
{mobileTab === "history" && hasContent && (
<>
{showSelectToggle && <GallerySelectToggle />}
{showAutoplayToggle && <GalleryAutoplayToggle />}
<GalleryViewToggle />
</>
)}
</div>
</div>
<div className="min-h-0 flex-1 overflow-hidden">
@@ -0,0 +1,101 @@
import { useMemo, useState } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faArrowDownToLine,
faSpinnerThird,
} from "@fortawesome/pro-solid-svg-icons";
import {
SelectionActionBar,
useGallerySelectionStore,
type GalleryItem,
} from "@storyteller/ui-generation-list";
import { toast } from "../toast/toast";
import { downloadItemsAsZip } from "../../lib/download-media-zip";
import { useIsMobile } from "../ui/use-mobile";
// Floating bar shown while the generation feed is in select mode:
// "Download selected" zips the chosen items client-side. Centered within the
// content area next to the sidebar; on desktop it sits above the floating
// promptbox via the measured `bottomOffset`, on mobile (History tab — no
// promptbox) it uses a fixed offset like the library's bulk bar.
export function GallerySelectionDownloadBar({
allItems,
bottomOffset,
}: {
allItems: GalleryItem[];
bottomOffset?: number;
}) {
const isMobile = useIsMobile();
const ids = useGallerySelectionStore((s) => s.ids);
const setActive = useGallerySelectionStore((s) => s.setActive);
const [progress, setProgress] = useState<{
done: number;
total: number;
} | null>(null);
const isDownloading = progress !== null;
const selectedItems = useMemo(() => {
const byId = new Map(allItems.map((it) => [it.id, it] as const));
return Array.from(ids)
.map((id) => byId.get(id))
.filter(
(it): it is GalleryItem & { fullImage: string } => !!it?.fullImage,
);
}, [allItems, ids]);
const handleDownload = async () => {
if (isDownloading || selectedItems.length === 0) return;
setProgress({ done: 0, total: selectedItems.length });
try {
const { succeeded, failed } = await downloadItemsAsZip(
selectedItems.map((it) => ({
id: it.id,
url: it.fullImage,
mediaClass: it.mediaClass,
})),
{ onProgress: (done, total) => setProgress({ done, total }) },
);
if (failed === 0) {
toast.success(
`Downloaded ${succeeded} ${succeeded === 1 ? "file" : "files"}`,
);
} else if (succeeded > 0) {
toast.error(
`${failed} of ${succeeded + failed} files failed to download`,
);
} else {
toast.error("Could not download the selected files.");
}
if (succeeded > 0) setActive(false);
} finally {
setProgress(null);
}
};
return (
<SelectionActionBar
className={isMobile ? "bottom-20" : ""}
// Center within the content area (viewport minus the app sidebar); on
// desktop, sit above the fixed promptbox whatever its measured height.
style={{
left: "var(--ac-sidebar-offset, 0px)",
...(isMobile ? {} : { bottom: bottomOffset ?? 16 }),
}}
>
<button
type="button"
onClick={handleDownload}
disabled={isDownloading || selectedItems.length === 0}
className="flex items-center gap-2 rounded-full bg-ui-controls/60 px-3 py-1.5 text-sm font-medium text-white transition-colors hover:bg-ui-controls/90 focus:outline-none focus-visible:ring-2 focus-visible:ring-primary disabled:opacity-60"
>
<FontAwesomeIcon
icon={isDownloading ? faSpinnerThird : faArrowDownToLine}
className={`text-xs ${isDownloading ? "animate-spin" : ""}`}
/>
{isDownloading && progress
? `Downloading ${progress.done}/${progress.total}…`
: "Download"}
</button>
</SelectionActionBar>
);
}
@@ -1,2 +1,6 @@
// Moved to the shared generation-list lib (used by both webapp and desktop).
export { GalleryViewToggle } from "@storyteller/ui-generation-list";
export {
GalleryViewToggle,
GalleryAutoplayToggle,
GallerySelectToggle,
} from "@storyteller/ui-generation-list";
@@ -1,6 +1,9 @@
import { useEffect, useMemo } from "react";
import { useGallerySelectionStore } from "@storyteller/ui-generation-list";
import { useGalleryViewStore } from "../../lib/gallery-view-store";
import { GenerationGalleryGrid } from "./GenerationGalleryGrid";
import { GenerationGalleryList } from "./GenerationGalleryList";
import { GallerySelectionDownloadBar } from "./GallerySelectionDownloadBar";
import type { GenerationGalleryProps } from "./types";
// Renders the generation feed in whichever layout the user picked via the
@@ -8,9 +11,34 @@ import type { GenerationGalleryProps } from "./types";
// rendering the grid directly on the create pages.
export function GenerationGallery(props: GenerationGalleryProps) {
const viewMode = useGalleryViewStore((s) => s.viewMode);
return viewMode === "list" ? (
<GenerationGalleryList {...props} />
) : (
<GenerationGalleryGrid {...props} />
const { selectable, selectionBarBottomOffset, newlyCompletedItems, galleryItems } =
props;
// The selection store is global (like the view-mode store) — drop select
// mode when the feed unmounts so it can't leak onto other pages.
useEffect(() => {
if (!selectable) return;
return () => useGallerySelectionStore.getState().setActive(false);
}, [selectable]);
const allItems = useMemo(
() => [...newlyCompletedItems, ...galleryItems],
[newlyCompletedItems, galleryItems],
);
return (
<>
{viewMode === "list" ? (
<GenerationGalleryList {...props} />
) : (
<GenerationGalleryGrid {...props} />
)}
{selectable && (
<GallerySelectionDownloadBar
allItems={allItems}
bottomOffset={selectionBarBottomOffset}
/>
)}
</>
);
}
@@ -13,6 +13,7 @@ import {
type GalleryItem,
is3DMediaClass,
type RecreateSlotContext,
useGallerySelectionStore,
} from "@storyteller/ui-generation-list";
import { useRecreateFromPromptToken } from "../../lib/recreate";
import { useGalleryItemActions } from "./useGalleryItemActions";
@@ -36,7 +37,10 @@ export function GenerationGalleryGrid({
onLoadMore,
onGalleryItemClick,
enableMakeVideo,
selectable,
}: GenerationGalleryProps) {
const selectionActive = useGallerySelectionStore((s) => s.active);
const selectedIds = useGallerySelectionStore((s) => s.ids);
return (
<GenerationGridView
inProgressJobs={inProgressJobs}
@@ -54,10 +58,17 @@ export function GenerationGalleryGrid({
renderGalleryActions={(item) => (
<CardActions item={item} enableMakeVideo={enableMakeVideo} />
)}
selectionMode={!!selectable && selectionActive}
selectedIds={selectedIds}
onToggleSelect={handleToggleSelect}
/>
);
}
function handleToggleSelect(item: GalleryItem) {
useGallerySelectionStore.getState().toggle(item.id);
}
// Recreate affordance for pending + failed cards. The pending card shows a
// hover-revealed icon button by the prompt; the failed card a labeled button
// next to Dismiss.
@@ -13,6 +13,7 @@ import {
type GalleryItem,
is3DMediaClass,
type RecreateSlotContext,
useGallerySelectionStore,
} from "@storyteller/ui-generation-list";
import { toast } from "../toast/toast";
import { useRecreateFromPromptToken } from "../../lib/recreate";
@@ -40,7 +41,10 @@ export function GenerationGalleryList({
onLoadMore,
onGalleryItemClick,
enableMakeVideo,
selectable,
}: GenerationGalleryProps) {
const selectionActive = useGallerySelectionStore((s) => s.active);
const selectedIds = useGallerySelectionStore((s) => s.ids);
return (
<GenerationListView
inProgressJobs={inProgressJobs}
@@ -63,10 +67,17 @@ export function GenerationGalleryList({
/>
)}
onCopyPromptResult={handleCopyResult}
selectionMode={!!selectable && selectionActive}
selectedIds={selectedIds}
onToggleSelect={handleToggleSelect}
/>
);
}
function handleToggleSelect(item: GalleryItem) {
useGallerySelectionStore.getState().toggle(item.id);
}
function handleCopyResult(success: boolean) {
if (success) {
toast.success("Prompt copied");
@@ -1,7 +1,7 @@
export { GenerationGallery } from "./GenerationGallery";
export { GenerationGalleryGrid } from "./GenerationGalleryGrid";
export { GenerationGalleryList } from "./GenerationGalleryList";
export { GalleryViewToggle } from "./GalleryViewToggle";
export { GalleryViewToggle, GalleryAutoplayToggle } from "./GalleryViewToggle";
export { useGalleryData, type GalleryItem } from "./useGalleryData";
export { useSessionMediaByClass } from "@storyteller/ui-generation-list";
export {
@@ -16,4 +16,10 @@ export interface GenerationGalleryProps {
onLoadMore: () => void;
onGalleryItemClick: (item: GalleryItem) => void;
enableMakeVideo?: boolean;
/** Enables the multi-select + batch download flow (select toggle in the
* TopBar, checkboxes on completed items, floating download bar). */
selectable?: boolean;
/** Pixel offset that keeps the floating download bar above the page's
* fixed prompt box (pass the measured promptbox height + gap). */
selectionBarBottomOffset?: number;
}
@@ -15,12 +15,12 @@ import {
faLink,
faPause,
faPencil,
faPhotoFilm,
faPlay,
faSpinnerThird,
faTrashCan,
faUser,
faVideo,
faWaveformLines,
faXmark,
} from "@fortawesome/pro-solid-svg-icons";
import {
@@ -34,6 +34,7 @@ import {
getProviderIconByName,
} from "@storyteller/model-list";
import { downloadMediaFile } from "../../lib/download-media";
import { TagsSection } from "../tags/TagsSection";
import {
formatAspectRatio,
formatDuration,
@@ -76,6 +77,7 @@ export interface LightboxDetailsProps {
onRecreate?: () => void;
onMakeVideo?: () => void;
onEditOnCanvas?: () => void;
onExtractFrames?: () => void;
onDelete?: () => void;
showDownloadAppCta?: boolean;
}
@@ -92,6 +94,7 @@ export function LightboxDetails({
onRecreate,
onMakeVideo,
onEditOnCanvas,
onExtractFrames,
onDelete,
showDownloadAppCta,
}: LightboxDetailsProps) {
@@ -430,6 +433,8 @@ export function LightboxDetails({
</div>
</div>
)}
<TagsSection mediaToken={mediaToken} creator={creator} />
</>
)}
</div>
@@ -492,6 +497,16 @@ export function LightboxDetails({
Edit on Canvas
</Button>
)}
{onExtractFrames && (
<Button
icon={faPhotoFilm}
className="w-full border border-ui-panel-border bg-ui-controls/40 hover:bg-ui-controls/60 text-white"
variant="secondary"
onClick={onExtractFrames}
>
Extract Frames
</Button>
)}
{onRecreate && promptData.hasToken && (
<Button
icon={faArrowRotateRight}
@@ -32,6 +32,7 @@ import {
applyMakeVideoFromImage,
applyRecreateFromMediaToken,
} from "../../lib/recreate";
import { useLightboxSoundStore } from "../../lib/lightbox-sound-store";
// ── Types ──────────────────────────────────────────────────────────────────
@@ -80,6 +81,7 @@ export function Lightbox({
showBatchCarousel = true,
}: LightboxProps) {
const navigate = useNavigate();
const soundEnabled = useLightboxSoundStore((s) => s.soundEnabled);
const [mediaLoaded, setMediaLoaded] = useState(false);
const [promptData, setPromptData] = useState<PromptData>(EMPTY_PROMPT);
const [creator, setCreator] = useState<UserInfo | null>(null);
@@ -350,6 +352,12 @@ export function Lightbox({
applyEditOnCanvasFromImage(selectedMediaToken, selectedImageUrl, navigate);
}, [selectedMediaToken, selectedImageUrl, canEditOnCanvas, navigate, onClose]);
const handleExtractFrames = useCallback(() => {
if (!selectedMediaToken) return;
onClose();
navigate(`/frame-extractor?media=${selectedMediaToken}`);
}, [selectedMediaToken, navigate, onClose]);
return (
<>
<Modal
@@ -379,7 +387,7 @@ export function Lightbox({
controls
loop
autoPlay
muted
muted={!soundEnabled}
playsInline
disablePictureInPicture
controlsList="nodownload noplaybackrate nofullscreen"
@@ -389,6 +397,22 @@ export function Lightbox({
const el = e.currentTarget;
setMediaWidth(el.videoWidth);
setMediaHeight(el.videoHeight);
// Restore the user's last volume (getState: subscribing
// would re-render on every tick of a volume drag).
el.volume = useLightboxSoundStore.getState().volume;
// If the browser blocked unmuted autoplay, fall back to
// muted playback rather than leaving the video frozen.
if (soundEnabled && el.paused) {
el.play().catch(() => {
el.muted = true;
el.play().catch(() => {});
});
}
}}
onVolumeChange={(e) => {
useLightboxSoundStore
.getState()
.setVolume(e.currentTarget.volume);
}}
ref={(el) => {
if (el) {
@@ -401,11 +425,13 @@ export function Lightbox({
</video>
) : isAudio ? (
<div className="flex h-full w-full items-center justify-center px-4 sm:px-8">
<div className="w-full max-w-xl rounded-2xl border border-white/10 bg-white/[0.04] px-2 py-8">
<FontAwesomeIcon
icon={faMusic}
className="mx-auto mb-5 block text-3xl text-white/20"
/>
<div className="w-full max-w-xl rounded-2xl border border-white/10 bg-gradient-to-br from-white/[0.08] via-white/[0.04] to-white/[0.02] px-4 py-10 sm:px-6">
<div className="mx-auto mb-6 flex h-16 w-16 items-center justify-center rounded-full bg-white/10 ring-1 ring-white/15">
<FontAwesomeIcon
icon={faMusic}
className="text-2xl text-white/70"
/>
</div>
<WaveformAudioPlayer
key={selectedImageUrl}
src={selectedImageUrl}
@@ -539,6 +565,9 @@ export function Lightbox({
? handleEditOnCanvas
: undefined
}
onExtractFrames={
isVideo && selectedMediaToken ? handleExtractFrames : undefined
}
onDelete={
selectedMediaToken
? () => setConfirmDeleteOpen(true)
@@ -20,7 +20,7 @@ import {
import { toast } from "../toast/toast";
import { v4 as uuidv4 } from "uuid";
import { Button } from "@storyteller/ui-button";
import { useCharactersStore } from "./characters-store";
import { useCharactersStore } from "@storyteller/ui-promptbox";
interface CharactersModalProps {
isOpen: boolean;
@@ -89,11 +89,13 @@ export const CharactersModal = ({
setPendingCharacters((prev) => prev.filter((p) => p.name !== name));
}, []);
// Poll the server while the modal is open and any creation is pending, so
// a creation that finishes while the user is on the create/edit view still
// gets cleaned up — preventing the duplicate (real + pending) card.
// Poll the server while any creation is pending (even with the modal
// closed), so the pending card gets cleaned up and — once the character is
// active — the mention store learns its real character token. The create
// response only carries an inference job token, so @-mentions must wait for
// the server list to include the character before it can be referenced.
useEffect(() => {
if (!isOpen || pendingCharacters.length === 0) return;
if (pendingCharacters.length === 0) return;
const interval = setInterval(async () => {
try {
@@ -112,14 +114,25 @@ export const CharactersModal = ({
return true;
}),
);
if (resolved) setRefreshKey((k) => k + 1);
if (resolved) {
const store = useCharactersStore.getState();
store.setCharacters(
res.data.map((c) => ({
character_token: c.token,
name: c.name,
avatar_image_url: c.maybe_avatar?.cdn_url,
})),
);
store.setLoaded(true);
setRefreshKey((k) => k + 1);
}
} catch {
// retry next tick
}
}, POLL_INTERVAL_MS);
return () => clearInterval(interval);
}, [isOpen, pendingCharacters.length]);
}, [pendingCharacters.length]);
// Time out failed creations so the "Creating..." card never sticks forever.
useEffect(() => {
@@ -610,7 +623,6 @@ const NewCharacterView = ({
onBack: () => void;
onCreated: (pending: { name: string; previewUrl?: string }) => void;
}) => {
const addCharacterToStore = useCharactersStore((s) => s.addCharacter);
const [name, setName] = useState("");
const [description, setDescription] = useState("");
const [images, setImages] = useState<UploadedImage[]>([]);
@@ -753,11 +765,6 @@ const NewCharacterView = ({
if (res.success && res.data) {
toast.success(`Character "${name.trim()}" is being created`);
addCharacterToStore({
character_token: res.data.inference_job_token,
name: name.trim(),
avatar_image_url: uploadedImages[0]!.url,
});
onCreated({
name: name.trim(),
previewUrl: uploadedImages[0]!.url,
@@ -10,6 +10,11 @@ import {
} from "@fortawesome/pro-solid-svg-icons";
import { twMerge } from "tailwind-merge";
import { UploaderStates } from "@storyteller/common";
import {
AUDIO_FILE_ACCEPT,
AUDIO_FILE_TYPE_ERROR,
isAudioFile,
} from "@storyteller/ui-promptbox";
import { toast } from "../toast/toast";
import { AddButton } from "./ImagePromptRow";
import type { RefVideo, RefAudio } from "./types";
@@ -32,6 +37,7 @@ interface MediaReferenceRowProps {
onReferenceAudiosChange: (audios: RefAudio[]) => void;
maxAudioCount: number;
maxAudioRefDuration: number;
onPickAudioFromLibrary?: () => void;
className?: string;
}
@@ -47,6 +53,7 @@ export const MediaReferenceRow = ({
onReferenceAudiosChange,
maxAudioCount,
maxAudioRefDuration,
onPickAudioFromLibrary,
className,
}: MediaReferenceRowProps) => {
const videoInputRef = useRef<HTMLInputElement>(null);
@@ -135,6 +142,10 @@ export const MediaReferenceRow = ({
const baseAudios = [...referenceAudios];
const file = files[0];
if (!isAudioFile(file)) {
toast.error(AUDIO_FILE_TYPE_ERROR);
return;
}
const duration = await getAudioDuration(file);
if (duration <= 0) {
@@ -206,12 +217,12 @@ export const MediaReferenceRow = ({
type="file"
ref={audioInputRef}
className="hidden"
accept="audio/*"
accept={AUDIO_FILE_ACCEPT}
onChange={handleAudioUpload}
/>
<div
className={twMerge(
"glass flex flex-col sm:flex-row rounded-2xl sm:rounded-none",
"glass flex flex-col sm:flex-row rounded-2xl",
className,
)}
onMouseDown={(e) => e.stopPropagation()}
@@ -303,7 +314,11 @@ export const MediaReferenceRow = ({
</div>
)}
{canAddAudio && (
<AddButton onUpload={() => audioInputRef.current?.click()} />
<AddButton
onUpload={() => audioInputRef.current?.click()}
onPickFromLibrary={onPickAudioFromLibrary}
title="Add audio"
/>
)}
</div>
</div>
@@ -1,693 +0,0 @@
import {
forwardRef,
useCallback,
useEffect,
useImperativeHandle,
useMemo,
useRef,
useState,
} from "react";
import { twMerge } from "tailwind-merge";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faUserGroup } from "@fortawesome/pro-solid-svg-icons";
import { faVideo, faMusic } from "@fortawesome/pro-regular-svg-icons";
import type { MentionItem } from "./types";
interface MentionTextareaProps {
value: string;
onChange: (value: string) => void;
mentionItems: MentionItem[];
placeholder?: string;
className?: string;
/** Inline style applied to the editable element (e.g. a maxHeight cap or
* resize override for the fullscreen modal). */
style?: React.CSSProperties;
onKeyDown?: (e: React.KeyboardEvent<HTMLDivElement>) => void;
onFocus?: () => void;
onBlur?: () => void;
disabled?: boolean;
colorMap: Record<string, string>;
}
interface MentionState {
isOpen: boolean;
triggerIndex: number;
query: string;
activeIndex: number;
}
function escapeRegex(s: string): string {
return s.replace(/[.*+?^${}()|[\]\\]/g, "\\$&");
}
function escapeHTML(text: string): string {
return text
.replace(/&/g, "&amp;")
.replace(/</g, "&lt;")
.replace(/>/g, "&gt;")
.replace(/"/g, "&quot;");
}
// ---------------------------------------------------------------------------
// Cursor helpers for contentEditable
// ---------------------------------------------------------------------------
function getCaretOffset(el: HTMLElement): number {
try {
const sel = window.getSelection();
if (!sel?.rangeCount || !sel.anchorNode || !el.contains(sel.anchorNode)) {
return 0;
}
const anchorNode = sel.anchorNode;
const anchorOffset = sel.anchorOffset;
let offset = 0;
if (anchorNode === el) {
for (let i = 0; i < anchorOffset; i++) {
const child = el.childNodes[i];
if (!child) break;
if (child.nodeType === Node.TEXT_NODE) {
offset += child.textContent?.length ?? 0;
} else if (child.nodeName === "BR") {
offset += 1;
} else {
offset += child.textContent?.length ?? 0;
}
}
return offset;
}
function countBefore(parent: Node): boolean {
for (const child of Array.from(parent.childNodes)) {
if (child === anchorNode) {
if (child.nodeType === Node.TEXT_NODE) {
offset += anchorOffset;
}
return true;
}
if (child.contains(anchorNode)) {
return countBefore(child);
}
if (child.nodeType === Node.TEXT_NODE) {
offset += child.textContent?.length ?? 0;
} else if (child.nodeName === "BR") {
offset += 1;
} else {
offset += child.textContent?.length ?? 0;
}
}
return false;
}
countBefore(el);
return offset;
} catch {
return 0;
}
}
function setCaretOffset(el: HTMLElement, offset: number) {
try {
let remaining = offset;
function findPosition(parent: Node): boolean {
for (let i = 0; i < parent.childNodes.length; i++) {
const child = parent.childNodes[i];
if (child.nodeType === Node.TEXT_NODE) {
const len = child.textContent?.length ?? 0;
if (remaining <= len) {
const sel = window.getSelection();
const range = document.createRange();
range.setStart(child, remaining);
range.collapse(true);
sel?.removeAllRanges();
sel?.addRange(range);
return true;
}
remaining -= len;
} else if (child.nodeName === "BR") {
if (remaining === 0) {
const sel = window.getSelection();
const range = document.createRange();
range.setStart(parent, i);
range.collapse(true);
sel?.removeAllRanges();
sel?.addRange(range);
return true;
}
remaining -= 1;
} else if (child.nodeType === Node.ELEMENT_NODE) {
if (findPosition(child)) {
return true;
}
}
}
return false;
}
if (!findPosition(el)) {
const sel = window.getSelection();
if (sel) {
sel.selectAllChildren(el);
sel.collapseToEnd();
}
}
} catch {
// DOM changed during caret restore
}
}
function scrollCaretIntoView(el: HTMLElement) {
try {
const sel = window.getSelection();
if (!sel?.rangeCount || !el.contains(sel.anchorNode)) return;
const range = sel.getRangeAt(0).cloneRange();
range.collapse(false);
const span = document.createElement("span");
span.textContent = "\u200B";
range.insertNode(span);
const spanRect = span.getBoundingClientRect();
const elRect = el.getBoundingClientRect();
if (spanRect.bottom > elRect.bottom) {
el.scrollTop += spanRect.bottom - elRect.bottom;
} else if (spanRect.top < elRect.top) {
el.scrollTop -= elRect.top - spanRect.top;
}
const parent = span.parentNode;
if (parent) {
const next = span.nextSibling;
parent.removeChild(span);
const restored = document.createRange();
if (next) {
restored.setStartBefore(next);
} else if (parent.lastChild) {
restored.setStartAfter(parent.lastChild);
} else {
restored.selectNodeContents(parent);
}
restored.collapse(true);
sel.removeAllRanges();
sel.addRange(restored);
}
} catch {
// DOM changed during measurement
}
}
// ---------------------------------------------------------------------------
// Mention Dropdown
// ---------------------------------------------------------------------------
function MentionDropdown({
items,
activeIndex,
onSelect,
onHover,
position,
}: {
items: MentionItem[];
activeIndex: number;
onSelect: (item: MentionItem) => void;
onHover: (index: number) => void;
position: { left: number; bottom: number };
}) {
const listRef = useRef<HTMLDivElement>(null);
useEffect(() => {
const list = listRef.current;
if (!list) return;
const activeItem = list.children[1]?.children[activeIndex] as HTMLElement;
activeItem?.scrollIntoView({ block: "nearest" });
}, [activeIndex]);
return (
<div
ref={listRef}
className="absolute z-50 mb-1 w-64 max-h-72 overflow-y-auto rounded-lg border border-white/10 bg-[#2a2a2e] shadow-lg backdrop-blur-xl"
style={{ left: position.left, bottom: position.bottom }}
>
<div className="px-3 py-1.5 text-[11px] font-semibold uppercase tracking-wider text-white/50">
Mentions
</div>
<div>
{items.map((item, i) => (
<button
key={item.label}
type="button"
className={twMerge(
"flex w-full items-center gap-2.5 px-3 py-2 text-sm text-white transition-colors cursor-pointer",
i === activeIndex ? "bg-white/10" : "hover:bg-white/5",
)}
onMouseDown={(e) => {
e.preventDefault();
onSelect(item);
}}
onMouseEnter={() => onHover(i)}
>
<div className="h-8 w-8 flex-shrink-0 overflow-hidden rounded-md border border-white/20 flex items-center justify-center bg-black/20">
{(item.type === "character" || item.type === "image") && item.preview ? (
<img
src={item.preview}
alt={item.label}
className="h-full w-full object-cover"
/>
) : item.type === "character" ? (
<FontAwesomeIcon
icon={faUserGroup}
className="h-3.5 w-3.5 text-white/60"
/>
) : item.type === "video" && item.preview ? (
<video
src={item.preview}
muted
preload="metadata"
className="h-full w-full object-cover"
/>
) : (
<FontAwesomeIcon
icon={item.type === "video" ? faVideo : faMusic}
className="h-3.5 w-3.5 text-white/60"
/>
)}
</div>
<span className="font-medium">{item.label}</span>
</button>
))}
</div>
</div>
);
}
// ---------------------------------------------------------------------------
// MentionTextarea Component
// ---------------------------------------------------------------------------
export const MentionTextarea = forwardRef<HTMLDivElement, MentionTextareaProps>(
function MentionTextarea(
{
value,
onChange,
mentionItems,
placeholder,
className,
style,
onKeyDown: externalOnKeyDown,
onFocus,
onBlur,
disabled,
colorMap,
},
ref,
) {
const editorRef = useRef<HTMLDivElement>(null);
useImperativeHandle(ref, () => editorRef.current!, []);
const isInternalUpdate = useRef(false);
const isComposing = useRef(false);
const pendingCaret = useRef<number | null>(null);
const [mentionState, setMentionState] = useState<MentionState>({
isOpen: false,
triggerIndex: -1,
query: "",
activeIndex: 0,
});
const [dropdownPos, setDropdownPos] = useState<{
left: number;
bottom: number;
}>({ left: 0, bottom: 0 });
const filteredItems = useMemo(() => {
if (!mentionState.isOpen) return [];
return mentionItems.filter((item) =>
mentionState.query
? item.label.toLowerCase().includes(mentionState.query.toLowerCase())
: true,
);
}, [mentionItems, mentionState.isOpen, mentionState.query]);
// Build a regex that matches any known mention label. Case-insensitive so
// a typed `@image1` still highlights when the canonical label is `@Image1`.
// Sort longest-first so `@Pumpkin Head` matches before `@Pumpkin`.
const mentionRegex = useMemo(() => {
const labels = mentionItems.map((item) => item.label);
if (labels.length === 0) return null;
const sorted = [...labels].sort((a, b) => b.length - a.length);
const pattern = sorted.map((l) => escapeRegex(l)).join("|");
return new RegExp(`(${pattern})`, "gi");
}, [mentionItems]);
// Case-insensitive colorMap lookup: "@image1" finds "@Image1".
const lowerColorMap = useMemo(() => {
const m: Record<string, string> = {};
for (const [k, v] of Object.entries(colorMap)) {
m[k.toLowerCase()] = v;
}
return m;
}, [colorMap]);
// Build innerHTML with colored @mentions inline
const buildHTML = useCallback(
(text: string): string => {
if (!text) return "";
if (!mentionRegex) {
let html = escapeHTML(text);
html = html.replace(/\n/g, "<br>");
if (html.endsWith("<br>")) html += "<br>";
return html;
}
let html = "";
let lastIndex = 0;
const regex = new RegExp(mentionRegex);
let match: RegExpExecArray | null;
while ((match = regex.exec(text)) !== null) {
const fullMatch = match[0];
const color = lowerColorMap[fullMatch.toLowerCase()];
if (match.index > lastIndex) {
html += escapeHTML(text.slice(lastIndex, match.index));
}
if (color) {
html += `<span style="color:${color}">${escapeHTML(fullMatch)}</span>`;
} else {
html += escapeHTML(fullMatch);
}
lastIndex = match.index + fullMatch.length;
}
if (lastIndex < text.length) {
html += escapeHTML(text.slice(lastIndex));
}
html = html.replace(/\n/g, "<br>");
if (html.endsWith("<br>")) {
html += "<br>";
}
return html;
},
[lowerColorMap, mentionRegex],
);
// Sync DOM when value changes from parent
useEffect(() => {
if (isInternalUpdate.current) {
isInternalUpdate.current = false;
return;
}
const el = editorRef.current;
if (!el) return;
// This effect re-runs whenever `buildHTML`'s identity changes (the parent
// recomputing `colorMap`/`mentionItems`), not only when `value` genuinely
// changes from outside. If the DOM already shows this exact text while the
// user is focused, skip the innerHTML rewrite entirely: reassigning
// innerHTML under a live caret detaches it on iOS Safari, which makes the
// cursor disappear mid-typing. A pending caret (mention insert) must still
// resync, and an unfocused editor is safe to recolor.
if (pendingCaret.current === null && document.activeElement === el) {
let domText = el.innerText;
if (domText.endsWith("\n")) domText = domText.slice(0, -1);
if (domText === value) return;
}
const sel = window.getSelection();
if (document.activeElement === el && sel && !sel.isCollapsed) return;
try {
const caret = pendingCaret.current ?? getCaretOffset(el);
el.innerHTML = buildHTML(value);
if (pendingCaret.current !== null) {
setCaretOffset(el, pendingCaret.current);
pendingCaret.current = null;
} else if (document.activeElement === el) {
setCaretOffset(el, caret);
}
} catch {
el.innerHTML = buildHTML(value);
}
}, [value, buildHTML]);
// Get pixel coordinates of a text offset relative to the wrapper
const getOffsetRect = useCallback((charOffset: number) => {
try {
const el = editorRef.current;
if (!el) return null;
const saved = getCaretOffset(el);
setCaretOffset(el, charOffset);
const sel = window.getSelection();
if (!sel?.rangeCount) {
setCaretOffset(el, saved);
return null;
}
const range = sel.getRangeAt(0);
const rect = range.getBoundingClientRect();
const wrapperRect = el.parentElement!.getBoundingClientRect();
setCaretOffset(el, saved);
return {
left: rect.left - wrapperRect.left,
bottom: wrapperRect.bottom - rect.top,
};
} catch {
return null;
}
}, []);
// Detect @mention trigger from cursor position. The `@` is a valid mention
// trigger when it follows anything that isn't a word character — that covers
// whitespace, punctuation, CJK/Chinese text, emoji, line starts, etc. Only
// reject it when it's sitting mid-word after an alphanumeric/underscore.
const detectMention = useCallback(
(text: string, cursorPos: number) => {
const textBefore = text.slice(0, cursorPos);
const lastAt = textBefore.lastIndexOf("@");
if (
lastAt !== -1 &&
(lastAt === 0 || !/[A-Za-z0-9_]/.test(text[lastAt - 1]))
) {
const query = text.slice(lastAt, cursorPos);
if (!query.includes("\n")) {
const pos = getOffsetRect(lastAt);
if (pos) setDropdownPos(pos);
setMentionState({
isOpen: true,
triggerIndex: lastAt,
query,
activeIndex: 0,
});
return;
}
}
setMentionState((prev) =>
prev.isOpen ? { ...prev, isOpen: false } : prev,
);
},
[getOffsetRect],
);
const handleInput = useCallback(() => {
if (isComposing.current) return;
const el = editorRef.current;
if (!el) return;
try {
let text = el.innerText;
if (text.endsWith("\n")) {
text = text.slice(0, -1);
}
const caret = getCaretOffset(el);
const html = buildHTML(text);
if (el.innerHTML !== html) {
el.innerHTML = html;
setCaretOffset(el, caret);
}
isInternalUpdate.current = true;
onChange(text);
detectMention(text, caret);
requestAnimationFrame(() => {
scrollCaretIntoView(el);
});
} catch {
const text = el.innerText?.replace(/\n$/, "") ?? "";
isInternalUpdate.current = true;
onChange(text);
}
}, [onChange, buildHTML, detectMention]);
const handleCompositionStart = useCallback(() => {
isComposing.current = true;
}, []);
const handleCompositionEnd = useCallback(() => {
isComposing.current = false;
handleInput();
}, [handleInput]);
// Select a mention from the dropdown
const handleSelect = useCallback(
(item: MentionItem) => {
const el = editorRef.current;
if (!el) return;
// Use the known mention bounds — `getCaretOffset` can return 0 when the
// dropdown click collapses the selection out of the editor, which would
// leave the typed query in the prompt alongside the inserted label.
const queryEnd =
mentionState.triggerIndex + mentionState.query.length;
const before = value.slice(0, mentionState.triggerIndex);
const after = value.slice(queryEnd);
const mention = `${item.label} `;
const newValue = before + mention + after;
pendingCaret.current = before.length + mention.length;
setMentionState({
isOpen: false,
triggerIndex: -1,
query: "",
activeIndex: 0,
});
onChange(newValue);
requestAnimationFrame(() => {
el.focus();
});
},
[value, mentionState.triggerIndex, mentionState.query, onChange],
);
const handleKeyDown = useCallback(
(e: React.KeyboardEvent<HTMLDivElement>) => {
if (mentionState.isOpen && filteredItems.length > 0) {
if (e.key === "ArrowDown") {
e.preventDefault();
setMentionState((prev) => ({
...prev,
activeIndex: Math.min(
prev.activeIndex + 1,
filteredItems.length - 1,
),
}));
return;
}
if (e.key === "ArrowUp") {
e.preventDefault();
setMentionState((prev) => ({
...prev,
activeIndex: Math.max(prev.activeIndex - 1, 0),
}));
return;
}
if (e.key === "Enter" || e.key === "Tab") {
e.preventDefault();
handleSelect(filteredItems[mentionState.activeIndex]);
return;
}
if (e.key === "Escape") {
e.preventDefault();
setMentionState((prev) => ({ ...prev, isOpen: false }));
return;
}
}
if (e.key === "Enter") {
e.preventDefault();
document.execCommand("insertLineBreak");
handleInput();
scrollCaretIntoView(editorRef.current!);
return;
}
externalOnKeyDown?.(e);
},
[
mentionState.isOpen,
mentionState.activeIndex,
filteredItems,
handleSelect,
externalOnKeyDown,
handleInput,
],
);
const handleClick = useCallback(() => {
const el = editorRef.current;
if (!el) return;
const sel = window.getSelection();
if (sel && !sel.isCollapsed) return;
detectMention(value, getCaretOffset(el));
}, [value, detectMention]);
const handlePaste = useCallback(
(e: React.ClipboardEvent<HTMLDivElement>) => {
e.preventDefault();
const text = e.clipboardData.getData("text/plain");
document.execCommand("insertText", false, text);
},
[],
);
return (
<div className="relative flex-1 min-w-0 pb-[7px]">
{!value && placeholder && (
<div
className={twMerge(
className,
"absolute inset-0 pointer-events-none text-white/50 z-[1]",
)}
>
{placeholder}
</div>
)}
<div
ref={editorRef}
contentEditable={!disabled}
onInput={handleInput}
onCompositionStart={handleCompositionStart}
onCompositionEnd={handleCompositionEnd}
onKeyDown={handleKeyDown}
onClick={handleClick}
onPaste={handlePaste}
onFocus={onFocus}
onBlur={onBlur}
style={style}
className={twMerge(
className,
"outline-none whitespace-pre-wrap break-words overflow-y-auto resize-y",
)}
/>
{mentionState.isOpen && filteredItems.length > 0 && (
<MentionDropdown
items={filteredItems}
activeIndex={mentionState.activeIndex}
onSelect={handleSelect}
onHover={(i) =>
setMentionState((prev) => ({ ...prev, activeIndex: i }))
}
position={dropdownPos}
/>
)}
</div>
);
},
);
@@ -16,12 +16,29 @@ import {
faUserGroup,
faVideo,
} from "@fortawesome/pro-solid-svg-icons";
import { GenerateButton } from "@storyteller/ui-button";
import { GenerateIconButton } from "@storyteller/ui-button";
import { Tooltip } from "@storyteller/ui-tooltip";
import { ImagePromptRow } from "./ImagePromptRow";
import { MentionTextarea } from "./MentionTextarea";
import { getMentionColor, buildMentionColorMap } from "./mention-colors";
import type { RefImage, MentionItem } from "./types";
import {
KeyframeCards,
MentionTextarea,
PromptClearAllButton,
ReferenceDeck,
buildMentionColorMap,
getMentionColor,
useDeckMedia,
type DeckAddAction,
type DeckItem,
} from "@storyteller/ui-promptbox";
import { arrayMove } from "@dnd-kit/sortable";
import {
PromptBoxDropOverlay,
usePromptBoxDrop,
type DroppedFiles,
} from "./PromptBoxDropZone";
import { toast } from "../toast/toast";
import { uploadImage } from "./upload-image";
import { uploadVideo, uploadAudio } from "./upload-media";
import type { RefImage, RefVideo, RefAudio, MentionItem } from "./types";
import { useEnterToGenerateStore } from "../../lib/enter-to-generate-store";
import {
PromptFullscreenButton,
@@ -36,7 +53,6 @@ interface PromptBoxProps {
onPromptChange: (prompt: string) => void;
onSubmit: () => void;
isSubmitting: boolean;
submitLabel?: string;
placeholder?: string;
disabled?: boolean;
credits?: number | null;
@@ -61,11 +77,33 @@ interface PromptBoxProps {
// Pick from library
onPickFromLibrary?: () => void;
onPickEndFrameFromLibrary?: () => void;
onPickVideoFromLibrary?: () => void;
onPickAudioFromLibrary?: () => void;
// Clear all references (images, end frame, videos, audios)
onClearAllRefs?: () => void;
// Toolbar clear-all support. The button wipes the prompt and every
// reference the box renders; pages holding extra state outside the box
// (media reference row, secondary prompts) clear it via onClearAllExtras
// and report its presence via hasClearableExtras so the button enables.
onClearAllExtras?: () => void;
hasClearableExtras?: boolean;
// Media reference row (video/audio refs, rendered between image row and prompt)
mediaReferenceRow?: ReactNode;
// Video/audio references, rendered as cards in the reference deck (video
// page in reference mode, world page's guide video, audio page's tracks).
videoRefsSupported?: boolean;
referenceVideos?: RefVideo[];
onReferenceVideosChange?: (videos: RefVideo[]) => void;
maxVideoCount?: number;
maxVideoRefDuration?: number;
audioRefsSupported?: boolean;
referenceAudios?: RefAudio[];
onReferenceAudiosChange?: (audios: RefAudio[]) => void;
maxAudioCount?: number;
maxAudioRefDuration?: number;
// Always-visible named slot cards rendered beside the reference deck
// (object page's multi-view angles + input mesh), built from DeckSlotCard.
referenceSlots?: ReactNode;
// Model selector (rendered above the toolbar, typically hidden on desktop via lg:hidden)
modelSelector?: ReactNode;
@@ -77,6 +115,14 @@ interface PromptBoxProps {
// @-mention support (enables colored prompt overlay + autocomplete)
mentionItems?: MentionItem[];
// Records which character token a mention name refers to (dropdown pick or
// chip-menu replace) — needed because several characters can share a name.
onMentionSelect?: (item: MentionItem) => void;
// name (without "@") -> character token; picks which character's thumbnail
// renders in a mention chip when labels collide.
mentionSelections?: Record<string, string>;
// Soft prompt-length limit from the model API (`text_prompt_max_length`).
// Undefined = unlimited (no counter). The limit is not enforced here; the
// page's submit handler blocks generation when over.
@@ -90,7 +136,6 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
onPromptChange,
onSubmit,
isSubmitting,
submitLabel = "Generate",
placeholder = "Describe what you want...",
disabled,
credits,
@@ -107,11 +152,27 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
rightToolbar,
onPickFromLibrary,
onPickEndFrameFromLibrary,
onPickVideoFromLibrary,
onPickAudioFromLibrary,
onClearAllRefs,
mediaReferenceRow,
onClearAllExtras,
hasClearableExtras,
videoRefsSupported,
referenceVideos = [],
onReferenceVideosChange,
maxVideoCount = 3,
maxVideoRefDuration = 30,
audioRefsSupported,
referenceAudios = [],
onReferenceAudiosChange,
maxAudioCount = 2,
maxAudioRefDuration = 30,
referenceSlots,
modelSelector,
secondaryPromptRow,
mentionItems,
onMentionSelect,
mentionSelections,
maxPromptLength,
},
ref,
@@ -122,7 +183,6 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
const enterToGenerate = useEnterToGenerateStore((s) => s.enabled);
const [isFocused, setIsFocused] = useState(false);
const [isExpanded, setIsExpanded] = useState(false);
const [showImagePrompts, setShowImagePrompts] = useState(false);
const { isFullscreen, openFullscreen, closeFullscreen } =
useFullscreenPrompt();
@@ -148,12 +208,386 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
const [mentionIndex, setMentionIndex] = useState(0);
const mentionAnchorRef = useRef<number | null>(null);
const isImageRowVisible =
supportsImagePrompts &&
(isVideo || showImagePrompts || referenceImages.length > 0);
const hasMentionItems = (mentionItems?.length ?? 0) > 0;
const hasAnyRowAbove = isImageRowVisible || !!mediaReferenceRow;
const deck = useDeckMedia({
referenceImages,
setReferenceImages: onReferenceImagesChange,
// 0 blocks image uploads (incl. via the combined picker) on pages whose
// model only takes audio/video/mesh refs.
maxImages: supportsImagePrompts ? maxImagePromptCount : 0,
setEndFrameImage: onEndFrameImageChange,
referenceVideos,
setReferenceVideos: onReferenceVideosChange,
maxVideos: maxVideoCount,
maxVideoTotalSec: maxVideoRefDuration,
referenceAudios,
setReferenceAudios: onReferenceAudiosChange,
maxAudios: maxAudioCount,
maxAudioTotalSec: maxAudioRefDuration,
uploadImage,
uploadVideo,
uploadAudio,
// Library picking stays with the page-level GalleryModals.
ownGalleryModal: false,
});
// Drag & drop onto the box bounds: files route to the reference kind
// their MIME matches, gated on what the page's model supports. Keyframe
// mode renders no video/audio deck, so those kinds only land in
// reference mode where the user can see (and remove) them.
const isKeyframeMode = !!isVideo && !isReferenceMode;
const dropAcceptsImages =
!!supportsImagePrompts && maxImagePromptCount > 0;
const dropAcceptsVideos =
!isKeyframeMode && !!videoRefsSupported && !!onReferenceVideosChange;
const dropAcceptsAudio =
!isKeyframeMode && !!audioRefsSupported && !!onReferenceAudiosChange;
const handleDroppedFiles = ({ images, videos, audios }: DroppedFiles) => {
if (images.length > 0) {
if (isKeyframeMode) {
// Fill the empty keyframe slots in order: start frame, then end.
const queue = [...images];
const startOpen =
referenceImages.length === 0 && deck.uploadingImages.length === 0;
const endOpen =
!!showEndFrameSection && !endFrameImage && !deck.uploadingEnd;
if (startOpen) deck.processImageFiles([queue.shift()!], "start");
if (endOpen && queue.length > 0) {
deck.processImageFiles([queue.shift()!], "end");
}
if (!startOpen && !endOpen) {
toast.error(
showEndFrameSection
? "Start and end frames are already set"
: "The start frame is already set",
);
}
} else if (deck.availableImageSlots <= 0) {
toast.error(
`Max ${maxImagePromptCount} image reference${maxImagePromptCount === 1 ? "" : "s"}`,
);
} else {
deck.processImageFiles(images, "start");
}
}
if (videos.length > 0) deck.processVideoFiles(videos);
if (audios.length > 0) deck.processAudioFiles(audios);
};
const drop = usePromptBoxDrop({
acceptsImages: dropAcceptsImages,
acceptsVideos: dropAcceptsVideos,
acceptsAudio: dropAcceptsAudio,
onDropFiles: handleDroppedFiles,
});
// Mixed deck items ordered images → videos → audios so the page's
// index-derived @ImageN/@VideoN/@AudioN mention labels stay aligned.
const deckItems: DeckItem[] = useMemo(
() => [
...referenceImages.map((img, i) => ({
id: img.id,
kind: "image" as const,
url: img.url,
previewUrl: img.fullUrl ?? img.url,
name: `Image ${i + 1}`,
})),
...deck.uploadingImages.map((entry, i) => ({
id: entry.id,
kind: "image" as const,
url: entry.previewUrl,
name: `Image ${referenceImages.length + i + 1}`,
uploading: true,
})),
...referenceVideos.map((video, i) => ({
id: video.id,
kind: "video" as const,
url: video.url,
name: `Video ${i + 1}`,
duration: video.duration,
})),
...(deck.uploadingVideo
? [
{
id: deck.uploadingVideo.id,
kind: "video" as const,
url: deck.uploadingVideo.previewUrl,
name: `Video ${referenceVideos.length + 1}`,
uploading: true,
},
]
: []),
...referenceAudios.map((audio, i) => ({
id: audio.id,
kind: "audio" as const,
url: audio.url,
name: `Audio ${i + 1}`,
duration: audio.duration,
})),
...(deck.uploadingAudio
? [
{
id: deck.uploadingAudio.id,
kind: "audio" as const,
name: `Audio ${referenceAudios.length + 1}`,
uploading: true,
},
]
: []),
],
[
referenceImages,
referenceVideos,
referenceAudios,
deck.uploadingImages,
deck.uploadingVideo,
deck.uploadingAudio,
],
);
const deckAddActions: DeckAddAction[] = [];
if (
supportsImagePrompts &&
referenceImages.length + deck.uploadingImages.length <
maxImagePromptCount
) {
deckAddActions.push({
key: "upload-image",
label: "Upload",
group: "image",
onSelect: deck.openImageUpload,
});
if (onPickFromLibrary) {
deckAddActions.push({
key: "library-image",
label: isVideo ? "From library" : "Pick from library",
group: "image",
onSelect: onPickFromLibrary,
});
}
}
if (
videoRefsSupported &&
referenceVideos.length < maxVideoCount &&
!deck.uploadingVideo
) {
deckAddActions.push({
key: "upload-video",
label: "Upload",
group: "video",
onSelect: deck.openVideoUpload,
});
if (onPickVideoFromLibrary) {
deckAddActions.push({
key: "library-video",
label: "From library",
group: "video",
onSelect: onPickVideoFromLibrary,
});
}
}
if (
audioRefsSupported &&
referenceAudios.length < maxAudioCount &&
!deck.uploadingAudio
) {
deckAddActions.push({
key: "upload-audio",
label: "Upload",
group: "audio",
onSelect: deck.openAudioUpload,
});
if (onPickAudioFromLibrary) {
deckAddActions.push({
key: "library-audio",
label: "From library",
group: "audio",
onSelect: onPickAudioFromLibrary,
});
}
}
const handleRemoveDeckItem = (id: string) => {
if (referenceImages.some((img) => img.id === id)) {
onReferenceImagesChange(referenceImages.filter((img) => img.id !== id));
} else if (referenceVideos.some((video) => video.id === id)) {
onReferenceVideosChange?.(
referenceVideos.filter((video) => video.id !== id),
);
} else if (referenceAudios.some((audio) => audio.id === id)) {
onReferenceAudiosChange?.(
referenceAudios.filter((audio) => audio.id !== id),
);
}
};
const firstFrameItem: DeckItem | undefined = referenceImages[0]
? {
id: referenceImages[0].id,
kind: "image",
url: referenceImages[0].url,
previewUrl: referenceImages[0].fullUrl ?? referenceImages[0].url,
name: "First frame",
}
: deck.uploadingImages[0]
? {
id: deck.uploadingImages[0].id,
kind: "image",
url: deck.uploadingImages[0].previewUrl,
name: "First frame",
uploading: true,
}
: undefined;
const lastFrameItem: DeckItem | undefined = endFrameImage
? {
id: endFrameImage.id,
kind: "image",
url: endFrameImage.url,
previewUrl: endFrameImage.fullUrl ?? endFrameImage.url,
name: "Last frame",
}
: deck.uploadingEnd
? {
id: deck.uploadingEnd.id,
kind: "image",
url: deck.uploadingEnd.previewUrl,
name: "Last frame",
uploading: true,
}
: undefined;
const hasAttachedRefs =
referenceImages.length > 0 ||
!!endFrameImage ||
referenceVideos.length > 0 ||
referenceAudios.length > 0 ||
!!hasClearableExtras;
const hasClearableContent = prompt.length > 0 || hasAttachedRefs;
const handleClearAll = () => {
onPromptChange("");
// Prefer the page's single-shot clear (pages that keep all refs in one
// state object need it to avoid stale-closure partial updates).
if (onClearAllRefs) {
onClearAllRefs();
} else {
onReferenceImagesChange([]);
onEndFrameImageChange?.(undefined);
onReferenceVideosChange?.([]);
onReferenceAudiosChange?.([]);
}
onClearAllExtras?.();
};
const handleSwapFrames = () => {
const first = referenceImages[0];
if (!first || !endFrameImage) return;
onReferenceImagesChange([endFrameImage]);
onEndFrameImageChange?.(first);
};
// Left-of-textarea reference widget: image deck, keyframe cards, or the
// mixed deck depending on page/mode.
const renderReferenceWidget = (alwaysExpanded?: boolean) => {
if (isKeyframeMode) {
if (!supportsImagePrompts) return null;
return (
<KeyframeCards
firstFrame={firstFrameItem}
lastFrame={lastFrameItem}
showLastFrame={!!showEndFrameSection}
onFirstAddActions={[
{
key: "upload-first",
label: "Upload",
onSelect: deck.openImageUpload,
},
...(onPickFromLibrary
? [
{
key: "library-first",
label: "Pick from library",
onSelect: onPickFromLibrary,
},
]
: []),
]}
onLastAddActions={[
{
key: "upload-last",
label: "Upload",
onSelect: deck.openEndUpload,
},
...(onPickEndFrameFromLibrary
? [
{
key: "library-last",
label: "Pick from library",
onSelect: onPickEndFrameFromLibrary,
},
]
: []),
]}
onRemoveFirst={() => onReferenceImagesChange([])}
onRemoveLast={() => onEndFrameImageChange?.(undefined)}
onSwap={handleSwapFrames}
/>
);
}
if (!supportsImagePrompts && !videoRefsSupported && !audioRefsSupported) {
return null;
}
const totalVideoRefSeconds = referenceVideos.reduce(
(sum, video) => sum + video.duration,
0,
);
const totalAudioRefSeconds = referenceAudios.reduce(
(sum, audio) => sum + audio.duration,
0,
);
const groupHints: Record<string, string> = {};
if (supportsImagePrompts) {
groupHints.image = `${referenceImages.length}/${maxImagePromptCount}`;
}
// A non-finite duration cap means "no limit" — show counts only.
if (videoRefsSupported) {
groupHints.video =
`${referenceVideos.length}/${maxVideoCount}` +
(isFinite(maxVideoRefDuration)
? ` · ${totalVideoRefSeconds}/${maxVideoRefDuration}s`
: "");
}
if (audioRefsSupported) {
groupHints.audio =
`${referenceAudios.length}/${maxAudioCount}` +
(isFinite(maxAudioRefDuration)
? ` · ${totalAudioRefSeconds}/${maxAudioRefDuration}s`
: "");
}
return (
<ReferenceDeck
items={deckItems}
canAdd={deckAddActions.length > 0}
addActions={deckAddActions}
addMenuGroupHints={groupHints}
onAddClick={
supportsImagePrompts || videoRefsSupported || audioRefsSupported
? deck.openAnyUpload
: undefined
}
onRemove={handleRemoveDeckItem}
onReorderImages={(from, to) =>
onReferenceImagesChange(arrayMove(referenceImages, from, to))
}
onClearAll={onClearAllRefs}
alwaysExpanded={alwaysExpanded}
/>
);
};
// Filtered mention items for autocomplete
const filteredMentionItems = useMemo(() => {
@@ -334,80 +768,28 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
});
}, [prompt, hasMentionItems, mentionRegex, mentionLabelMap, mentionItems]);
// Shared reference-image row. Inline it attaches to the top of the box
// (square bottom); the fullscreen modal passes a className to round the
// bottom corners since it sits in-flow above the controls there.
const renderImagePromptRow = (className?: string) =>
supportsImagePrompts ? (
<ImagePromptRow
maxImagePromptCount={maxImagePromptCount}
referenceImages={referenceImages}
setReferenceImages={onReferenceImagesChange}
onPickFromLibrary={onPickFromLibrary}
onClearAll={onClearAllRefs}
isVideo={isVideo}
isReferenceMode={isReferenceMode}
endFrameImage={endFrameImage}
setEndFrameImage={onEndFrameImageChange}
showEndFrameSection={showEndFrameSection}
onPickEndFrameFromLibrary={onPickEndFrameFromLibrary}
className={className}
/>
) : null;
return (
<div ref={ref} className="prompt-box-root">
<div className="relative flex flex-col">
{isImageRowVisible && renderImagePromptRow()}
{/* When the media row is the topmost band (no image row above it —
e.g. the audio page), clip it to rounded top corners; mid-stack
(video's start-frame row above) it stays square. */}
{mediaReferenceRow && (
<div className={twMerge(!isImageRowVisible && "sm:rounded-t-2xl")}>
{mediaReferenceRow}
</div>
)}
{deck.fileInputs}
<div
className={twMerge(
"glass rounded-2xl p-3 sm:p-4 !transition-all duration-200",
hasAnyRowAbove && "rounded-t-none border-t-0",
isFocused && "ring-1 ring-primary",
)}
{...drop.dropZoneProps}
>
<PromptBoxDropOverlay
dragState={drop.dragState}
acceptsImages={dropAcceptsImages}
acceptsVideos={dropAcceptsVideos}
acceptsAudio={dropAcceptsAudio}
keyframeMode={isKeyframeMode}
/>
<div className="flex gap-3">
{supportsImagePrompts && !isVideo && (
<Tooltip
content="Add Image"
position="top"
closeOnClick={true}
className={twMerge(isImageRowVisible && "hidden opacity-0")}
>
<button
type="button"
className={twMerge(
"flex h-6 w-6 shrink-0 items-center justify-center rounded-md bg-transparent p-0 transition-all hover:text-primary-500",
isImageRowVisible && "text-primary",
)}
onClick={() => setShowImagePrompts((prev) => !prev)}
>
<svg
width="24"
height="20"
viewBox="0 0 24 20"
fill="none"
xmlns="http://www.w3.org/2000/svg"
className="opacity-80 transition-all hover:opacity-100"
>
<path
d="M2.66667 2H16C16.3667 2 16.6667 2.3 16.6667 2.66667V6.1125C17.1 6.04167 17.5458 6 18 6C18.225 6 18.4458 6.00833 18.6667 6.02917V2.66667C18.6667 1.19583 17.4708 0 16 0H2.66667C1.19583 0 0 1.19583 0 2.66667V16C0 17.4708 1.19583 18.6667 2.66667 18.6667H11.5C11.0625 18.0583 10.7083 17.3875 10.4542 16.6667H2.66667C2.3 16.6667 2 16.3667 2 16V2.66667C2 2.3 2.3 2 2.66667 2ZM11.8625 7.49167C11.6833 7.1875 11.3542 7 11 7C10.6458 7 10.3167 7.1875 10.1375 7.49167L8.2 10.7833L7.48333 9.75833C7.29583 9.49167 6.99167 9.33333 6.6625 9.33333C6.33333 9.33333 6.02917 9.49167 5.84167 9.75833L3.50833 13.0917C3.29583 13.3958 3.26667 13.7958 3.44167 14.125C3.61667 14.4542 3.9625 14.6667 4.33333 14.6667H10.0292C10.0125 14.4458 10 14.225 10 14C10 11.7833 10.9 9.77917 12.3542 8.33333L11.8625 7.49583V7.49167ZM5.33333 6.66667C6.07083 6.66667 6.66667 6.07083 6.66667 5.33333C6.66667 4.59583 6.07083 4 5.33333 4C4.59583 4 4 4.59583 4 5.33333C4 6.07083 4.59583 6.66667 5.33333 6.66667ZM18 20C21.3125 20 24 17.3125 24 14C24 10.6875 21.3125 8 18 8C14.6875 8 12 10.6875 12 14C12 17.3125 14.6875 20 18 20ZM18.6667 11.3333V13.3333H20.6667C21.0333 13.3333 21.3333 13.6333 21.3333 14C21.3333 14.3667 21.0333 14.6667 20.6667 14.6667H18.6667V16.6667C18.6667 17.0333 18.3667 17.3333 18 17.3333C17.6333 17.3333 17.3333 17.0333 17.3333 16.6667V14.6667H15.3333C14.9667 14.6667 14.6667 14.3667 14.6667 14C14.6667 13.6333 14.9667 13.3333 15.3333 13.3333H17.3333V11.3333C17.3333 10.9667 17.6333 10.6667 18 10.6667C18.3667 10.6667 18.6667 10.9667 18.6667 11.3333Z"
fill="currentColor"
/>
</svg>
</button>
</Tooltip>
)}
{renderReferenceWidget()}
{referenceSlots}
<div className="promptbox-resize-wrap relative flex-1">
{hasMentionItems && mentionItems ? (
@@ -418,10 +800,13 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
mentionItems={mentionItems}
placeholder={placeholder}
className={twMerge(
"promptbox-scrollbar min-h-[2.5em] w-full pr-8 text-base-fg placeholder-base-fg/60",
"promptbox-scrollbar min-h-[2.5em] w-full resize-y pr-8 text-base-fg placeholder-base-fg/60",
isExpanded ? "max-h-[500px]" : "max-h-[5.5em]",
)}
colorMap={mentionColorMap}
enterToGenerate={enterToGenerate}
onMentionSelect={onMentionSelect}
selectedTokens={mentionSelections}
onKeyDown={(e) => {
if (
e.key === "Enter" &&
@@ -561,22 +946,24 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
)}
{/* Toolbar */}
<div className="mt-2 flex flex-col gap-2 sm:flex-row sm:items-center sm:justify-between">
<div className="mt-3.5 flex flex-col gap-2 sm:flex-row sm:items-center sm:justify-between">
<div className="flex flex-wrap items-center gap-1.5 sm:gap-2">
{modelSelector}
{leftToolbar}
</div>
<div className="flex items-center gap-2 sm:shrink-0">
<div className="flex items-center gap-1.5 sm:shrink-0">
{rightToolbar}
<GenerateButton
className="flex flex-1 sm:flex-none items-center justify-center border-none bg-primary px-3 text-sm text-white disabled:cursor-not-allowed disabled:opacity-50"
<PromptClearAllButton
onClick={handleClearAll}
disabled={!hasClearableContent}
confirmClear={hasAttachedRefs}
/>
<GenerateIconButton
onClick={onSubmit}
disabled={disabled ?? (!prompt.trim() || isSubmitting)}
loading={isSubmitting}
credits={credits}
>
{submitLabel}
</GenerateButton>
/>
</div>
</div>
@@ -611,7 +998,21 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
{leftToolbar}
</>
}
imagePromptRow={renderImagePromptRow("rounded-2xl sm:rounded-b-2xl")}
imagePromptRow={
renderReferenceWidget(true) || referenceSlots ? (
<div className="flex flex-wrap items-center gap-3">
{renderReferenceWidget(true)}
{referenceSlots}
</div>
) : undefined
}
clearAllButton={
<PromptClearAllButton
onClick={handleClearAll}
disabled={!hasClearableContent}
confirmClear={hasAttachedRefs}
/>
}
>
{hasMentionItems && mentionItems ? (
<MentionTextarea
@@ -622,6 +1023,9 @@ export const PromptBox = forwardRef<HTMLDivElement, PromptBoxProps>(
className="promptbox-scrollbar h-full min-h-0 w-full overflow-y-auto text-base-fg placeholder-base-fg/60"
style={{ resize: "none" }}
colorMap={mentionColorMap}
enterToGenerate={enterToGenerate}
onMentionSelect={onMentionSelect}
selectedTokens={mentionSelections}
onKeyDown={(e) => {
if (
e.key === "Enter" &&
@@ -0,0 +1,289 @@
import { useCallback, useEffect, useRef, useState } from "react";
import { twMerge } from "tailwind-merge";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faImages, faMusic, faVideo } from "@fortawesome/pro-solid-svg-icons";
import { toast } from "../toast/toast";
// ── Types ───────────────────────────────────────────────────────────────
type DropKind = "image" | "video" | "audio";
/** "accept" / "reject" reflect whether the hovered payload contains at
* least one file the current model can take as a reference. */
export type DropDragState = "idle" | "accept" | "reject";
export interface DroppedFiles {
images: File[];
videos: File[];
audios: File[];
}
interface UsePromptBoxDropOptions {
acceptsImages: boolean;
acceptsVideos: boolean;
acceptsAudio: boolean;
onDropFiles: (files: DroppedFiles) => void;
}
interface PromptBoxDropOverlayProps {
dragState: DropDragState;
acceptsImages: boolean;
acceptsVideos: boolean;
acceptsAudio: boolean;
/** Video keyframe mode: images fill the start/end frame slots. */
keyframeMode?: boolean;
}
const kindOfMime = (mime: string): DropKind | null =>
mime.startsWith("image/")
? "image"
: mime.startsWith("video/")
? "video"
: mime.startsWith("audio/")
? "audio"
: null;
const capitalize = (s: string) => s.charAt(0).toUpperCase() + s.slice(1);
const listKinds = (labels: string[]) =>
labels.length <= 1
? (labels[0] ?? "")
: `${labels.slice(0, -1).join(", ")} or ${labels[labels.length - 1]}`;
/**
* Makes the prompt box a drop target for reference media, and catches
* clipboard pastes of files anywhere on the page (focus-independent). Files
* are routed by MIME type; kinds the current model doesn't take are
* rejected with a toast instead of silently vanishing.
*/
export function usePromptBoxDrop({
acceptsImages,
acceptsVideos,
acceptsAudio,
onDropFiles,
}: UsePromptBoxDropOptions) {
const [dragState, setDragState] = useState<DropDragState>("idle");
const enabled = acceptsImages || acceptsVideos || acceptsAudio;
const acceptsKind = useCallback(
(kind: DropKind | null) =>
kind === "image"
? acceptsImages
: kind === "video"
? acceptsVideos
: kind === "audio"
? acceptsAudio
: false,
[acceptsImages, acceptsVideos, acceptsAudio],
);
// While the box is on screen, a stray drop outside it must not navigate
// the browser to the file (which would blow away the user's session).
useEffect(() => {
if (!enabled) return;
const preventNavigation = (e: DragEvent) => {
if (e.dataTransfer?.types.includes("Files")) e.preventDefault();
};
// Drops that land outside the box still need the overlay dismissed.
const clearDragState = () => setDragState("idle");
window.addEventListener("dragover", preventNavigation);
window.addEventListener("drop", preventNavigation);
window.addEventListener("drop", clearDragState);
window.addEventListener("dragend", clearDragState);
return () => {
window.removeEventListener("dragover", preventNavigation);
window.removeEventListener("drop", preventNavigation);
window.removeEventListener("drop", clearDragState);
window.removeEventListener("dragend", clearDragState);
};
}, [enabled]);
// MIME types are unreliable mid-drag (browsers may report empty strings),
// so unknown types count as acceptable until the drop resolves them.
const inspectDrag = useCallback(
(dt: DataTransfer): DropDragState => {
const fileItems = Array.from(dt.items).filter((i) => i.kind === "file");
if (fileItems.length === 0) return "accept";
const ok = fileItems.some(
(i) => i.type === "" || acceptsKind(kindOfMime(i.type)),
);
return ok ? "accept" : "reject";
},
[acceptsKind],
);
const handleDragOver = (e: React.DragEvent<HTMLDivElement>) => {
if (!enabled || !e.dataTransfer.types.includes("Files")) return;
e.preventDefault();
e.stopPropagation();
e.dataTransfer.dropEffect = "copy";
setDragState(inspectDrag(e.dataTransfer));
};
const handleDragLeave = (e: React.DragEvent<HTMLDivElement>) => {
if (dragState === "idle") return;
// Child elements fire enter/leave pairs constantly; only reset when the
// pointer actually exits the drop target's bounds.
const rect = e.currentTarget.getBoundingClientRect();
if (
e.clientX < rect.left ||
e.clientX >= rect.right ||
e.clientY < rect.top ||
e.clientY >= rect.bottom
) {
setDragState("idle");
}
};
const routeFiles = (files: File[]) => {
const accepted: DroppedFiles = { images: [], videos: [], audios: [] };
const rejectedKinds = new Set<DropKind>();
let unknownCount = 0;
for (const file of files) {
const kind = kindOfMime(file.type);
if (kind === null) {
unknownCount++;
} else if (!acceptsKind(kind)) {
rejectedKinds.add(kind);
} else if (kind === "image") {
accepted.images.push(file);
} else if (kind === "video") {
accepted.videos.push(file);
} else {
accepted.audios.push(file);
}
}
const anyAccepted =
accepted.images.length > 0 ||
accepted.videos.length > 0 ||
accepted.audios.length > 0;
if (anyAccepted) onDropFiles(accepted);
// Mode-neutral wording: a kind can be off because the model lacks it or
// because the current input mode (e.g. keyframes) doesn't show it.
if (rejectedKinds.size > 0) {
toast.error(
`${capitalize(listKinds([...rejectedKinds]))} references aren't available here`,
);
} else if (unknownCount > 0 && !anyAccepted) {
toast.error(
`Only ${listKinds(acceptedKindLabels(acceptsImages, acceptsVideos, acceptsAudio))} files can be added here`,
);
}
};
const handleDrop = (e: React.DragEvent<HTMLDivElement>) => {
if (!enabled || !e.dataTransfer.types.includes("Files")) return;
e.preventDefault();
e.stopPropagation();
setDragState("idle");
const files = Array.from(e.dataTransfer.files);
if (files.length > 0) routeFiles(files);
};
// Paste-to-add: a copied image (or media file) pasted anywhere on the
// page lands in the deck, whether or not the textarea is focused. The
// listener re-reads the freshest router through a ref so it can stay
// subscribed once.
const routeFilesRef = useRef(routeFiles);
useEffect(() => {
routeFilesRef.current = routeFiles;
});
useEffect(() => {
if (!enabled) return;
const handlePaste = (e: ClipboardEvent) => {
const files = Array.from(e.clipboardData?.files ?? []);
// Plain text pastes carry no files — leave them entirely alone.
if (files.length === 0) return;
// Guard against double-adding if more than one box ever mounts.
const marked = e as ClipboardEvent & { promptBoxHandled?: boolean };
if (marked.promptBoxHandled) return;
marked.promptBoxHandled = true;
// No preventDefault: any text alongside the file still pastes into
// whichever field is focused.
routeFilesRef.current(files);
};
window.addEventListener("paste", handlePaste);
return () => window.removeEventListener("paste", handlePaste);
}, [enabled]);
return {
dragState,
dropZoneProps: {
onDragEnter: handleDragOver,
onDragOver: handleDragOver,
onDragLeave: handleDragLeave,
onDrop: handleDrop,
},
};
}
/**
* Full-bleed dashed overlay shown while files hover the prompt box, listing
* the reference kinds the current model takes (red when none match).
*/
export function PromptBoxDropOverlay({
dragState,
acceptsImages,
acceptsVideos,
acceptsAudio,
keyframeMode,
}: PromptBoxDropOverlayProps) {
if (dragState === "idle") return null;
const kinds = [
...(acceptsImages
? [{ icon: faImages, label: keyframeMode ? "Frames" : "Images" }]
: []),
...(acceptsVideos ? [{ icon: faVideo, label: "Video" }] : []),
...(acceptsAudio ? [{ icon: faMusic, label: "Audio" }] : []),
];
const rejected = dragState === "reject";
return (
<div
className={twMerge(
"promptbox-drop-overlay pointer-events-none absolute inset-0 z-40 flex flex-col items-center justify-center gap-2 rounded-2xl border-2 border-dashed bg-[#161618]/85 backdrop-blur-sm",
rejected ? "border-red-400/80" : "border-primary",
)}
>
<div className="flex items-center gap-2">
{kinds.map((kind, i) => (
<div
key={kind.label}
className="promptbox-drop-chip flex h-9 w-9 items-center justify-center rounded-lg bg-white/10 text-base-fg/90"
style={{ animationDelay: `${i * 50}ms` }}
>
<FontAwesomeIcon icon={kind.icon} className="text-sm" />
</div>
))}
</div>
<div className="text-sm font-semibold text-base-fg">
{rejected
? "That file type isn't supported"
: keyframeMode && !acceptsVideos && !acceptsAudio
? "Drop to set your frames"
: "Drop to add references"}
</div>
<div className="-mt-1 text-xs text-base-fg/60">
{rejected
? `Accepts ${listKinds(acceptedKindLabels(acceptsImages, acceptsVideos, acceptsAudio))} files`
: kinds.map((kind) => kind.label).join(" · ")}
</div>
</div>
);
}
function acceptedKindLabels(
acceptsImages: boolean,
acceptsVideos: boolean,
acceptsAudio: boolean,
): string[] {
return [
...(acceptsImages ? ["image"] : []),
...(acceptsVideos ? ["video"] : []),
...(acceptsAudio ? ["audio"] : []),
];
}
@@ -1,10 +1,4 @@
import {
ReactNode,
useCallback,
useEffect,
useRef,
useState,
} from "react";
import { ReactNode, useCallback, useEffect, useRef, useState } from "react";
import { Modal } from "@storyteller/ui-modal";
import { Button } from "@storyteller/ui-button";
import { Tooltip } from "@storyteller/ui-tooltip";
@@ -66,6 +60,8 @@ interface PromptFullscreenModalProps {
footerControls?: ReactNode;
/** Reference-image row, always shown in focus mode. */
imagePromptRow?: ReactNode;
/** Clear-all control rendered on the footer's right side, left of Done. */
clearAllButton?: ReactNode;
}
export const PromptFullscreenModal = ({
@@ -76,6 +72,7 @@ export const PromptFullscreenModal = ({
maxPromptLength,
footerControls,
imagePromptRow,
clearAllButton,
}: PromptFullscreenModalProps) => {
const showCounter =
promptLength !== undefined && maxPromptLength !== undefined;
@@ -86,7 +83,10 @@ export const PromptFullscreenModal = ({
// Focus the editor and drop the caret at the end once the overlay opens.
useEffect(() => {
if (!isOpen) return;
const id = window.setTimeout(() => focusEditorAtEnd(contentRef.current), 60);
const id = window.setTimeout(
() => focusEditorAtEnd(contentRef.current),
60,
);
return () => window.clearTimeout(id);
}, [isOpen]);
@@ -123,14 +123,18 @@ export const PromptFullscreenModal = ({
overLimit ? "text-red-500" : "text-base-fg/40"
}`}
>
{promptLength} / {isFinite(maxPromptLength) ? maxPromptLength : "∞"}
{promptLength} /{" "}
{isFinite(maxPromptLength) ? maxPromptLength : "∞"}
</span>
</div>
)}
{imagePromptRow && <div className="shrink-0">{imagePromptRow}</div>}
<div className="flex shrink-0 items-center justify-between gap-2">
<div className="flex shrink-0 items-center justify-between gap-2 mt-1">
<div className="flex items-center gap-2">{footerControls}</div>
<Button onClick={onClose}>Done</Button>
<div className="flex items-center gap-2">
{clearAllButton}
<Button onClick={onClose}>Done</Button>
</div>
</div>
</div>
</Modal>
@@ -1,36 +0,0 @@
import { create } from "zustand";
export interface StoredCharacter {
character_token: string;
name: string;
avatar_image_url?: string;
}
interface CharactersStore {
characters: StoredCharacter[];
loaded: boolean;
setCharacters: (characters: StoredCharacter[]) => void;
addCharacter: (character: StoredCharacter) => void;
updateCharacter: (token: string, updates: Partial<StoredCharacter>) => void;
removeCharacter: (token: string) => void;
setLoaded: (loaded: boolean) => void;
}
export const useCharactersStore = create<CharactersStore>()((set) => ({
characters: [],
loaded: false,
setCharacters: (characters) => set({ characters }),
addCharacter: (character) =>
set((state) => ({ characters: [...state.characters, character] })),
updateCharacter: (token, updates) =>
set((state) => ({
characters: state.characters.map((c) =>
c.character_token === token ? { ...c, ...updates } : c,
),
})),
removeCharacter: (token) =>
set((state) => ({
characters: state.characters.filter((c) => c.character_token !== token),
})),
setLoaded: (loaded) => set({ loaded }),
}));
@@ -2,10 +2,13 @@ export { PromptBox } from "./PromptBox";
export { ImagePromptRow, AddButton } from "./ImagePromptRow";
export { MediaReferenceRow } from "./MediaReferenceRow";
export { CharactersModal } from "./CharactersModal";
export { useCharactersStore } from "./characters-store";
export type { StoredCharacter } from "./characters-store";
export { useCharactersStore } from "@storyteller/ui-promptbox";
export type { StoredCharacter } from "@storyteller/ui-promptbox";
export type { RefImage, RefVideo, RefAudio, MentionItem } from "./types";
export { getVideoDurationFromUrl } from "./upload-media";
export {
getAudioDurationFromUrl,
getVideoDurationFromUrl,
} from "./upload-media";
export {
MobilePromptForm,
MobileSelectField,
@@ -1,9 +1,11 @@
import { type ReactNode, useMemo } from "react";
import { GenerateButton } from "@storyteller/ui-button";
import { MentionTextarea } from "../MentionTextarea";
import { buildMentionColorMap } from "../mention-colors";
import {
MentionTextarea,
buildMentionColorMap,
type MentionItem,
} from "@storyteller/ui-promptbox";
import { useMobileCreateTabs } from "../../generation-gallery/mobile-create-tabs";
import type { MentionItem } from "../types";
interface MobilePromptFormProps {
prompt: string;
@@ -14,6 +16,8 @@ interface MobilePromptFormProps {
submitLabel?: string;
credits?: number | null;
mentionItems?: MentionItem[];
onMentionSelect?: (item: MentionItem) => void;
mentionSelections?: Record<string, string>;
// Whether tapping Create will actually start a generation (logged in, prompt
// present, no missing required input). Gates the auto-switch to History.
autoAdvance?: boolean;
@@ -38,6 +42,8 @@ export function MobilePromptForm({
submitLabel = "Create",
credits,
mentionItems,
onMentionSelect,
mentionSelections,
autoAdvance = true,
banner,
inputModeSelector,
@@ -79,6 +85,10 @@ export function MobilePromptForm({
colorMap={colorMap}
placeholder={placeholder}
className="min-h-[120px] max-h-[40vh] w-full text-md text-base-fg"
// Mobile has no Enter-submit; Enter always inserts a newline.
enterToGenerate={false}
onMentionSelect={onMentionSelect}
selectedTokens={mentionSelections}
/>
</div>
@@ -23,8 +23,4 @@ export interface RefAudio {
duration: number;
}
export interface MentionItem {
label: string;
type: "image" | "video" | "audio" | "character";
preview?: string;
}
export type { MentionItem } from "@storyteller/ui-promptbox";
@@ -83,6 +83,16 @@ export const getVideoDurationFromUrl = (url: string): Promise<number> =>
video.src = url;
});
// Resolves 0 when metadata can't be loaded.
export const getAudioDurationFromUrl = (url: string): Promise<number> =>
new Promise((resolve) => {
const audio = document.createElement("audio");
audio.preload = "metadata";
audio.onloadedmetadata = () => resolve(Math.round(audio.duration));
audio.onerror = () => resolve(0);
audio.src = url;
});
export const getVideoDuration = (file: File): Promise<number> => {
const url = URL.createObjectURL(file);
return getVideoDurationFromUrl(url).finally(() => URL.revokeObjectURL(url));
@@ -1,11 +1,11 @@
import { useEffect, useMemo, useState } from "react";
import { useEffect, useState } from "react";
import { Modal } from "@storyteller/ui-modal";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faCog, faUser, faKey } from "@fortawesome/pro-solid-svg-icons";
import { Switch } from "@storyteller/ui-switch";
import { USER_FEATURE_FLAGS } from "@storyteller/api";
import { twMerge } from "tailwind-merge";
import { useEnterToGenerateStore } from "../../lib/enter-to-generate-store";
import { useLightboxSoundStore } from "../../lib/lightbox-sound-store";
import { useSession } from "../../lib/session";
import { AccountSection } from "./AccountSection";
import { ApiKeySection } from "./ApiKeySection";
@@ -17,41 +17,20 @@ interface SettingsModalProps {
onClose: () => void;
}
const BASE_TABS: { id: Tab; label: string; icon: typeof faCog }[] = [
const TABS: { id: Tab; label: string; icon: typeof faCog }[] = [
{ id: "general", label: "General", icon: faCog },
{ id: "account", label: "Account", icon: faUser },
{ id: "apiKeys", label: "API Keys", icon: faKey },
];
const API_KEYS_TAB: { id: Tab; label: string; icon: typeof faCog } = {
id: "apiKeys",
label: "API Keys",
icon: faKey,
};
export function SettingsModal({ isOpen, onClose }: SettingsModalProps) {
const { user } = useSession();
const [tab, setTab] = useState<Tab>("general");
const hasApiKeyFlag = !!user?.maybe_feature_flags?.includes(
USER_FEATURE_FLAGS.API_KEY,
);
const tabs = useMemo(
() => (hasApiKeyFlag ? [...BASE_TABS, API_KEYS_TAB] : BASE_TABS),
[hasApiKeyFlag],
);
useEffect(() => {
if (isOpen) setTab("general");
}, [isOpen]);
// If the only-conditional tab disappears (e.g. flag revoked while open),
// fall back to a visible tab.
useEffect(() => {
if (!tabs.some((t) => t.id === tab)) setTab("general");
}, [tabs, tab]);
const activeLabel = tabs.find((t) => t.id === tab)?.label ?? "";
const activeLabel = TABS.find((t) => t.id === tab)?.label ?? "";
return (
<Modal
@@ -68,7 +47,7 @@ export function SettingsModal({ isOpen, onClose }: SettingsModalProps) {
</div>
<hr className="my-2 hidden w-full border-ui-panel-border sm:block" />
<div className="flex gap-2 overflow-x-auto pe-10 sm:block sm:space-y-1 sm:overflow-visible sm:pe-0">
{tabs.map((t) => (
{TABS.map((t) => (
<button
key={t.id}
onClick={() => setTab(t.id)}
@@ -105,6 +84,8 @@ export function SettingsModal({ isOpen, onClose }: SettingsModalProps) {
function GeneralPanel() {
const enterToGenerate = useEnterToGenerateStore((s) => s.enabled);
const setEnterToGenerate = useEnterToGenerateStore((s) => s.setEnabled);
const lightboxSound = useLightboxSoundStore((s) => s.soundEnabled);
const setLightboxSound = useLightboxSoundStore((s) => s.setSoundEnabled);
return (
<div className="space-y-4 text-base-fg">
@@ -123,6 +104,20 @@ function GeneralPanel() {
offClassName="bg-white/20"
/>
</div>
<div className="flex flex-col gap-2">
<div className="flex flex-col gap-0.5">
<p className="text-sm font-medium">Play videos with sound</p>
<p className="text-xs opacity-70">
When on (default), videos in the media viewer start unmuted so you
don't have to click unmute every time. When off, they start muted.
</p>
</div>
<Switch
enabled={lightboxSound}
setEnabled={setLightboxSound}
offClassName="bg-white/20"
/>
</div>
</div>
);
}
@@ -18,6 +18,7 @@ import {
faDownload,
faGift,
faPencil,
faPhotoFilm,
} from "@fortawesome/pro-solid-svg-icons";
import { faDiscord } from "@fortawesome/free-brands-svg-icons";
import { Button } from "@storyteller/ui-button";
@@ -78,6 +79,12 @@ const STUDIO_ITEMS_STATIC: NavItem[] = [
icon: faObjectGroup,
badge: "BETA",
},
{
label: "Frame Extract",
href: "/frame-extractor",
icon: faPhotoFilm,
badge: "NEW",
},
];
// Rewrite the "Edit 3D" item's href to point at the user's last visited
@@ -165,7 +172,11 @@ function NavMenuItem({
/>
<span className="group-data-[collapsible=icon]:hidden">{item.label}</span>
{item.badge && (
<span className="ml-auto bg-amber-600 px-1.5 py-0.5 text-[10px] font-semibold uppercase leading-none text-white group-data-[collapsible=icon]:hidden rounded-full">
<span
className={`ml-auto px-1.5 py-0.5 text-[10px] font-semibold uppercase leading-none text-white group-data-[collapsible=icon]:hidden rounded-full ${
item.badge === "NEW" ? "bg-purple-600" : "bg-amber-600"
}`}
>
{item.badge}
</span>
)}
@@ -1,14 +1,17 @@
import { useMemo } from "react";
import { useMemo, useState } from "react";
import { Link, useNavigate } from "react-router-dom";
import { AnimatePresence, motion } from "framer-motion";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faBorderAll,
faChevronRight,
faFolder,
faFolderOpen,
faPlus,
faStar,
} from "@fortawesome/pro-solid-svg-icons";
import { compareFolders } from "@storyteller/ui-gallery-modal";
import { EASE_EMPHASIS } from "../../lib/motion";
import {
SidebarGroup,
SidebarGroupContent,
@@ -18,6 +21,7 @@ import {
SidebarMenuItem,
} from "../ui/sidebar";
import { useLibraryFoldersStore } from "../../pages/library/library-folders-store";
import { LibraryTagsNav } from "./library-tags-nav";
/**
* The "Library" sidebar section: Unsorted + Folders entries, plus the user's
@@ -36,17 +40,23 @@ export function LibraryFoldersNav({
pathname === "/library/folders" ||
pathname.startsWith("/library/folder_");
const onFolderless = pathname === "/library/folderless";
const onTags =
pathname === "/library/tags" || pathname.startsWith("/library/tag_");
const onUnsorted =
!onFolders &&
!onFolderless &&
!onTags &&
(pathname === "/library" || pathname.startsWith("/library/"));
const inLibraryArea = onUnsorted || onFolders || onFolderless;
const inLibraryArea = onUnsorted || onFolders || onFolderless || onTags;
const folders = useLibraryFoldersStore((s) => s.folders);
const activeFolderId = useLibraryFoldersStore((s) => s.activeFolderId);
const openNewFolderModal = useLibraryFoldersStore((s) => s.openNewFolderModal);
const setContextMenu = useLibraryFoldersStore((s) => s.setContextMenu);
// Folders are the primary organization tool — default expanded.
const [expanded, setExpanded] = useState(true);
const rootFolders = useMemo(
() => folders.filter((f) => !f.parentId).sort(compareFolders),
[folders],
@@ -95,18 +105,6 @@ export function LibraryFoldersNav({
</Link>
</SidebarMenuButton>
</SidebarMenuItem>
<SidebarMenuItem>
<SidebarMenuButton
asChild
isActive={onFolders && !activeFolderId}
tooltip="Folders"
>
<Link to="/library/folders" onClick={onNavClick}>
<FontAwesomeIcon icon={faFolder} />
<span>Folders</span>
</Link>
</SidebarMenuButton>
</SidebarMenuItem>
<SidebarMenuItem>
<SidebarMenuButton
asChild
@@ -119,45 +117,85 @@ export function LibraryFoldersNav({
</Link>
</SidebarMenuButton>
</SidebarMenuItem>
{inLibraryArea &&
rootFolders.map((folder) => (
<SidebarMenuItem
key={folder.id}
className="group-data-[collapsible=icon]:hidden"
<SidebarMenuItem>
<SidebarMenuButton
asChild
isActive={onFolders && !activeFolderId}
tooltip="Folders"
>
<Link to="/library/folders" onClick={onNavClick}>
<FontAwesomeIcon icon={faFolder} />
<span>Folders</span>
</Link>
</SidebarMenuButton>
{inLibraryArea && rootFolders.length > 0 && (
<button
type="button"
onClick={() => setExpanded((v) => !v)}
aria-label={expanded ? "Collapse folders" : "Expand folders"}
className="absolute right-1 top-1/2 -translate-y-1/2 flex h-5 w-5 items-center justify-center rounded text-sidebar-foreground/50 hover:bg-sidebar-accent hover:text-sidebar-foreground transition-colors group-data-[collapsible=icon]:hidden"
>
<SidebarMenuButton
isActive={activeRootId === folder.id}
tooltip={folder.name}
data-folder-id={folder.id}
onClick={() => goToFolder(folder.id)}
onContextMenu={(e) => {
e.preventDefault();
setContextMenu({
folderId: folder.id,
x: e.clientX,
y: e.clientY,
});
}}
className="pl-5 [&.folder-drag-over]:bg-primary/20 [&.folder-drag-over]:text-sidebar-foreground"
>
<FontAwesomeIcon
icon={faFolder}
className={folder.colorCode ? "" : "text-primary"}
style={
folder.colorCode ? { color: folder.colorCode } : undefined
}
/>
<span className="truncate">{folder.name}</span>
{folder.hasStar && (
<FontAwesomeIcon
icon={faStar}
className="ml-auto text-[10px] text-amber-400"
/>
)}
</SidebarMenuButton>
</SidebarMenuItem>
))}
<FontAwesomeIcon
icon={faChevronRight}
className={`text-[10px] transition-transform duration-200 ${expanded ? "rotate-90" : ""}`}
/>
</button>
)}
</SidebarMenuItem>
<AnimatePresence initial={false}>
{inLibraryArea && expanded && rootFolders.length > 0 && (
<motion.li
key="folder-rows"
initial={{ height: 0, opacity: 0 }}
animate={{ height: "auto", opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.28, ease: EASE_EMPHASIS }}
className="list-none overflow-hidden group-data-[collapsible=icon]:hidden"
>
<ul className="flex w-full min-w-0 flex-col gap-0.5">
{rootFolders.map((folder) => (
<SidebarMenuItem key={folder.id}>
<SidebarMenuButton
isActive={activeRootId === folder.id}
tooltip={folder.name}
data-folder-id={folder.id}
onClick={() => goToFolder(folder.id)}
onContextMenu={(e) => {
e.preventDefault();
setContextMenu({
folderId: folder.id,
x: e.clientX,
y: e.clientY,
});
}}
className="pl-5 [&.folder-drag-over]:bg-primary/20 [&.folder-drag-over]:text-sidebar-foreground"
>
<FontAwesomeIcon
icon={faFolder}
className={folder.colorCode ? "" : "text-primary"}
style={
folder.colorCode
? { color: folder.colorCode }
: undefined
}
/>
<span className="truncate">{folder.name}</span>
{folder.hasStar && (
<FontAwesomeIcon
icon={faStar}
className="ml-auto text-[10px] text-amber-400"
/>
)}
</SidebarMenuButton>
</SidebarMenuItem>
))}
</ul>
</motion.li>
)}
</AnimatePresence>
<LibraryTagsNav pathname={pathname} onNavClick={onNavClick} />
</SidebarMenu>
</SidebarGroupContent>
</SidebarGroup>
@@ -0,0 +1,163 @@
import { useMemo, useState } from "react";
import { Link, useNavigate } from "react-router-dom";
import { AnimatePresence, motion } from "framer-motion";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faChevronRight,
faEllipsis,
faTag,
faTags,
} from "@fortawesome/pro-solid-svg-icons";
import { SidebarMenuButton, SidebarMenuItem } from "../ui/sidebar";
import {
compareTagsByUseCount,
useLibraryTagsStore,
} from "../../pages/library/library-tags-store";
import { EASE_EMPHASIS } from "../../lib/motion";
const MAX_SIDEBAR_TAGS = 5;
/**
* The "Tags" rows of the Assets sidebar group, rendered below the folders
* list: a Tags entry plus the user's most-used tags (collapsed by default)
* while in the library area. Hidden entirely until the user has tags — the
* tag editor in the media details panel is where tags get created.
* Right-click opens the library page's tag context menu (same contract as
* the folder rows). Renders bare menu items; the parent supplies the
* `SidebarMenu` list.
*/
export function LibraryTagsNav({
pathname,
onNavClick,
}: {
pathname: string;
onNavClick: () => void;
}) {
const navigate = useNavigate();
const tags = useLibraryTagsStore((s) => s.tags);
const tagsLoaded = useLibraryTagsStore((s) => s.tagsLoaded);
const setContextMenu = useLibraryTagsStore((s) => s.setContextMenu);
// Folders are the primary organization tool — tags start collapsed.
const [expanded, setExpanded] = useState(false);
const inLibraryArea =
pathname === "/library" || pathname.startsWith("/library/");
const topTags = useMemo(
() => [...tags].sort(compareTagsByUseCount).slice(0, MAX_SIDEBAR_TAGS),
[tags],
);
if (!tagsLoaded || tags.length === 0) return null;
return (
<>
<SidebarMenuItem>
<SidebarMenuButton
asChild
isActive={pathname === "/library/tags"}
tooltip="Tags"
>
<Link to="/library/tags" onClick={onNavClick}>
<FontAwesomeIcon icon={faTags} />
<span>Tags</span>
</Link>
</SidebarMenuButton>
{inLibraryArea && topTags.length > 0 && (
<button
type="button"
onClick={() => setExpanded((v) => !v)}
aria-label={expanded ? "Collapse tags" : "Expand tags"}
className="absolute right-1 top-1/2 -translate-y-1/2 flex h-5 w-5 items-center justify-center rounded text-sidebar-foreground/50 hover:bg-sidebar-accent hover:text-sidebar-foreground transition-colors group-data-[collapsible=icon]:hidden"
>
<FontAwesomeIcon
icon={faChevronRight}
className={`text-[10px] transition-transform duration-200 ${expanded ? "rotate-90" : ""}`}
/>
</button>
)}
</SidebarMenuItem>
<AnimatePresence initial={false}>
{inLibraryArea && expanded && topTags.length > 0 && (
<motion.li
key="tag-rows"
initial={{ height: 0, opacity: 0 }}
animate={{ height: "auto", opacity: 1 }}
exit={{ height: 0, opacity: 0 }}
transition={{ duration: 0.28, ease: EASE_EMPHASIS }}
className="list-none overflow-hidden group-data-[collapsible=icon]:hidden"
>
<ul className="flex w-full min-w-0 flex-col gap-0.5">
{topTags.map((tag) => (
<SidebarMenuItem key={tag.token}>
<SidebarMenuButton
isActive={pathname === `/library/${tag.token}`}
tooltip={tag.value}
onClick={() => {
navigate(`/library/${tag.token}`);
onNavClick();
}}
onContextMenu={(e) => {
e.preventDefault();
setContextMenu({
tagToken: tag.token,
x: e.clientX,
y: e.clientY,
});
}}
className="pl-5"
>
<FontAwesomeIcon
icon={faTag}
className="text-violet-400"
/>
<span className="truncate">{tag.value}</span>
<span className="ml-auto text-[10px] text-sidebar-foreground/40 transition-opacity group-hover/menu-item:opacity-0">
{tag.useCount}
</span>
</SidebarMenuButton>
{/* Hover-revealed rename/delete menu (swaps with the count). */}
<button
type="button"
onClick={(e) => {
e.stopPropagation();
const rect = e.currentTarget.getBoundingClientRect();
setContextMenu({
tagToken: tag.token,
x: rect.left,
y: rect.bottom + 4,
});
}}
aria-label={`Options for tag "${tag.value}"`}
className="absolute right-1 top-1/2 -translate-y-1/2 flex h-5 w-5 items-center justify-center rounded text-sidebar-foreground/50 opacity-0 transition-opacity group-hover/menu-item:opacity-100 hover:bg-sidebar-accent hover:text-sidebar-foreground"
>
<FontAwesomeIcon icon={faEllipsis} className="text-xs" />
</button>
</SidebarMenuItem>
))}
{tags.length > MAX_SIDEBAR_TAGS && (
<SidebarMenuItem>
<SidebarMenuButton asChild tooltip="See all tags">
<Link
to="/library/tags"
onClick={onNavClick}
className="pl-5 text-sidebar-foreground/60"
>
<FontAwesomeIcon icon={faEllipsis} />
<span>See all tags</span>
<span className="ml-auto text-[10px] text-sidebar-foreground/40">
{tags.length}
</span>
</Link>
</SidebarMenuButton>
</SidebarMenuItem>
)}
</ul>
</motion.li>
)}
</AnimatePresence>
</>
);
}
@@ -0,0 +1,54 @@
import { useEffect, useMemo } from "react";
import type { UserInfo } from "@storyteller/api";
import { TagsSection as SharedTagsSection } from "@storyteller/ui-lightbox-modal";
import { useSession } from "../../lib/session";
import {
compareTagsByUseCount,
useLibraryTagsStore,
} from "../../pages/library/library-tags-store";
interface TagsSectionProps {
mediaToken?: string | null;
creator?: UserInfo | null;
}
/**
* Store-wired wrapper around the shared tags editor (from
* `@storyteller/ui-lightbox-modal`, also used by the desktop gallery):
* session comes from the webapp session store, autocomplete suggestions from
* the preloaded tags store, and saves feed fresh use counts back into it so
* the sidebar and tag browser stay current.
*/
export function TagsSection({ mediaToken, creator }: TagsSectionProps) {
const { user, loggedIn } = useSession();
const storeTags = useLibraryTagsStore((s) => s.tags);
const loadTags = useLibraryTagsStore((s) => s.loadTags);
const upsertTags = useLibraryTagsStore((s) => s.upsertTags);
const currentUsername = loggedIn ? (user?.username ?? null) : null;
const isOwner =
!!currentUsername && creator?.username === currentUsername;
// Preload the user's tag list once for autocomplete.
useEffect(() => {
if (isOwner && !useLibraryTagsStore.getState().tagsLoaded) loadTags();
}, [isOwner, loadTags]);
const suggestions = useMemo(
() =>
[...storeTags]
.sort(compareTagsByUseCount)
.map((t) => ({ value: t.value, useCount: t.useCount })),
[storeTags],
);
return (
<SharedTagsSection
mediaToken={mediaToken}
creator={creator}
currentUsername={currentUsername}
suggestions={suggestions}
onSaved={upsertTags}
/>
);
}

Some files were not shown because too many files have changed in this diff Show More