mirror of
https://github.com/storytold/artcraft.git
synced 2026-10-30 15:25:57 -04:00
Compare commits
9
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
de780b0355 | ||
|
|
cb5abeff08 | ||
|
|
78c688e104 | ||
|
|
b3d281a7f5 | ||
|
|
fe0610b779 | ||
|
|
24e2b44207 | ||
|
|
db344608b6 | ||
|
|
146b78dcb6 | ||
|
|
c822872f36 |
-47
@@ -1,47 +0,0 @@
|
||||
{
|
||||
"db_name": "MySQL",
|
||||
"query": "\nSELECT\n api_keys.token as `api_key_token: ApiKeyToken`,\n users.token as `user_token: UserToken`,\n users.is_banned\nFROM api_keys\nINNER JOIN users\n ON users.token = api_keys.owner_user_token\nWHERE api_keys.api_key = ?\n AND api_keys.maybe_deleted_at IS NULL\nLIMIT 1\n ",
|
||||
"describe": {
|
||||
"columns": [
|
||||
{
|
||||
"ordinal": 0,
|
||||
"name": "api_key_token: ApiKeyToken",
|
||||
"type_info": {
|
||||
"type": "VarString",
|
||||
"flags": "NOT_NULL | UNIQUE_KEY | BINARY | NO_DEFAULT_VALUE",
|
||||
"char_set": 224,
|
||||
"max_size": 128
|
||||
}
|
||||
},
|
||||
{
|
||||
"ordinal": 1,
|
||||
"name": "user_token: UserToken",
|
||||
"type_info": {
|
||||
"type": "VarString",
|
||||
"flags": "NOT_NULL | UNIQUE_KEY | BINARY | NO_DEFAULT_VALUE",
|
||||
"char_set": 224,
|
||||
"max_size": 128
|
||||
}
|
||||
},
|
||||
{
|
||||
"ordinal": 2,
|
||||
"name": "is_banned",
|
||||
"type_info": {
|
||||
"type": "Tiny",
|
||||
"flags": "NOT_NULL",
|
||||
"char_set": 63,
|
||||
"max_size": 1
|
||||
}
|
||||
}
|
||||
],
|
||||
"parameters": {
|
||||
"Right": 1
|
||||
},
|
||||
"nullable": [
|
||||
false,
|
||||
false,
|
||||
false
|
||||
]
|
||||
},
|
||||
"hash": "f52c495a7ff5cafb357f79c4cdb503bbbef045fd193a5a37a66180fcac77f363"
|
||||
}
|
||||
@@ -1,203 +0,0 @@
|
||||
# PageDraw — Visual Design & Architecture Review
|
||||
|
||||
**Surface:** `@storyteller/ui-pagedraw` image editor + its `artcraft-webapp` integration (`/edit-image`).
|
||||
**Lens:** Awwwards / Linear-tier "premium app chrome" rubric, adapted for an in-product creative tool.
|
||||
**Date:** 2026-06-15 · **Branch:** `worktree-pagedraw-webapp`
|
||||
|
||||
---
|
||||
|
||||
## 0. How to read this
|
||||
|
||||
PageDraw is a **professional canvas tool**, not a marketing page. So the elite-design rubric is
|
||||
applied in its *app-chrome* register — the north star is **Figma / Linear / Photoshop-web**, not an
|
||||
Awwwards landing page. That means:
|
||||
|
||||
- **Applies:** token discipline, concentric radii, hairline + inset-highlight surfaces ("double-bezel"),
|
||||
island/pill controls, custom-eased micro-motion, z-index hygiene, premium type ramp.
|
||||
- **Does _not_ apply:** `py-40` macro-whitespace, hero typography, editorial-split layouts, scroll-reveal
|
||||
entrances. A tool surface is dense and persistent by design; forcing landing-page whitespace would hurt it.
|
||||
|
||||
Citations are `file:line` against the current tree.
|
||||
|
||||
---
|
||||
|
||||
## 1. Verdict
|
||||
|
||||
| Axis | Score | One-liner |
|
||||
|---|---|---|
|
||||
| Color system | ⚠️ 5/10 | Good token foundation, but hardcoded greys (`#1b1b1b`, `#303030`, `#404040`) leak through a *themeable lib*. |
|
||||
| Typography | ❌ 3/10 | No type ramp, no font-family ownership; rides whatever the host sets. Banned-by-rubric default stack risk. |
|
||||
| Radii | ⚠️ 4/10 | Five radius scales in play (`sm`→`2xl`→`full`) with no concentric nesting. |
|
||||
| Surfaces (border/shadow) | ⚠️ 5/10 | Flat `shadow-lg` everywhere; no inset highlights; no double-bezel. Reads "functional", not "machined". |
|
||||
| Motion | ❌ 3/10 | Only `transition-colors`/`ease-in-out`, durations 75–200ms; **zero custom easing**, no spring, no entrance choreography. |
|
||||
| Layout / z-index | ⚠️ 6/10 | Clean fixed-panel composition, but a `z-[9999999]` escape hatch and a 6-tier ad-hoc z-stack. |
|
||||
| Icons | ✅ 7/10 | FontAwesome Pro (solid + regular) — acceptable, but solid weight is heavier than the rubric's "ultra-light line" ideal. |
|
||||
| Architecture | ⚠️ 5/10 | Clean adapter boundary; but `PaintSurface.tsx` (2005 LOC), `SceneState.ts` (1539), `PageDraw.tsx` (1292) are oversized, and the lib hardcodes host-theme colors. |
|
||||
|
||||
**Bottom line:** the editor is *structurally competent and functional* but visually reads as
|
||||
"capable internal tool," not "$150k product surface." The single highest-leverage fix is a **shared
|
||||
design-token layer** (color + radius + motion + type), because today the lib both (a) reaches for
|
||||
host tokens (`bg-ui-panel`, `text-base-fg`) *and* (b) hardcodes its own greys — so it can never be
|
||||
fully reskinned by the webapp vs. the desktop app.
|
||||
|
||||
---
|
||||
|
||||
## 2. What's already good (keep)
|
||||
|
||||
- **Token-aware base.** Most surfaces use semantic tokens (`bg-ui-panel`, `bg-ui-controls`,
|
||||
`text-base-fg`, `border-ui-panel-border`) — the right instinct (SideToolbar, PromptEditor, HistoryStack).
|
||||
- **Clean platform boundary.** The lib has zero REST/Tauri imports; everything I/O flows through
|
||||
`PageDrawAdapter` (`adapter.ts`). The webapp adapter (`web-adapter.tsx`) and desktop adapter stay
|
||||
swappable. This is genuinely good architecture and should be protected.
|
||||
- **Performance-minded canvas.** Composite pre-baking on idle (`PageDraw.tsx` ~661–710) and worker
|
||||
encoding (`generatePipeline.worker.ts`) keep the generate click off the main thread. Don't regress this.
|
||||
- **CORS already handled** for browser canvas export (`crossOrigin="anonymous"` in `Node.ts:134`,
|
||||
`SceneState.ts:1473`, `imageHelpers.ts:16`) — the web port works because of this.
|
||||
|
||||
---
|
||||
|
||||
## 3. Findings & recommendations
|
||||
|
||||
### 3.1 Color — stop hardcoding greys in a themeable lib ⚠️ P0
|
||||
**Current:** semantic tokens *and* raw hex coexist:
|
||||
- `#1b1b1b` pegboard (`pagedraw.css:6`), `#1A1A1A`/`#333333` context menu (`ContextMenu.tsx:44`),
|
||||
`#303030`/`#404040` dividers + popovers (`SideToolbar.tsx:381,550`), `bg-zinc-700/600` swatch
|
||||
(`SideToolbar.tsx:173`), plus ad-hoc `bg-blue-600/500` accents (`PageDraw.tsx:157`, base selector).
|
||||
- Accent color is inconsistent: the lib's primary action uses raw **blue-600**, while the webapp's
|
||||
design system uses `--primary` / `brand-primary`. So the editor's blue doesn't match the product.
|
||||
|
||||
**Why it fails the bar:** a premium component library exposes *one* surface ramp and *one* accent, both
|
||||
driven by tokens, so it reskins per host. Hardcoded greys mean the desktop and web themes can drift, and
|
||||
the raw `blue-600` clashes with `brand-primary` used elsewhere in the webapp.
|
||||
|
||||
**Do:**
|
||||
- Replace every raw grey with the existing ramp: `#1b1b1b`→`bg-ui-background`, `#1A1A1A`→`bg-ui-panel`,
|
||||
`#303030`/`#404040`→`border-ui-panel-border` / `bg-ui-controls`. For the pegboard, add a token
|
||||
`--st-canvas` so hosts can theme the canvas backdrop.
|
||||
- Replace `blue-600/500` accents with `bg-primary` / `text-primary` (the token the rest of the webapp uses),
|
||||
so the Generate/Edit-3D buttons match `brand-primary`.
|
||||
- Move the react-colorful overrides off pure `rgb(255,255,255)` borders (`pagedraw.css:15,19`) onto
|
||||
`white/15` hairlines for consistency with the rest of the chrome.
|
||||
|
||||
### 3.2 Typography — own a type ramp ❌ P0
|
||||
**Current:** no `font-family` anywhere in the lib (grep: 0 hits); only `text-xs/sm/base/lg/xl` +
|
||||
`font-medium/semibold/bold`. The editor inherits whatever the host's body font is.
|
||||
|
||||
**Why it fails the bar:** the rubric explicitly bans the default web stack (Inter/Roboto/Arial/Helvetica).
|
||||
Inheriting an unspecified stack means the tool can render in a banned font with no recourse, and there's
|
||||
no deliberate hierarchy (numerics in sliders, labels, modal titles all share one undifferentiated ramp).
|
||||
|
||||
**Do:**
|
||||
- Adopt the product's display/grotesk token (e.g. a `font-sans` mapped to Geist / Plus Jakarta Sans at
|
||||
the app level) and reference it in the lib's root container, not per-component.
|
||||
- Define a small semantic ramp: `text-[11px] tracking-[0.16em] uppercase` for toolbar/section labels,
|
||||
`text-sm` for controls, `text-base font-semibold` for modal titles. Apply `tabular-nums` to slider
|
||||
read-outs (`SliderWithIndicator.tsx`) and dimension fields (`BlankCanvasModal`) so numbers don't jitter.
|
||||
|
||||
### 3.3 Radii — collapse to a concentric scale ⚠️ P1
|
||||
**Current:** `rounded-sm / md / lg / xl / 2xl / full` all appear, with no parent→child relationship
|
||||
(e.g. `rounded-2xl` card at `PageDraw.tsx:1123`, `rounded-xl` panels in SideToolbar, `rounded-lg`
|
||||
buttons in InpaintToolBar).
|
||||
|
||||
**Do:** pick a 3-step system — **`rounded-2xl` shells, `rounded-xl` inner cards, `rounded-lg` controls,
|
||||
`rounded-full` pills** — and apply concentrically (the "double-bezel" curve math: inner radius =
|
||||
outer − padding). Drop `rounded-sm`/`rounded-md` from the editor entirely.
|
||||
|
||||
### 3.4 Surfaces — add the "machined" double-bezel ⚠️ P1
|
||||
**Current:** panels are a single `bg-ui-panel` + `border` + `shadow-lg`. No nested enclosure, no inset
|
||||
top-highlight. Reads flat.
|
||||
|
||||
**Do (for the four floating panels — SideToolbar, HistoryStack, PromptEditor, InpaintToolBar):**
|
||||
- Outer shell: `bg-ui-panel/80 ring-1 ring-white/10 backdrop-blur-xl p-1.5 rounded-2xl`.
|
||||
- Inner core: own bg + `shadow-[inset_0_1px_0_rgba(255,255,255,0.08)] rounded-xl`.
|
||||
- This is the single change that most moves the surface from "tool" to "premium hardware." The panels are
|
||||
already `fixed` (`PageDraw.tsx:1144,1175,1195`), so `backdrop-blur` here is GPU-safe per the perf rules.
|
||||
|
||||
### 3.5 Motion — replace linear/ease with spring-eased micro-motion ❌ P0 (cheap, high impact)
|
||||
**Current:** `transition-colors`/`transition-all` with `ease-in-out` and `duration-75/150/200`
|
||||
(`SideToolbar.tsx:381,550`, `SliderWithIndicator.tsx:65`, `ContextMenu.tsx:53`). No custom easing,
|
||||
no entrance/exit on popovers, color pickers, or the context menu (they pop instantly).
|
||||
|
||||
**Why it fails the bar:** instant state changes and `ease-in-out` are explicitly banned. A pro tool still
|
||||
deserves *interpolated* motion — it's what separates Figma from a jQuery plugin.
|
||||
|
||||
**Do:**
|
||||
- Standardize on one curve: `transition-[transform,opacity,background-color] duration-300
|
||||
ease-[cubic-bezier(0.32,0.72,0,1)]`.
|
||||
- Tool buttons: `active:scale-[0.97]` press physics; selected tool gets a spring-eased indicator slide
|
||||
rather than a hard background swap.
|
||||
- Popovers / color picker / context menu: animate in with `opacity-0 translate-y-1 scale-[0.98]` →
|
||||
resolved, ~180ms. Transform+opacity only (perf rule).
|
||||
- History tiles: stagger their `opacity` reveal when a generation lands (they already use
|
||||
`transition-opacity`, just add per-index delay).
|
||||
|
||||
### 3.6 Layout & z-index — tame the stack ⚠️ P1
|
||||
**Current:** clean fixed-panel composition, but z-index is ad-hoc across 6 tiers and includes a
|
||||
`z-[9999999]` on the context menu (`ContextMenu.tsx:44`) and a one-off `z-[60]` (`PageDraw.tsx:237`).
|
||||
|
||||
**Do:** define a named scale in one place — `canvas:0, chrome:10, floating-controls:20, overlay-button:40,
|
||||
menu:50` — and replace the magic numbers. The portal menu only needs to clear the chrome, not `9999999`.
|
||||
|
||||
### 3.7 Responsive — declare intent ⚠️ P2
|
||||
**Current:** essentially desktop-only; fixed sidebars assume a wide viewport; only LightboxDetails has
|
||||
`sm:` variants. The webapp wrapper scopes the lib correctly (`pagedraw.tsx` `translateZ(0)` containing
|
||||
block) but there's no mobile story.
|
||||
|
||||
**Do:** mirror the Edit-3D page's explicit **mobile gate** (`pagescene.tsx` `MobileGate`) on `/edit-image`
|
||||
rather than letting fixed toolbars overlap on touch. A precision canvas tool is legitimately desktop-first;
|
||||
just say so deliberately instead of degrading silently.
|
||||
|
||||
---
|
||||
|
||||
## 4. Architecture notes
|
||||
|
||||
- **Oversized modules.** `PaintSurface.tsx` (2005 LOC), `SceneState.ts` (1539), `PageDraw.tsx` (1292).
|
||||
These are review/maintenance hotspots. Suggest extracting from `SceneState.ts`: history/undo slice,
|
||||
base-image-load slice, and generation/pending slice into composable Zustand slices. `PageDraw.tsx`'s
|
||||
generate-flow (cases 1–3, ~780–868) could move to a `useGenerate()` hook.
|
||||
- **Theme ownership.** Per §3.1, the lib should *only* consume tokens. Today it mixes tokens + hex, which
|
||||
defeats the adapter pattern's promise (one component, many themed hosts).
|
||||
- **Adapter surface is healthy.** `PageDrawAdapter` is small and intentional; the web vs. Tauri split is
|
||||
clean. The webapp's `enqueueInpaint` stub and 402→credits-modal handling are explicit and documented.
|
||||
- **Icon weight.** FontAwesome **solid** is the heaviest option; if a line/light set is available
|
||||
(Phosphor/Remix or FA light), the toolbar would read more refined. Low priority.
|
||||
|
||||
---
|
||||
|
||||
## 5. Prioritized roadmap
|
||||
|
||||
| P | Item | Effort | Payoff |
|
||||
|---|---|---|---|
|
||||
| **P0** | Token sweep: kill hardcoded greys; unify accent on `--primary` (§3.1) | M | Highest — enables true theming + product-consistent accent |
|
||||
| **P0** | Own a font-family + semantic type ramp at the lib root (§3.2) | S | High — removes banned-font risk, adds hierarchy |
|
||||
| **P0** | One custom easing curve + press/popover micro-motion (§3.5) | S | High — cheapest "feels expensive" win |
|
||||
| **P1** | Double-bezel + inset highlights on the 4 floating panels (§3.4) | M | High — "machined" surface upgrade |
|
||||
| **P1** | Concentric 3-step radius system (§3.3) | S | Medium — visual coherence |
|
||||
| **P1** | Named z-index scale; remove `z-[9999999]` (§3.6) | S | Medium — maintainability |
|
||||
| **P2** | Explicit mobile gate on `/edit-image` (§3.7) | S | Medium — correctness on touch |
|
||||
| **P2** | Split `SceneState.ts` / `PaintSurface.tsx` into slices (§4) | L | Medium — maintainability |
|
||||
|
||||
---
|
||||
|
||||
## 6. Appendix — proposed token additions
|
||||
|
||||
```css
|
||||
/* canvas + chrome surface ramp (lib should consume only these) */
|
||||
--st-canvas: <theme>; /* replaces #1b1b1b pegboard */
|
||||
--st-accent: var(--primary); /* replaces blue-600/500 */
|
||||
--st-hairline: rgb(255 255 255 / 0.10);
|
||||
--st-inset-hi: inset 0 1px 0 rgb(255 255 255 / 0.08);
|
||||
|
||||
/* motion */
|
||||
--st-ease: cubic-bezier(0.32, 0.72, 0, 1);
|
||||
--st-dur-fast: 180ms;
|
||||
--st-dur: 300ms;
|
||||
|
||||
/* radius scale (concentric) */
|
||||
--st-r-shell: 1.25rem; /* rounded-2xl */
|
||||
--st-r-card: 0.875rem; /* rounded-xl */
|
||||
--st-r-control: 0.5rem; /* rounded-lg */
|
||||
```
|
||||
|
||||
> Scope note: every recommendation here is **styling/architecture only** — none change the generate,
|
||||
> polling, upload, bg-removal, or adapter contracts validated during the functional debug pass.
|
||||
@@ -8,7 +8,6 @@ pub mod generate;
|
||||
pub mod jobs;
|
||||
pub mod media_file;
|
||||
pub mod moderation;
|
||||
pub mod omni_api;
|
||||
pub mod omni_gen;
|
||||
pub mod prompts;
|
||||
pub mod stripe_artcraft;
|
||||
|
||||
@@ -1,3 +0,0 @@
|
||||
pub mod omni_upload_audio;
|
||||
pub mod omni_upload_image;
|
||||
pub mod omni_upload_video;
|
||||
@@ -1,15 +0,0 @@
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
|
||||
/// Success response for the API-key authenticated audio upload endpoint
|
||||
/// (`POST /v1/omni_api/upload/audio`).
|
||||
///
|
||||
/// NB: The request is a `multipart/form-data` body. Because the multipart form type is bound to the
|
||||
/// web framework, it is declared alongside the handler in `storyteller_web` rather than here.
|
||||
#[derive(Clone, Serialize, Deserialize, ToSchema, Debug)]
|
||||
pub struct OmniUploadAudioMediaFileSuccessResponse {
|
||||
pub success: bool,
|
||||
pub media_file_token: MediaFileToken,
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
|
||||
/// Success response for the API-key authenticated image upload endpoint
|
||||
/// (`POST /v1/omni_api/upload/image`).
|
||||
///
|
||||
/// NB: The request is a `multipart/form-data` body. Because the multipart form type is bound to the
|
||||
/// web framework, it is declared alongside the handler in `storyteller_web` rather than here.
|
||||
#[derive(Clone, Serialize, Deserialize, ToSchema, Debug)]
|
||||
pub struct OmniUploadImageMediaFileSuccessResponse {
|
||||
pub success: bool,
|
||||
pub media_file_token: MediaFileToken,
|
||||
}
|
||||
@@ -1,15 +0,0 @@
|
||||
use serde_derive::{Deserialize, Serialize};
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
|
||||
/// Success response for the API-key authenticated video upload endpoint
|
||||
/// (`POST /v1/omni_api/upload/video`).
|
||||
///
|
||||
/// NB: The request is a `multipart/form-data` body. Because the multipart form type is bound to the
|
||||
/// web framework, it is declared alongside the handler in `storyteller_web` rather than here.
|
||||
#[derive(Clone, Serialize, Deserialize, ToSchema, Debug)]
|
||||
pub struct OmniUploadVideoMediaFileSuccessResponse {
|
||||
pub success: bool,
|
||||
pub media_file_token: MediaFileToken,
|
||||
}
|
||||
-63
@@ -1,63 +0,0 @@
|
||||
use std::marker::PhantomData;
|
||||
|
||||
use sqlx::{Executor, MySql};
|
||||
|
||||
use artcraft_api_keys::ArtcraftApiKey;
|
||||
use tokens::tokens::api_keys::ApiKeyToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
use crate::helpers::boolean_converters::i8_to_bool;
|
||||
|
||||
pub struct GetApiKeyUserByValueArgs<'e, 'c, E>
|
||||
where
|
||||
E: 'e + Executor<'c, Database = MySql>,
|
||||
{
|
||||
/// The actual API key secret presented by the client.
|
||||
pub api_key: &'e ArtcraftApiKey,
|
||||
pub mysql_executor: E,
|
||||
pub phantom: PhantomData<&'c E>,
|
||||
}
|
||||
|
||||
/// A live API key joined to its owning user. This is the minimal identity needed to authenticate
|
||||
/// an API-key request — far smaller than a cookie `SessionUserRecord`.
|
||||
pub struct ApiKeyUserRecord {
|
||||
pub api_key_token: ApiKeyToken,
|
||||
pub user_token: UserToken,
|
||||
pub is_banned: bool,
|
||||
}
|
||||
|
||||
/// Look up the user behind an API key secret, for authenticating an incoming request.
|
||||
///
|
||||
/// Returns `Ok(None)` when no row matches, when the matching API key is soft-deleted
|
||||
/// (`maybe_deleted_at IS NOT NULL`), or when the owning user row does not exist (the `INNER JOIN`
|
||||
/// requires both records to be present). The full secret is never echoed back.
|
||||
pub async fn get_api_key_user_by_value<'e, 'c: 'e, E>(
|
||||
args: GetApiKeyUserByValueArgs<'e, 'c, E>,
|
||||
) -> Result<Option<ApiKeyUserRecord>, sqlx::Error>
|
||||
where
|
||||
E: 'e + Executor<'c, Database = MySql>,
|
||||
{
|
||||
let result = sqlx::query!(
|
||||
r#"
|
||||
SELECT
|
||||
api_keys.token as `api_key_token: ApiKeyToken`,
|
||||
users.token as `user_token: UserToken`,
|
||||
users.is_banned
|
||||
FROM api_keys
|
||||
INNER JOIN users
|
||||
ON users.token = api_keys.owner_user_token
|
||||
WHERE api_keys.api_key = ?
|
||||
AND api_keys.maybe_deleted_at IS NULL
|
||||
LIMIT 1
|
||||
"#,
|
||||
args.api_key.as_str_be_careful(),
|
||||
)
|
||||
.fetch_optional(args.mysql_executor)
|
||||
.await?;
|
||||
|
||||
Ok(result.map(|r| ApiKeyUserRecord {
|
||||
api_key_token: r.api_key_token,
|
||||
user_token: r.user_token,
|
||||
is_banned: i8_to_bool(r.is_banned),
|
||||
}))
|
||||
}
|
||||
@@ -1,7 +1,6 @@
|
||||
pub mod api_key_row;
|
||||
pub mod delete_api_key;
|
||||
pub mod get_api_key_by_token;
|
||||
pub mod get_api_key_user_by_value;
|
||||
pub mod get_live_api_key_by_value;
|
||||
pub mod insert_api_key;
|
||||
pub mod list_api_keys_for_user;
|
||||
|
||||
@@ -19,7 +19,6 @@ pub enum UserFeatureFlag {
|
||||
/// (For now, it's hidden until we get an NSFW filter.)
|
||||
ExploreMedia,
|
||||
|
||||
|
||||
/// Access to studio features
|
||||
Studio,
|
||||
|
||||
@@ -45,10 +44,6 @@ pub enum UserFeatureFlag {
|
||||
/// Access to the referrals program
|
||||
#[serde(rename = "referrals")]
|
||||
ReferralsProgram,
|
||||
|
||||
/// Access to API key creation
|
||||
#[serde(rename = "api_key")]
|
||||
ApiKey,
|
||||
}
|
||||
|
||||
// TODO(bt, 2022-12-21): This desperately needs MySQL integration tests!
|
||||
@@ -68,7 +63,6 @@ impl UserFeatureFlag {
|
||||
Self::HappyHorse => "hh",
|
||||
Self::HappyHorseRateLimit => "hh_rl",
|
||||
Self::ReferralsProgram => "referrals",
|
||||
Self::ApiKey => "api_key",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -82,7 +76,6 @@ impl UserFeatureFlag {
|
||||
"hh" => Ok(Self::HappyHorse),
|
||||
"hh_rl" => Ok(Self::HappyHorseRateLimit),
|
||||
"referrals" => Ok(Self::ReferralsProgram),
|
||||
"api_key" => Ok(Self::ApiKey),
|
||||
_ => Err(format!("invalid value: {:?}", value)),
|
||||
}
|
||||
}
|
||||
@@ -97,7 +90,6 @@ impl UserFeatureFlag {
|
||||
Self::HappyHorse => "Happy Horse (no Rate Limit)",
|
||||
Self::HappyHorseRateLimit => "Happy Horse (with Rate Limit)",
|
||||
Self::ReferralsProgram => "Referrals Program",
|
||||
Self::ApiKey => "API Keys",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -111,7 +103,6 @@ impl UserFeatureFlag {
|
||||
Self::HappyHorse => "Access to Happy Horse (without a rate limit - dangerous)",
|
||||
Self::HappyHorseRateLimit => "Access to Happy Horse (with a rate limit)",
|
||||
Self::ReferralsProgram => "Access to the referrals program",
|
||||
Self::ApiKey => "Access to API key creation",
|
||||
}
|
||||
}
|
||||
|
||||
@@ -127,7 +118,6 @@ impl UserFeatureFlag {
|
||||
Self::HappyHorse,
|
||||
Self::HappyHorseRateLimit,
|
||||
Self::ReferralsProgram,
|
||||
Self::ApiKey,
|
||||
])
|
||||
}
|
||||
}
|
||||
@@ -150,7 +140,6 @@ mod tests {
|
||||
assert_serialization(UserFeatureFlag::HappyHorse, "hh");
|
||||
assert_serialization(UserFeatureFlag::HappyHorseRateLimit, "hh_rl");
|
||||
assert_serialization(UserFeatureFlag::ReferralsProgram, "referrals");
|
||||
assert_serialization(UserFeatureFlag::ApiKey, "api_key");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -163,7 +152,6 @@ mod tests {
|
||||
assert_eq!(UserFeatureFlag::HappyHorse.to_str(), "hh");
|
||||
assert_eq!(UserFeatureFlag::HappyHorseRateLimit.to_str(), "hh_rl");
|
||||
assert_eq!(UserFeatureFlag::ReferralsProgram.to_str(), "referrals");
|
||||
assert_eq!(UserFeatureFlag::ApiKey.to_str(), "api_key");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -176,14 +164,13 @@ mod tests {
|
||||
assert_eq!(UserFeatureFlag::from_str("hh").unwrap(), UserFeatureFlag::HappyHorse);
|
||||
assert_eq!(UserFeatureFlag::from_str("hh_rl").unwrap(), UserFeatureFlag::HappyHorseRateLimit);
|
||||
assert_eq!(UserFeatureFlag::from_str("referrals").unwrap(), UserFeatureFlag::ReferralsProgram);
|
||||
assert_eq!(UserFeatureFlag::from_str("api_key").unwrap(), UserFeatureFlag::ApiKey);
|
||||
assert!(UserFeatureFlag::from_str("foo").is_err());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn all_variants() {
|
||||
let mut variants = UserFeatureFlag::all_variants();
|
||||
assert_eq!(variants.len(), 9);
|
||||
assert_eq!(variants.len(), 8);
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::ExploreMedia));
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::Studio));
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::Upload3d));
|
||||
@@ -192,7 +179,6 @@ mod tests {
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::HappyHorse));
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::HappyHorseRateLimit));
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::ReferralsProgram));
|
||||
assert_eq!(variants.pop_first(), Some(UserFeatureFlag::ApiKey));
|
||||
assert_eq!(variants.pop_first(), None);
|
||||
}
|
||||
}
|
||||
|
||||
@@ -26,9 +26,6 @@ pub enum PlatformType {
|
||||
/// Desktop app users
|
||||
DesktopApp,
|
||||
|
||||
/// Requests authenticated via an `Authorization` header API key (the Omni API surface).
|
||||
ApiKey,
|
||||
|
||||
// ========== Other clients ==========
|
||||
|
||||
Curl,
|
||||
@@ -45,7 +42,6 @@ impl PlatformType {
|
||||
match self {
|
||||
Self::Web => "web",
|
||||
Self::DesktopApp => "desktop_app",
|
||||
Self::ApiKey => "api_key",
|
||||
Self::Curl => "curl",
|
||||
Self::PythonRequests => "python_requests",
|
||||
Self::Postman => "postman",
|
||||
@@ -56,7 +52,6 @@ impl PlatformType {
|
||||
match value {
|
||||
"web" => Ok(Self::Web),
|
||||
"desktop_app" => Ok(Self::DesktopApp),
|
||||
"api_key" => Ok(Self::ApiKey),
|
||||
"curl" => Ok(Self::Curl),
|
||||
"python_requests" => Ok(Self::PythonRequests),
|
||||
"postman" => Ok(Self::Postman),
|
||||
@@ -69,7 +64,6 @@ impl PlatformType {
|
||||
BTreeSet::from([
|
||||
Self::Web,
|
||||
Self::DesktopApp,
|
||||
Self::ApiKey,
|
||||
Self::Curl,
|
||||
Self::PythonRequests,
|
||||
Self::Postman,
|
||||
@@ -90,7 +84,6 @@ mod tests {
|
||||
fn test_serialization() {
|
||||
assert_serialization(PlatformType::Web, "web");
|
||||
assert_serialization(PlatformType::DesktopApp, "desktop_app");
|
||||
assert_serialization(PlatformType::ApiKey, "api_key");
|
||||
assert_serialization(PlatformType::Curl, "curl");
|
||||
assert_serialization(PlatformType::PythonRequests, "python_requests");
|
||||
assert_serialization(PlatformType::Postman, "postman");
|
||||
@@ -100,7 +93,6 @@ mod tests {
|
||||
fn to_str() {
|
||||
assert_eq!(PlatformType::Web.to_str(), "web");
|
||||
assert_eq!(PlatformType::DesktopApp.to_str(), "desktop_app");
|
||||
assert_eq!(PlatformType::ApiKey.to_str(), "api_key");
|
||||
assert_eq!(PlatformType::Curl.to_str(), "curl");
|
||||
assert_eq!(PlatformType::PythonRequests.to_str(), "python_requests");
|
||||
assert_eq!(PlatformType::Postman.to_str(), "postman");
|
||||
@@ -110,7 +102,6 @@ mod tests {
|
||||
fn from_str() {
|
||||
assert_eq!(PlatformType::from_str("web").unwrap(), PlatformType::Web);
|
||||
assert_eq!(PlatformType::from_str("desktop_app").unwrap(), PlatformType::DesktopApp);
|
||||
assert_eq!(PlatformType::from_str("api_key").unwrap(), PlatformType::ApiKey);
|
||||
assert_eq!(PlatformType::from_str("curl").unwrap(), PlatformType::Curl);
|
||||
assert_eq!(PlatformType::from_str("python_requests").unwrap(), PlatformType::PythonRequests);
|
||||
assert_eq!(PlatformType::from_str("postman").unwrap(), PlatformType::Postman);
|
||||
@@ -130,10 +121,9 @@ mod tests {
|
||||
#[test]
|
||||
fn all_variants() {
|
||||
let mut variants = PlatformType::all_variants();
|
||||
assert_eq!(variants.len(), 6);
|
||||
assert_eq!(variants.len(), 5);
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::Web));
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::DesktopApp));
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::ApiKey));
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::Curl));
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::PythonRequests));
|
||||
assert_eq!(variants.pop_first(), Some(PlatformType::Postman));
|
||||
|
||||
@@ -80,9 +80,6 @@ use crate::http_server::endpoints::media_files::upload::upload_spz_media_file_ha
|
||||
use crate::http_server::endpoints::media_files::upload::upload_studio_shot::upload_studio_shot_media_file_handler::*;
|
||||
use crate::http_server::endpoints::media_files::upload::upload_video_new::upload_new_video_media_file_handler::*;
|
||||
use crate::http_server::endpoints::media_files::upload::upload_video_old::upload_video_media_file_handler::*;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_audio_media_file_handler::*;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_image_media_file_handler::*;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_video_media_file_handler::*;
|
||||
use crate::http_server::endpoints::media_files::upsert_upload::write_engine_asset::write_engine_asset_media_file_handler::*;
|
||||
use crate::http_server::endpoints::media_files::upsert_upload::write_error::MediaFileWriteError;
|
||||
use crate::http_server::endpoints::media_files::upsert_upload::write_scene_file::write_scene_file_media_file_handler::*;
|
||||
@@ -346,9 +343,6 @@ use artcraft_api_defs::omni_gen::generate_response::omni_gen_image_generate_resp
|
||||
use artcraft_api_defs::omni_gen::generate_response::omni_gen_video_generate_response::*;
|
||||
use artcraft_api_defs::omni_gen::models::omni_gen_image_models::*;
|
||||
use artcraft_api_defs::omni_gen::models::omni_gen_video_models::*;
|
||||
use artcraft_api_defs::omni_api::omni_upload_audio::*;
|
||||
use artcraft_api_defs::omni_api::omni_upload_image::*;
|
||||
use artcraft_api_defs::omni_api::omni_upload_video::*;
|
||||
use artcraft_api_defs::moderation::user::list_subscribing_users_by_signup_date::*;
|
||||
use artcraft_api_defs::moderation::user::list_users_by_signup_date::*;
|
||||
use artcraft_api_defs::moderation::user::user_lookup::*;
|
||||
@@ -556,12 +550,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
|
||||
crate::http_server::endpoints::omni_gen::generate::video::omni_gen_video_generate_handler::omni_gen_video_generate_handler,
|
||||
crate::http_server::endpoints::omni_gen::models::image::omni_gen_image_models_handler::omni_gen_image_models_handler,
|
||||
crate::http_server::endpoints::omni_gen::models::video::omni_gen_video_models_handler::omni_gen_video_models_handler,
|
||||
// Omni API (API-key authenticated generate)
|
||||
crate::http_server::endpoints::omni_api::generate::image::omni_api_image_generate_handler::omni_api_image_generate_handler,
|
||||
crate::http_server::endpoints::omni_api::generate::video::omni_api_video_generate_handler::omni_api_video_generate_handler,
|
||||
crate::http_server::endpoints::omni_api::upload::omni_upload_audio_media_file_handler::omni_upload_audio_media_file_handler,
|
||||
crate::http_server::endpoints::omni_api::upload::omni_upload_image_media_file_handler::omni_upload_image_media_file_handler,
|
||||
crate::http_server::endpoints::omni_api::upload::omni_upload_video_media_file_handler::omni_upload_video_media_file_handler,
|
||||
// Moderation
|
||||
crate::http_server::endpoints::moderation::alerts::moderation_send_alert_handler::moderation_send_alert_handler,
|
||||
crate::http_server::endpoints::moderation::info::moderator_token_info_handler::moderator_get_token_info_handler,
|
||||
@@ -1089,12 +1077,6 @@ use crate::http_server::endpoints::media_files::list::list_batch_generated_redux
|
||||
UploadEngineAssetMediaSuccessResponse,
|
||||
UploadImageMediaFileForm,
|
||||
UploadImageMediaFileSuccessResponse,
|
||||
OmniUploadAudioMediaFileForm,
|
||||
OmniUploadAudioMediaFileSuccessResponse,
|
||||
OmniUploadImageMediaFileForm,
|
||||
OmniUploadImageMediaFileSuccessResponse,
|
||||
OmniUploadVideoMediaFileForm,
|
||||
OmniUploadVideoMediaFileSuccessResponse,
|
||||
UserImpersonationRequestResponse,
|
||||
UploadMediaSuccessResponse,
|
||||
UploadNewEngineAssetFileForm,
|
||||
|
||||
-10
@@ -10,7 +10,6 @@ use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use mysql_queries::queries::api_keys::insert_api_key::{insert_api_key, InsertApiKeyArgs};
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::session::lookup::user_session_feature_flags::UserSessionFeatureFlags;
|
||||
use crate::http_server::web_utils::user_session::require_user_session::require_user_session;
|
||||
use crate::state::server_state::ServerState;
|
||||
use crate::util::generate_api_key::generate_api_key;
|
||||
@@ -30,7 +29,6 @@ const MAX_DESCRIPTION_LEN: usize = 512;
|
||||
(status = 200, body = CreateApiKeySuccessResponse),
|
||||
(status = 400, body = CommonWebError),
|
||||
(status = 401, body = CommonWebError),
|
||||
(status = 403, body = CommonWebError),
|
||||
(status = 500, body = CommonWebError),
|
||||
),
|
||||
)]
|
||||
@@ -46,14 +44,6 @@ pub async fn create_api_key_handler(
|
||||
|
||||
let user_session = require_user_session(&http_request, &server_state.session_checker, &mut *conn).await?;
|
||||
|
||||
// API key creation is gated behind the `api_key` feature flag.
|
||||
let feature_flags = UserSessionFeatureFlags::new(user_session.maybe_feature_flags.as_deref());
|
||||
|
||||
if !feature_flags.can_create_api_key() {
|
||||
warn!("User {:?} attempted to create an API key without the api_key feature flag", user_session.user_token.as_str());
|
||||
return Err(CommonWebError::Forbidden);
|
||||
}
|
||||
|
||||
let name = request.name.trim().to_string();
|
||||
|
||||
let maybe_description = request.maybe_description.as_ref().map(|s| s.trim().to_string());
|
||||
|
||||
@@ -17,7 +17,6 @@ pub mod media_files;
|
||||
pub mod misc;
|
||||
pub mod model_download;
|
||||
pub mod moderation;
|
||||
pub mod omni_api;
|
||||
pub mod omni_gen;
|
||||
pub mod prompts;
|
||||
pub mod service;
|
||||
|
||||
-100
@@ -1,100 +0,0 @@
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_image_cost_and_generate_request::OmniGenImageCostAndGenerateRequest;
|
||||
use artcraft_router::api::router_aspect_ratio::RouterAspectRatio;
|
||||
use artcraft_router::api::router_image_model::RouterImageModel;
|
||||
use artcraft_router::api::router_quality::RouterQuality;
|
||||
use artcraft_router::api::router_resolution::RouterResolution;
|
||||
use artcraft_router::api::image_list_ref::ImageListRef;
|
||||
use artcraft_router::api::router_provider::RouterProvider;
|
||||
use artcraft_router::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
|
||||
use artcraft_router::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
|
||||
use enums::common::generation::common_aspect_ratio::CommonAspectRatio as CommonAspectRatioEnum;
|
||||
use enums::common::generation::common_image_model::CommonImageModel as CommonImageModelEnum;
|
||||
use enums::common::generation::common_quality::CommonQuality as CommonQualityEnum;
|
||||
use enums::common::generation::common_resolution::CommonResolution as CommonResolutionEnum;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
pub fn hydrate_to_router_request(
|
||||
request: &OmniGenImageCostAndGenerateRequest,
|
||||
) -> Result<GenerateImageRequestBuilder, CommonWebError> {
|
||||
let api_model = request.model.as_ref()
|
||||
.ok_or_else(|| CommonWebError::BadInputWithSimpleMessage(
|
||||
"model is required".to_string(),
|
||||
))?;
|
||||
|
||||
let model = convert_model(api_model)?;
|
||||
|
||||
let aspect_ratio = request.aspect_ratio.as_ref()
|
||||
.map(convert_aspect_ratio)
|
||||
.transpose()?;
|
||||
|
||||
let resolution = request.resolution.as_ref()
|
||||
.map(convert_resolution)
|
||||
.transpose()?;
|
||||
|
||||
let quality = request.quality.as_ref()
|
||||
.map(convert_quality)
|
||||
.transpose()?;
|
||||
|
||||
Ok(GenerateImageRequestBuilder {
|
||||
model,
|
||||
provider: RouterProvider::Artcraft,
|
||||
prompt: request.prompt.clone(),
|
||||
image_inputs: request.image_media_tokens.clone()
|
||||
.map(ImageListRef::MediaFileTokens),
|
||||
resolution,
|
||||
aspect_ratio,
|
||||
quality,
|
||||
image_batch_count: request.image_batch_count,
|
||||
horizontal_angle: request.adjust_horizontal_angle,
|
||||
vertical_angle: request.adjust_vertical_angle,
|
||||
zoom: request.adjust_zoom,
|
||||
request_mismatch_mitigation_strategy: RequestMismatchMitigationStrategy::PayMoreUpgrade,
|
||||
generation_mode_mismatch_strategy: None,
|
||||
idempotency_token: request.idempotency_token.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_model(
|
||||
model: &CommonImageModelEnum,
|
||||
) -> Result<RouterImageModel, CommonWebError> {
|
||||
let json = serde_json::to_string(model)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported image model: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_aspect_ratio(
|
||||
ar: &CommonAspectRatioEnum,
|
||||
) -> Result<RouterAspectRatio, CommonWebError> {
|
||||
let json = serde_json::to_string(ar)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported aspect ratio: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_resolution(
|
||||
res: &CommonResolutionEnum,
|
||||
) -> Result<RouterResolution, CommonWebError> {
|
||||
let json = serde_json::to_string(res)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported resolution: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_quality(
|
||||
quality: &CommonQualityEnum,
|
||||
) -> Result<RouterQuality, CommonWebError> {
|
||||
let json = serde_json::to_string(quality)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported quality: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
-61
@@ -1,61 +0,0 @@
|
||||
use log::warn;
|
||||
|
||||
use enums::common::visibility::Visibility;
|
||||
use mysql_queries::queries::generic_inference::api_providers::fal::insert_generic_inference_job_for_fal_queue::FalCategory;
|
||||
use mysql_queries::queries::generic_inference::api_providers::fal::insert_generic_inference_job_for_fal_queue_with_apriori_job_token::{
|
||||
insert_generic_inference_job_for_fal_queue_with_apriori_job_token,
|
||||
InsertGenericInferenceForFalWithAprioriJobTokenArgs,
|
||||
};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertFalJobArgs<'a, 'tx> {
|
||||
pub external_job_id: &'a str,
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
/// Insert a single Fal-queue inference job row for an image generation.
|
||||
/// Mirrors `omni_gen::generate::video::insert_db_job::insert_fal_job::insert_fal_job`
|
||||
/// but tags the row with `FalCategory::ImageGeneration` so the worker
|
||||
/// routes it to the image-generation queue.
|
||||
pub async fn insert_fal_job(args: InsertFalJobArgs<'_, '_>) -> Result<InferenceJobToken, CommonWebError> {
|
||||
let InsertFalJobArgs {
|
||||
external_job_id,
|
||||
shared,
|
||||
} = args;
|
||||
|
||||
let db_result = insert_generic_inference_job_for_fal_queue_with_apriori_job_token(
|
||||
InsertGenericInferenceForFalWithAprioriJobTokenArgs {
|
||||
apriori_job_token: shared.apriori_job_token,
|
||||
uuid_idempotency_token: shared.idempotency_token,
|
||||
maybe_external_third_party_id: external_job_id,
|
||||
fal_category: FalCategory::ImageGeneration,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
creator_set_visibility: Visibility::Public,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: None,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
maybe_inference_args: None,
|
||||
starting_job_status_override: None,
|
||||
maybe_frontend_failure_category: None,
|
||||
maybe_failure_reason: None,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => Ok(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting fal inference job (image): {:?}", err);
|
||||
Err(CommonWebError::from_error(err))
|
||||
}
|
||||
}
|
||||
}
|
||||
-117
@@ -1,117 +0,0 @@
|
||||
use log::{error, warn};
|
||||
|
||||
use enums::common::visibility::Visibility;
|
||||
use mysql_queries::queries::generic_inference::api_providers::seedance2pro::insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token::{
|
||||
insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token,
|
||||
InsertGenericInferenceForSeedance2ProWithAprioriJobTokenArgs, KinoviVersion,
|
||||
};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::wallet_ledger_entries::WalletLedgerEntryToken;
|
||||
use uuid_utils::uuid::generate_random_uuid;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertSeedance2proJobsArgs<'a, 'tx> {
|
||||
/// The order_id for the primary job (always inserted first).
|
||||
pub primary_order_id: &'a str,
|
||||
|
||||
/// Additional order_ids returned by Kinovi when batch_count > 1. Each
|
||||
/// becomes its own inference job row.
|
||||
pub maybe_additional_order_ids: Option<&'a [String]>,
|
||||
|
||||
/// Wallet ledger entry from the up-front deduction. Stamped on each
|
||||
/// inserted row so the worker can issue a refund on failure.
|
||||
pub maybe_wallet_ledger_entry_token: Option<&'a WalletLedgerEntryToken>,
|
||||
|
||||
/// Which Kinovi account fulfilled the request — Volcengine, BytePlus,
|
||||
/// or BytePlusUltra. For Midjourney image generation today this is
|
||||
/// always Volcengine, but the helper stays version-agnostic so a future
|
||||
/// account-picking layer can use it unchanged.
|
||||
pub kinovi_version: KinoviVersion,
|
||||
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
pub struct InsertSeedance2proJobsResult {
|
||||
pub primary_job_token: InferenceJobToken,
|
||||
pub all_job_tokens: Vec<InferenceJobToken>,
|
||||
}
|
||||
|
||||
/// Insert one Seedance2Pro/Kinovi inference job per returned order_id.
|
||||
/// Mirrors the video-side helper at
|
||||
/// `omni_gen::generate::video::insert_db_job::insert_seedance2pro_jobs`.
|
||||
pub async fn insert_seedance2pro_jobs(args: InsertSeedance2proJobsArgs<'_, '_>) -> Result<InsertSeedance2proJobsResult, CommonWebError> {
|
||||
let InsertSeedance2proJobsArgs {
|
||||
primary_order_id,
|
||||
maybe_additional_order_ids,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
kinovi_version,
|
||||
mut shared,
|
||||
} = args;
|
||||
|
||||
// Build a deduplicated list with primary_order_id first.
|
||||
let mut all_order_ids = vec![primary_order_id.to_string()];
|
||||
|
||||
if let Some(additional) = maybe_additional_order_ids {
|
||||
for id in additional {
|
||||
if id != primary_order_id {
|
||||
all_order_ids.push(id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut all_job_tokens: Vec<InferenceJobToken> = Vec::with_capacity(all_order_ids.len());
|
||||
|
||||
for (i, order_id) in all_order_ids.iter().enumerate() {
|
||||
let job_token = if i == 0 { shared.apriori_job_token.clone() } else { InferenceJobToken::generate() };
|
||||
|
||||
// The idempotency_token column has a max length too tight for the
|
||||
// `{idempotency}-batch-{i}` suffix pattern used elsewhere; mint a
|
||||
// fresh UUID for each additional row instead. The primary row still
|
||||
// wants a deterministic value, but historic behavior on the video
|
||||
// side has been to mint UUIDs for all rows, which we mirror here.
|
||||
let idempotency_str = generate_random_uuid();
|
||||
|
||||
let db_result = insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token(
|
||||
InsertGenericInferenceForSeedance2ProWithAprioriJobTokenArgs {
|
||||
kinovi_version,
|
||||
apriori_job_token: &job_token,
|
||||
uuid_idempotency_token: &idempotency_str,
|
||||
maybe_external_third_party_id: order_id,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
creator_set_visibility: Visibility::Public,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: None,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
maybe_inference_args: None,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => all_job_tokens.push(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting seedance2pro inference job (image, order_id={}): {:?}", order_id, err);
|
||||
if i == 0 { return Err(CommonWebError::from_error(err)); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let primary_job_token = all_job_tokens.first().cloned().ok_or_else(|| {
|
||||
error!("No inference job token was created");
|
||||
CommonWebError::server_error_with_message("No inference job token was created")
|
||||
})?;
|
||||
|
||||
Ok(InsertSeedance2proJobsResult {
|
||||
primary_job_token,
|
||||
all_job_tokens,
|
||||
})
|
||||
}
|
||||
-3
@@ -1,3 +0,0 @@
|
||||
pub mod insert_fal_job;
|
||||
pub mod insert_seedance2pro_jobs;
|
||||
pub mod shared_job_args;
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
use enums::common::generation::common_model_type::CommonModelType;
|
||||
use enums::common::platform_type::PlatformType;
|
||||
use tokens::tokens::anonymous_visitor_tracking::AnonymousVisitorTrackingToken;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
|
||||
use tokens::tokens::prompts::PromptToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
/// Fields every image-side `insert_*_job` helper needs.
|
||||
/// Parallels `omni_gen::generate::video::insert_db_job::shared_job_args::SharedJobArgs`.
|
||||
pub struct SharedJobArgs<'a, 'tx> {
|
||||
pub apriori_job_token: &'a InferenceJobToken,
|
||||
pub idempotency_token: &'a str,
|
||||
pub user_token: &'a UserToken,
|
||||
pub maybe_avt_token: Option<&'a AnonymousVisitorTrackingToken>,
|
||||
pub maybe_prompt_token: Option<&'a PromptToken>,
|
||||
pub maybe_debug_log_event_token: Option<&'a DebugLogEventToken>,
|
||||
pub maybe_model_type: Option<CommonModelType>,
|
||||
pub maybe_platform_type: Option<PlatformType>,
|
||||
pub ip_address: &'a str,
|
||||
pub transaction: &'a mut sqlx::Transaction<'tx, sqlx::MySql>,
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
pub mod omni_api_image_generate_handler;
|
||||
pub(crate) mod hydrate_to_router_request;
|
||||
pub(crate) mod insert_db_job;
|
||||
pub(crate) mod pipeline_result;
|
||||
pub mod pipeline_v2;
|
||||
-310
@@ -1,310 +0,0 @@
|
||||
use std::sync::Arc;
|
||||
|
||||
use actix_web::web::Json;
|
||||
use actix_web::{web, HttpRequest};
|
||||
use log::{error, info, warn};
|
||||
use sqlx::Acquire;
|
||||
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_image_cost_and_generate_request::OmniGenImageCostAndGenerateRequest;
|
||||
use artcraft_api_defs::omni_gen::generate_response::omni_gen_image_generate_response::OmniGenImageGenerateResponse;
|
||||
use artcraft_router::generate::generate_image::generate_image_response::GenerateImageResponse;
|
||||
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
|
||||
use enums::by_table::prompt_context_items::prompt_context_semantic_type::PromptContextSemanticType;
|
||||
use enums::by_table::prompts::prompt_type::PromptType;
|
||||
use enums::common::generation::common_generation_mode::CommonGenerationMode;
|
||||
use enums::common::generation::common_model_type::CommonModelType;
|
||||
use enums::common::generation_provider::GenerationProvider;
|
||||
use enums::common::platform_type::PlatformType;
|
||||
use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use mysql_queries::queries::debug_logs::insert_debug_log::{insert_debug_log, InsertDebugLogArgs};
|
||||
use mysql_queries::queries::generic_inference::api_providers::seedance2pro::insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token::KinoviVersion;
|
||||
use mysql_queries::queries::idepotency_tokens::insert_idempotency_token::insert_idempotency_token;
|
||||
use mysql_queries::queries::prompt_context_items::insert_batch_prompt_context_items::{
|
||||
insert_batch_prompt_context_items, InsertBatchArgs, PromptContextItem,
|
||||
};
|
||||
use mysql_queries::queries::prompts::insert_prompt::{insert_prompt, InsertPromptArgs};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::generate::common::payments_error_test::payments_error_test;
|
||||
use crate::http_server::endpoints::omni_api::generate::image::hydrate_to_router_request::hydrate_to_router_request;
|
||||
use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::insert_fal_job::{insert_fal_job, InsertFalJobArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::image::insert_db_job::shared_job_args::SharedJobArgs;
|
||||
use crate::http_server::endpoints::omni_api::generate::image::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
|
||||
use crate::http_server::session::lookup::user_session_feature_flags::UserSessionFeatureFlags;
|
||||
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
|
||||
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
|
||||
use crate::state::server_state::ServerState;
|
||||
use crate::util::lookup::lookup_media_files_as_cdn_url_list_and_map::lookup_media_files_as_cdn_url_list_and_map;
|
||||
|
||||
/// Generate an image using the omni-api unified endpoint (API-key authenticated).
|
||||
#[utoipa::path(
|
||||
post,
|
||||
tag = "Omni API",
|
||||
path = "/v1/omni_api/generate/image",
|
||||
request_body = OmniGenImageCostAndGenerateRequest,
|
||||
responses(
|
||||
(status = 200, description = "Success", body = OmniGenImageGenerateResponse),
|
||||
(status = 400, description = "Bad input"),
|
||||
(status = 401, description = "Unauthorized"),
|
||||
(status = 402, description = "Payment required"),
|
||||
(status = 500, description = "Server error"),
|
||||
),
|
||||
)]
|
||||
pub async fn omni_api_image_generate_handler(
|
||||
http_request: HttpRequest,
|
||||
request: Json<OmniGenImageCostAndGenerateRequest>,
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
) -> Result<Json<OmniGenImageGenerateResponse>, CommonWebError> {
|
||||
|
||||
info!("request: {:?}", request);
|
||||
|
||||
payments_error_test(&request.prompt.as_deref().unwrap_or(""))?;
|
||||
|
||||
let debug_log_event_token = DebugLogEventToken::generate();
|
||||
|
||||
let maybe_prompt_model_type: Option<CommonModelType> = request.model
|
||||
.as_ref()
|
||||
.map(|m| m.to_common_model_type());
|
||||
|
||||
// ==================== API KEY USER ==================== //
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
|
||||
|
||||
// API-key authentication (Authorization header) instead of a session cookie. Never cached.
|
||||
let api_session = require_api_key_user(&http_request, &mut *mysql_connection).await?;
|
||||
|
||||
let user_token = &api_session.user_token;
|
||||
|
||||
// ==================== MODEL ACCESS CHECK ==================== //
|
||||
|
||||
// API-key users have no per-user feature flags today.
|
||||
let user_feature_flags = UserSessionFeatureFlags::new(None);
|
||||
|
||||
// ==================== IDEMPOTENCY ==================== //
|
||||
|
||||
let idempotency_token = request.idempotency_token.as_deref()
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
|
||||
if let Err(reason) = validate_idempotency_token_format(&idempotency_token) {
|
||||
return Err(CommonWebError::BadInputWithSimpleMessage(reason));
|
||||
}
|
||||
|
||||
insert_idempotency_token(&idempotency_token, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error inserting idempotency token: {:?}", err);
|
||||
CommonWebError::BadInputWithSimpleMessage("repeated idempotency token".to_string())
|
||||
})?;
|
||||
|
||||
// ==================== RESOLVE MEDIA TOKENS ==================== //
|
||||
|
||||
// Look up media file tokens BEFORE distilling. Pipeline execution should not do I/O.
|
||||
let resolved_media = lookup_media_files_as_cdn_url_list_and_map(
|
||||
&http_request,
|
||||
&mut mysql_connection,
|
||||
server_state.server_environment,
|
||||
request.image_media_tokens.as_deref().unwrap_or(&[]),
|
||||
).await?;
|
||||
|
||||
// ==================== HYDRATE ROUTER REQUEST ==================== //
|
||||
|
||||
let router_builder = hydrate_to_router_request(&request)?;
|
||||
|
||||
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
|
||||
|
||||
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
|
||||
apriori_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
debug_log_type: DebugLogType::HttpRequest,
|
||||
message: &serde_json::to_string(&*request).unwrap_or_default(),
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
warn!("Failed to insert HTTP request debug log: {:?}", err);
|
||||
}
|
||||
|
||||
// ==================== PIPELINE ==================== //
|
||||
|
||||
// NB: Release the pooled DB connection before the (slow, external) generation call so we don't
|
||||
// hold a pool slot idle while waiting on the provider — that's what starves the pool and causes
|
||||
// PoolTimedOut on unrelated endpoints. We re-acquire below to write the result.
|
||||
drop(mysql_connection);
|
||||
|
||||
let pipeline_result = run_pipeline_v2(RunPipelineV2Args {
|
||||
router_builder: &router_builder,
|
||||
server_state: &server_state,
|
||||
user_token,
|
||||
resolved_media: &resolved_media,
|
||||
}).await?;
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
|
||||
|
||||
// ==================== DEBUG LOG: FAL REQUEST ==================== //
|
||||
|
||||
if let GenerateImageResponse::Fal(ref fal_payload) = pipeline_result.response {
|
||||
if let Some(ref outbound_request) = fal_payload.maybe_outbound_request {
|
||||
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
|
||||
apriori_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
debug_log_type: DebugLogType::FalRequest,
|
||||
message: &format!("{:#?}", outbound_request),
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
warn!("Failed to insert Fal request debug log: {:?}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== WRITE RESULT ==================== //
|
||||
|
||||
let ip_address = get_request_ip(&http_request);
|
||||
// Omni API requests are always API-key authenticated; hardcode the platform type.
|
||||
let maybe_platform_type = Some(PlatformType::ApiKey);
|
||||
|
||||
let mut transaction = mysql_connection
|
||||
.begin()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error starting MySQL transaction: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
|
||||
// -- Prompt --
|
||||
|
||||
let generation_mode = if request.image_media_tokens.is_some() {
|
||||
CommonGenerationMode::Edit
|
||||
} else {
|
||||
CommonGenerationMode::Text
|
||||
};
|
||||
|
||||
let prompt_result = insert_prompt(InsertPromptArgs {
|
||||
maybe_apriori_prompt_token: None,
|
||||
prompt_type: PromptType::ArtcraftApp,
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
maybe_model_type: maybe_prompt_model_type,
|
||||
maybe_generation_provider: Some(GenerationProvider::Artcraft),
|
||||
maybe_positive_prompt: request.prompt.as_deref(),
|
||||
maybe_negative_prompt: None,
|
||||
maybe_other_args: None,
|
||||
maybe_generation_mode: Some(generation_mode),
|
||||
maybe_aspect_ratio: request.aspect_ratio, // TODO: should be saved from router's decision as it could have changed
|
||||
maybe_resolution: request.resolution,// TODO: should be saved from router's decision as it could have changed
|
||||
maybe_batch_count: request.image_batch_count.map(|c| c as u8),
|
||||
maybe_generate_audio: None, // NB: Images, not video
|
||||
maybe_duration_seconds: None, // NB: Images, not video
|
||||
creator_ip_address: &ip_address,
|
||||
mysql_executor: &mut *transaction,
|
||||
phantom: Default::default(),
|
||||
}).await;
|
||||
|
||||
let prompt_token = match prompt_result {
|
||||
Ok(token) => Some(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting prompt: {:?}", err);
|
||||
None
|
||||
}
|
||||
};
|
||||
|
||||
// -- Prompt context items --
|
||||
|
||||
if let Some(token) = prompt_token.as_ref() {
|
||||
let mut context_items = Vec::new();
|
||||
|
||||
if let Some(ref_tokens) = &request.image_media_tokens {
|
||||
for media_token in ref_tokens {
|
||||
context_items.push(PromptContextItem {
|
||||
media_token: media_token.clone(),
|
||||
context_semantic_type: PromptContextSemanticType::Imgref,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if !context_items.is_empty() {
|
||||
if let Err(err) = insert_batch_prompt_context_items(InsertBatchArgs {
|
||||
prompt_token: token.clone(),
|
||||
items: context_items,
|
||||
transaction: &mut transaction,
|
||||
}).await {
|
||||
warn!("Error inserting batch prompt context items: {:?}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// -- Inference job --
|
||||
//
|
||||
// Each provider has its own queue / worker. Branch by response variant
|
||||
// so the row lands on the correct queue (the Seedance2Pro/Kinovi worker
|
||||
// for Midjourney; the Fal worker for everything else).
|
||||
|
||||
let job_token: InferenceJobToken = match &pipeline_result.response {
|
||||
GenerateImageResponse::Seedance2Pro(payload) => {
|
||||
info!("Inserting seedance2pro image job(s) with token: {:?}", pipeline_result.apriori_job_token);
|
||||
|
||||
// The image-side omni pipeline always dispatches Midjourney via the
|
||||
// Volcengine Kinovi account today. If we ever route to BytePlus /
|
||||
// BytePlus Ultra here, mirror the video-side `kinovi_account` knob.
|
||||
let kinovi_version = KinoviVersion::Volcengine;
|
||||
|
||||
let result = insert_seedance2pro_jobs(InsertSeedance2proJobsArgs {
|
||||
primary_order_id: &payload.order_id,
|
||||
maybe_additional_order_ids: payload.maybe_order_ids.as_deref(),
|
||||
maybe_wallet_ledger_entry_token: pipeline_result.maybe_wallet_ledger_entry_token.as_ref(),
|
||||
kinovi_version,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?;
|
||||
result.primary_job_token
|
||||
}
|
||||
GenerateImageResponse::Fal(payload) => {
|
||||
info!("Inserting fal image job with token: {:?}", pipeline_result.apriori_job_token);
|
||||
let external_job_id = payload.request_id.clone().unwrap_or_default();
|
||||
insert_fal_job(InsertFalJobArgs {
|
||||
external_job_id: &external_job_id,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?
|
||||
}
|
||||
GenerateImageResponse::Artcraft(payload) => {
|
||||
// The omni image pipeline never dispatches via the Artcraft provider
|
||||
// itself today (everything routes to Fal or Kinovi), but the response
|
||||
// variant exists so we cover it defensively — Artcraft jobs come back
|
||||
// already inserted server-side, so just propagate the token.
|
||||
payload.inference_job_token.clone()
|
||||
}
|
||||
};
|
||||
|
||||
transaction.commit().await.map_err(|err| {
|
||||
error!("Error committing transaction: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
|
||||
Ok(Json(OmniGenImageGenerateResponse {
|
||||
success: true,
|
||||
inference_job_token: job_token,
|
||||
}))
|
||||
}
|
||||
-15
@@ -1,15 +0,0 @@
|
||||
use artcraft_router::generate::generate_image::generate_image_response::GenerateImageResponse;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::wallet_ledger_entries::WalletLedgerEntryToken;
|
||||
|
||||
/// Both image pipelines produce this, then the shared handler suffix writes DB rows.
|
||||
pub struct ImagePipelineResult {
|
||||
pub apriori_job_token: InferenceJobToken,
|
||||
pub response: GenerateImageResponse,
|
||||
|
||||
/// The wallet ledger entry created when the user was charged for this
|
||||
/// generation, if any (None when the generation is free). Used as a
|
||||
/// foreign key on the inference job row so the worker can refund on
|
||||
/// failure.
|
||||
pub maybe_wallet_ledger_entry_token: Option<WalletLedgerEntryToken>,
|
||||
}
|
||||
-1
@@ -1 +0,0 @@
|
||||
pub mod run_pipeline_v2;
|
||||
-208
@@ -1,208 +0,0 @@
|
||||
use log::{info, warn};
|
||||
|
||||
use artcraft_router::api::image_list_ref::ImageListRef;
|
||||
use artcraft_router::api::router_image_model::RouterImageModel;
|
||||
use artcraft_router::api::router_provider::RouterProvider;
|
||||
use artcraft_router::client::router_client::RouterClient;
|
||||
use artcraft_router::client::router_fal_client::RouterFalClient;
|
||||
use artcraft_router::client::router_seedance2pro_client::RouterSeedance2ProClient;
|
||||
use artcraft_router::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
|
||||
use artcraft_router::generate::generate_image::generate_image_response::GenerateImageResponse;
|
||||
use artcraft_router::generate::generate_image::image_generation_draft_context::ImageGenerationDraftContext;
|
||||
use artcraft_router::generate::generate_image::image_generation_draft_or_request::ImageGenerationDraftOrRequest;
|
||||
use artcraft_router::generate::generate_image::image_generation_request::ImageGenerationRequest;
|
||||
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
use crate::billing::wallets::attempt_wallet_deduction::attempt_wallet_deduction_else_common_web_error;
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::omni_api::generate::image::pipeline_result::ImagePipelineResult;
|
||||
use crate::http_server::endpoints::omni_api::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
|
||||
use crate::state::server_state::ServerState;
|
||||
use crate::util::lookup::lookup_media_files_as_cdn_url_list_and_map::MediaFilesAsCdnUrlListAndMap;
|
||||
|
||||
pub struct RunPipelineV2Args<'a> {
|
||||
pub router_builder: &'a GenerateImageRequestBuilder,
|
||||
pub server_state: &'a ServerState,
|
||||
pub user_token: &'a UserToken,
|
||||
pub resolved_media: &'a MediaFilesAsCdnUrlListAndMap,
|
||||
}
|
||||
|
||||
// NB: This pipeline does an external generation call (`finalize_and_generate`) that can take many
|
||||
// seconds. It deliberately does NOT hold a pooled DB connection across that call — it acquires a
|
||||
// short-lived connection only for the wallet deduction. Holding a pooled connection across the
|
||||
// external call is what starves the pool and causes `PoolTimedOut`.
|
||||
pub async fn run_pipeline_v2(
|
||||
args: RunPipelineV2Args<'_>,
|
||||
) -> Result<ImagePipelineResult, CommonWebError> {
|
||||
let RunPipelineV2Args {
|
||||
router_builder,
|
||||
server_state,
|
||||
user_token,
|
||||
resolved_media,
|
||||
} = args;
|
||||
|
||||
let hydrated_builder = apply_hydrated_media_inputs(
|
||||
router_builder,
|
||||
resolved_media,
|
||||
);
|
||||
|
||||
let cost = estimate_cost_in_credits(&router_builder)?;
|
||||
|
||||
let draft_or_request = build_execution_request(&hydrated_builder)?;
|
||||
|
||||
info!("Charging wallet: {} credits", cost);
|
||||
|
||||
let apriori_job_token = InferenceJobToken::generate();
|
||||
|
||||
let maybe_wallet_ledger_entry_token = if cost > 0 {
|
||||
// NB: Short-lived connection — released before the external call below.
|
||||
let mut billing_connection = server_state.mysql_pool.acquire().await
|
||||
.map_err(|err| {
|
||||
warn!("Failed to acquire MySQL connection for image billing: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
let deduction = attempt_wallet_deduction_else_common_web_error(
|
||||
user_token,
|
||||
Some(apriori_job_token.as_str()),
|
||||
cost,
|
||||
&mut billing_connection,
|
||||
).await?;
|
||||
Some(deduction.ledger_entry_token)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
// NB: No pooled DB connection is held across this external generation call.
|
||||
let response = finalize_and_generate(draft_or_request, server_state, resolved_media).await?;
|
||||
|
||||
Ok(ImagePipelineResult {
|
||||
apriori_job_token,
|
||||
response,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
})
|
||||
}
|
||||
|
||||
fn build_execution_request(
|
||||
router_builder: &GenerateImageRequestBuilder,
|
||||
) -> Result<ImageGenerationDraftOrRequest, CommonWebError> {
|
||||
let mut execution_builder = router_builder.clone();
|
||||
execution_builder.provider = provider_for_model(execution_builder.model);
|
||||
|
||||
execution_builder.build2().map_err(|e| {
|
||||
warn!("Failed to build2 for image v2 pipeline: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})
|
||||
}
|
||||
|
||||
/// Route each image model to the provider that fulfils it. Midjourney is
|
||||
/// served via the Seedance2Pro/Kinovi (Volcengine) backend; everything else
|
||||
/// flows through Fal.
|
||||
fn provider_for_model(model: RouterImageModel) -> RouterProvider {
|
||||
match model {
|
||||
RouterImageModel::Midjourney7
|
||||
| RouterImageModel::Midjourney7Niji
|
||||
| RouterImageModel::Midjourney8 => RouterProvider::Seedance2Pro,
|
||||
_ => RouterProvider::Fal,
|
||||
}
|
||||
}
|
||||
|
||||
fn apply_hydrated_media_inputs(
|
||||
router_builder: &GenerateImageRequestBuilder,
|
||||
resolved_media: &MediaFilesAsCdnUrlListAndMap,
|
||||
) -> GenerateImageRequestBuilder {
|
||||
let mut hydrated_builder = router_builder.clone();
|
||||
|
||||
match hydrated_builder.image_inputs.as_ref() {
|
||||
Some(ImageListRef::MediaFileTokens(tokens)) if !tokens.is_empty() => {
|
||||
hydrated_builder.image_inputs = Some(ImageListRef::Urls(
|
||||
resolved_media.ordered_url_list.clone(),
|
||||
));
|
||||
},
|
||||
_ => {},
|
||||
}
|
||||
|
||||
hydrated_builder
|
||||
}
|
||||
|
||||
fn estimate_cost_in_credits(
|
||||
router_builder: &GenerateImageRequestBuilder,
|
||||
) -> Result<u64, CommonWebError> {
|
||||
let mut cost_builder = router_builder.clone();
|
||||
cost_builder.provider = RouterProvider::Artcraft;
|
||||
|
||||
let request = cost_builder.build2().map_err(|e| {
|
||||
warn!("Failed to build image cost request: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})?;
|
||||
|
||||
let cost = request.estimate_cost().map_err(|e| {
|
||||
warn!("Failed to estimate image cost: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})?;
|
||||
|
||||
Ok(cost.cost_in_credits.unwrap_or(0))
|
||||
}
|
||||
|
||||
async fn finalize_and_generate(
|
||||
draft_or_request: ImageGenerationDraftOrRequest,
|
||||
server_state: &ServerState,
|
||||
resolved_media: &MediaFilesAsCdnUrlListAndMap,
|
||||
) -> Result<GenerateImageResponse, CommonWebError> {
|
||||
let provider = draft_or_request.get_provider();
|
||||
let client = build_router_client(provider, server_state)?;
|
||||
|
||||
// Inline the draft finalization so the draft context can borrow `client`.
|
||||
let request: ImageGenerationRequest = match draft_or_request {
|
||||
ImageGenerationDraftOrRequest::Request(request) => request,
|
||||
ImageGenerationDraftOrRequest::Draft(draft) => {
|
||||
let draft_context = ImageGenerationDraftContext {
|
||||
client: Some(&client),
|
||||
media_file_to_artcraft_url_map: Some(&resolved_media.token_to_url_map),
|
||||
};
|
||||
draft.finalize(draft_context)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("Failed to finalize image v2 draft: {:?}", err);
|
||||
map_router_error_to_web_error(err)
|
||||
})?
|
||||
}
|
||||
};
|
||||
|
||||
request.send_request(&client)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("v2 image generation failed: {:?}", err);
|
||||
map_router_error_to_web_error(err)
|
||||
})
|
||||
}
|
||||
|
||||
fn build_router_client(
|
||||
provider: RouterProvider,
|
||||
server_state: &ServerState,
|
||||
) -> Result<RouterClient, CommonWebError> {
|
||||
match provider {
|
||||
RouterProvider::Fal => {
|
||||
let fal_client = RouterFalClient::new_with_webhook(
|
||||
server_state.inference_providers.fal.api_key.clone(),
|
||||
server_state.inference_providers.fal.webhook_url.clone(),
|
||||
);
|
||||
Ok(RouterClient::Fal(fal_client))
|
||||
},
|
||||
RouterProvider::Seedance2Pro => {
|
||||
// Midjourney image generation always uses the Volcengine account.
|
||||
// Per-request account selection (like the video flow's
|
||||
// `KinoviAccount` enum) isn't needed here yet — if it ever is, plumb
|
||||
// a parallel knob through `RunPipelineV2Args`.
|
||||
let cookies = server_state.inference_providers.seedance2pro.cookies_volcengine.clone();
|
||||
let session = Seedance2ProSession::from_cookies_string(cookies);
|
||||
Ok(RouterClient::Seedance2Pro(RouterSeedance2ProClient::new(session)))
|
||||
},
|
||||
other => {
|
||||
Err(CommonWebError::server_error_with_message(
|
||||
&format!("Unsupported provider for image v2 generation: {:?}", other),
|
||||
))
|
||||
},
|
||||
}
|
||||
}
|
||||
@@ -1,2 +0,0 @@
|
||||
pub mod image;
|
||||
pub mod video;
|
||||
-44
@@ -1,44 +0,0 @@
|
||||
//! Shared wallet billing logic for both video generation pipelines.
|
||||
|
||||
use log::info;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
use tokens::tokens::wallet_ledger_entries::WalletLedgerEntryToken;
|
||||
|
||||
use crate::billing::wallets::attempt_wallet_deduction::attempt_wallet_deduction_else_common_web_error;
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
pub struct BillWalletResult {
|
||||
pub apriori_job_token: InferenceJobToken,
|
||||
pub maybe_wallet_ledger_entry_token: Option<WalletLedgerEntryToken>,
|
||||
}
|
||||
|
||||
/// Generate an apriori job token and bill the user's wallet for the given cost.
|
||||
///
|
||||
/// If cost is 0, no wallet deduction is made but the apriori token is still generated.
|
||||
pub async fn bill_wallet(
|
||||
user_token: &UserToken,
|
||||
cost: u64,
|
||||
mysql_connection: &mut sqlx::pool::PoolConnection<sqlx::MySql>,
|
||||
) -> Result<BillWalletResult, CommonWebError> {
|
||||
let apriori_job_token = InferenceJobToken::generate();
|
||||
|
||||
info!("Charging wallet: {} credits", cost);
|
||||
|
||||
let maybe_wallet_ledger_entry_token = if cost > 0 {
|
||||
let deduction_result = attempt_wallet_deduction_else_common_web_error(
|
||||
user_token,
|
||||
Some(apriori_job_token.as_str()),
|
||||
cost,
|
||||
mysql_connection,
|
||||
).await?;
|
||||
Some(deduction_result.ledger_entry_token)
|
||||
} else {
|
||||
None
|
||||
};
|
||||
|
||||
Ok(BillWalletResult {
|
||||
apriori_job_token,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
})
|
||||
}
|
||||
-61
@@ -1,61 +0,0 @@
|
||||
//! Build a RouterClient for the given provider from server state.
|
||||
|
||||
use artcraft_router::api::router_provider::RouterProvider;
|
||||
use artcraft_router::client::router_client::RouterClient;
|
||||
use artcraft_router::client::router_fal_client::RouterFalClient;
|
||||
use artcraft_router::client::router_gmicloud_client::RouterGmiCloudClient;
|
||||
use artcraft_router::client::router_grok_api_client::RouterGrokApiClient;
|
||||
use artcraft_router::client::router_seedance2pro_client::RouterSeedance2ProClient;
|
||||
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
pub fn build_router_client(
|
||||
provider: RouterProvider,
|
||||
server_state: &ServerState,
|
||||
kinovi_account: KinoviAccount,
|
||||
) -> Result<RouterClient, CommonWebError> {
|
||||
match provider {
|
||||
RouterProvider::Seedance2Pro => {
|
||||
kinovi_provider(server_state, kinovi_account)
|
||||
}
|
||||
RouterProvider::Fal => {
|
||||
let fal_client = RouterFalClient::new_with_webhook(
|
||||
server_state.inference_providers.fal.api_key.clone(),
|
||||
server_state.inference_providers.fal.webhook_url.clone(),
|
||||
);
|
||||
Ok(RouterClient::Fal(fal_client))
|
||||
}
|
||||
RouterProvider::GmiCloud => {
|
||||
Ok(RouterClient::GmiCloud(RouterGmiCloudClient::new(
|
||||
server_state.inference_providers.gmicloud.api_key.clone(),
|
||||
)))
|
||||
}
|
||||
RouterProvider::GrokApi => {
|
||||
Ok(RouterClient::GrokApi(RouterGrokApiClient::new(
|
||||
server_state.inference_providers.grok_api.api_key.clone(),
|
||||
)))
|
||||
}
|
||||
other => {
|
||||
Err(CommonWebError::server_error_with_message(
|
||||
&format!("Unsupported provider for video generation: {:?}", other),
|
||||
))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn kinovi_provider(server_state: &ServerState, kinovi_account: KinoviAccount) -> Result<RouterClient, CommonWebError> {
|
||||
let seedance2pro = &server_state.inference_providers.seedance2pro;
|
||||
|
||||
let cookies = match kinovi_account {
|
||||
KinoviAccount::Volcengine => seedance2pro.cookies_volcengine.clone(),
|
||||
KinoviAccount::BytePlus => seedance2pro.cookies_byteplus.clone(),
|
||||
KinoviAccount::BytePlusUltra => seedance2pro.cookies_byteplus_ultra.clone(),
|
||||
};
|
||||
|
||||
let session = Seedance2ProSession::from_cookies_string(cookies);
|
||||
|
||||
Ok(RouterClient::Seedance2Pro(RouterSeedance2ProClient::new(session)))
|
||||
}
|
||||
-97
@@ -1,97 +0,0 @@
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_video_cost_and_generate_request::OmniGenVideoCostAndGenerateRequest;
|
||||
use artcraft_router::api::audio_list_ref::AudioListRef;
|
||||
use artcraft_router::api::character_list_ref::CharacterListRef;
|
||||
use artcraft_router::api::router_aspect_ratio::RouterAspectRatio;
|
||||
use artcraft_router::api::router_resolution::RouterResolution;
|
||||
use artcraft_router::api::router_video_model::RouterVideoModel;
|
||||
use artcraft_router::api::image_list_ref::ImageListRef;
|
||||
use artcraft_router::api::image_ref::ImageRef;
|
||||
use artcraft_router::api::router_provider::RouterProvider;
|
||||
use artcraft_router::api::video_list_ref::VideoListRef;
|
||||
use artcraft_router::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
|
||||
use artcraft_router::generate::generate_video::generate_video_request_builder::GenerateVideoRequestBuilder;
|
||||
use enums::common::generation::common_aspect_ratio::CommonAspectRatio as CommonAspectRatioEnum;
|
||||
use enums::common::generation::common_resolution::CommonResolution as CommonResolutionEnum;
|
||||
use enums::common::generation::common_video_model::CommonVideoModel as CommonVideoModelEnum;
|
||||
|
||||
pub fn hydrate_to_router_request(
|
||||
request: &OmniGenVideoCostAndGenerateRequest,
|
||||
) -> Result<GenerateVideoRequestBuilder, CommonWebError> {
|
||||
let api_model = request.model
|
||||
.as_ref()
|
||||
.ok_or_else(|| CommonWebError::BadInputWithSimpleMessage(
|
||||
"model is required".to_string(),
|
||||
))?;
|
||||
|
||||
let model = convert_model(api_model)?;
|
||||
|
||||
let aspect_ratio = request.aspect_ratio
|
||||
.as_ref()
|
||||
.map(convert_aspect_ratio)
|
||||
.transpose()?;
|
||||
|
||||
let resolution = request.resolution
|
||||
.as_ref()
|
||||
.map(convert_resolution)
|
||||
.transpose()?;
|
||||
|
||||
Ok(GenerateVideoRequestBuilder {
|
||||
model,
|
||||
provider: RouterProvider::Artcraft,
|
||||
prompt: request.prompt.clone(),
|
||||
negative_prompt: request.negative_prompt.clone(),
|
||||
start_frame: request.start_frame_image_media_token.clone()
|
||||
.map(ImageRef::MediaFileToken),
|
||||
end_frame: request.end_frame_image_media_token.clone()
|
||||
.map(ImageRef::MediaFileToken),
|
||||
reference_images: request.reference_image_media_tokens.clone()
|
||||
.map(ImageListRef::MediaFileTokens),
|
||||
reference_videos: request.reference_video_media_tokens.clone()
|
||||
.map(VideoListRef::MediaFileTokens),
|
||||
reference_audio: request.reference_audio_media_tokens.clone()
|
||||
.map(AudioListRef::MediaFileTokens),
|
||||
reference_character_tokens: request.reference_character_tokens.clone()
|
||||
.map(CharacterListRef::CharacterTokens),
|
||||
resolution,
|
||||
aspect_ratio,
|
||||
duration_seconds: request.duration_seconds,
|
||||
video_batch_count: request.video_batch_count,
|
||||
generate_audio: request.generate_audio,
|
||||
request_mismatch_mitigation_strategy: RequestMismatchMitigationStrategy::PayMoreUpgrade,
|
||||
idempotency_token: request.idempotency_token.clone(),
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_model(
|
||||
model: &CommonVideoModelEnum,
|
||||
) -> Result<RouterVideoModel, CommonWebError> {
|
||||
let json = serde_json::to_string(model)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported video model: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_aspect_ratio(
|
||||
ar: &CommonAspectRatioEnum,
|
||||
) -> Result<RouterAspectRatio, CommonWebError> {
|
||||
let json = serde_json::to_string(ar)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported aspect ratio: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
|
||||
fn convert_resolution(
|
||||
res: &CommonResolutionEnum,
|
||||
) -> Result<RouterResolution, CommonWebError> {
|
||||
let json = serde_json::to_string(res)?;
|
||||
serde_json::from_str(&json).map_err(|e| {
|
||||
CommonWebError::BadInputWithSimpleMessage(
|
||||
format!("Unsupported resolution: {}", e),
|
||||
)
|
||||
})
|
||||
}
|
||||
-6
@@ -1,6 +0,0 @@
|
||||
pub mod bill_wallet;
|
||||
pub mod build_router_client;
|
||||
pub(crate) mod hydrate_router_request;
|
||||
pub mod pipeline_result;
|
||||
pub mod resolve_kinovi_character_ids;
|
||||
pub mod resolve_media_tokens_to_urls;
|
||||
-14
@@ -1,14 +0,0 @@
|
||||
use artcraft_router::generate::generate_video::generate_video_response::GenerateVideoResponse;
|
||||
use mysql_queries::queries::generic_inference::common::job_cost_estimates::JobCostEstimates;
|
||||
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::bill_wallet::BillWalletResult;
|
||||
|
||||
/// Both pipelines produce this, then the shared suffix handles DB writes.
|
||||
pub struct PipelineResult {
|
||||
pub billing: BillWalletResult,
|
||||
pub response: GenerateVideoResponse,
|
||||
|
||||
/// System (user-facing) and provider-side cost estimates, written onto the
|
||||
/// generic_inference_jobs rows.
|
||||
pub cost_estimates: JobCostEstimates,
|
||||
}
|
||||
-51
@@ -1,51 +0,0 @@
|
||||
//! Resolve character tokens to Kinovi character IDs for prompting.
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use log::warn;
|
||||
use sqlx::MySql;
|
||||
|
||||
use mysql_queries::queries::characters::batch_lookup_characters_by_token_for_prompting::batch_lookup_characters_by_token_for_prompting;
|
||||
use tokens::tokens::characters::CharacterToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
/// Resolve character tokens to a map of CharacterToken → Kinovi character ID.
|
||||
///
|
||||
/// Looks up the characters, filters to active ones with kinovi IDs, and warns about
|
||||
/// any that are missing or inactive (but doesn't fail the request).
|
||||
pub async fn resolve_kinovi_character_ids(
|
||||
maybe_tokens: Option<&[CharacterToken]>,
|
||||
connection: &mut sqlx::pool::PoolConnection<MySql>,
|
||||
) -> Result<Option<HashMap<CharacterToken, String>>, CommonWebError> {
|
||||
let tokens = match maybe_tokens {
|
||||
None => return Ok(None),
|
||||
Some(tokens) if tokens.is_empty() => return Ok(None),
|
||||
Some(tokens) => tokens,
|
||||
};
|
||||
|
||||
let characters = batch_lookup_characters_by_token_for_prompting(tokens, connection)
|
||||
.await?;
|
||||
|
||||
if characters.len() != tokens.len() {
|
||||
warn!(
|
||||
"Not all character tokens were found: requested {}, found {}",
|
||||
tokens.len(), characters.len(),
|
||||
);
|
||||
}
|
||||
|
||||
for character in &characters {
|
||||
if !character.is_active {
|
||||
warn!("Character {} is not yet active, skipping", character.token);
|
||||
}
|
||||
}
|
||||
|
||||
let map: HashMap<CharacterToken, String> = characters.iter()
|
||||
.filter(|c| c.is_active)
|
||||
.filter_map(|c| {
|
||||
c.kinovi_character_id.as_ref().map(|kid| (c.token.clone(), kid.clone()))
|
||||
})
|
||||
.collect();
|
||||
|
||||
if map.is_empty() { Ok(None) } else { Ok(Some(map)) }
|
||||
}
|
||||
-64
@@ -1,64 +0,0 @@
|
||||
use std::collections::HashMap;
|
||||
|
||||
use artcraft_router::api::audio_list_ref::AudioListRef;
|
||||
use artcraft_router::api::image_list_ref::ImageListRef;
|
||||
use artcraft_router::api::image_ref::ImageRef;
|
||||
use artcraft_router::api::video_list_ref::VideoListRef;
|
||||
use artcraft_router::generate::generate_video::generate_video_request_builder::GenerateVideoRequestBuilder;
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
|
||||
/// For providers that take URLs directly (GmiCloud, Fal), swap
|
||||
/// `ImageRef::MediaFileToken` → `ImageRef::Url` using the resolved map.
|
||||
pub fn resolve_media_tokens_to_urls(
|
||||
builder: &mut GenerateVideoRequestBuilder,
|
||||
url_map: Option<&HashMap<MediaFileToken, String>>,
|
||||
) {
|
||||
let map = match url_map {
|
||||
Some(m) => m,
|
||||
None => return,
|
||||
};
|
||||
|
||||
// start_frame
|
||||
if let Some(ImageRef::MediaFileToken(ref token)) = builder.start_frame {
|
||||
if let Some(url) = map.get(token) {
|
||||
builder.start_frame = Some(ImageRef::Url(url.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
// end_frame
|
||||
if let Some(ImageRef::MediaFileToken(ref token)) = builder.end_frame {
|
||||
if let Some(url) = map.get(token) {
|
||||
builder.end_frame = Some(ImageRef::Url(url.clone()));
|
||||
}
|
||||
}
|
||||
|
||||
// reference_images
|
||||
if let Some(ImageListRef::MediaFileTokens(ref tokens)) = builder.reference_images {
|
||||
let urls: Vec<String> = tokens.iter()
|
||||
.filter_map(|t| map.get(t).cloned())
|
||||
.collect();
|
||||
if !urls.is_empty() {
|
||||
builder.reference_images = Some(ImageListRef::Urls(urls));
|
||||
}
|
||||
}
|
||||
|
||||
// reference_videos
|
||||
if let Some(VideoListRef::MediaFileTokens(ref tokens)) = builder.reference_videos {
|
||||
let urls: Vec<String> = tokens.iter()
|
||||
.filter_map(|t| map.get(t).cloned())
|
||||
.collect();
|
||||
if !urls.is_empty() {
|
||||
builder.reference_videos = Some(VideoListRef::Urls(urls));
|
||||
}
|
||||
}
|
||||
|
||||
// reference_audio
|
||||
if let Some(AudioListRef::MediaFileTokens(ref tokens)) = builder.reference_audio {
|
||||
let urls: Vec<String> = tokens.iter()
|
||||
.filter_map(|t| map.get(t).cloned())
|
||||
.collect();
|
||||
if !urls.is_empty() {
|
||||
builder.reference_audio = Some(AudioListRef::Urls(urls));
|
||||
}
|
||||
}
|
||||
}
|
||||
-57
@@ -1,57 +0,0 @@
|
||||
use log::warn;
|
||||
|
||||
use enums::common::visibility::Visibility;
|
||||
use mysql_queries::queries::generic_inference::api_providers::fal::insert_generic_inference_job_for_fal_queue::FalCategory;
|
||||
use mysql_queries::queries::generic_inference::api_providers::fal::insert_generic_inference_job_for_fal_queue_with_apriori_job_token::{
|
||||
insert_generic_inference_job_for_fal_queue_with_apriori_job_token,
|
||||
InsertGenericInferenceForFalWithAprioriJobTokenArgs,
|
||||
};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertFalJobArgs<'a, 'tx> {
|
||||
pub external_job_id: &'a str,
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
pub async fn insert_fal_job(args: InsertFalJobArgs<'_, '_>) -> Result<InferenceJobToken, CommonWebError> {
|
||||
let InsertFalJobArgs {
|
||||
external_job_id,
|
||||
shared,
|
||||
} = args;
|
||||
|
||||
let db_result = insert_generic_inference_job_for_fal_queue_with_apriori_job_token(
|
||||
InsertGenericInferenceForFalWithAprioriJobTokenArgs {
|
||||
apriori_job_token: shared.apriori_job_token,
|
||||
uuid_idempotency_token: shared.idempotency_token,
|
||||
maybe_external_third_party_id: external_job_id,
|
||||
fal_category: FalCategory::VideoGeneration,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
creator_set_visibility: Visibility::Public,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: shared.maybe_cost_estimates,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
maybe_inference_args: None,
|
||||
starting_job_status_override: None,
|
||||
maybe_frontend_failure_category: None,
|
||||
maybe_failure_reason: None,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => Ok(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting fal inference job: {:?}", err);
|
||||
Err(CommonWebError::from_error(err))
|
||||
}
|
||||
}
|
||||
}
|
||||
-48
@@ -1,48 +0,0 @@
|
||||
use log::warn;
|
||||
|
||||
use mysql_queries::queries::generic_inference::api_providers::gmicloud::insert_generic_inference_job_for_gmicloud::{
|
||||
insert_generic_inference_job_for_gmicloud, InsertGmiCloudInferenceJobArgs,
|
||||
};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertGmiCloudJobArgs<'a, 'tx> {
|
||||
pub external_request_id: &'a str,
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
pub async fn insert_gmicloud_job(args: InsertGmiCloudJobArgs<'_, '_>) -> Result<InferenceJobToken, CommonWebError> {
|
||||
let InsertGmiCloudJobArgs {
|
||||
external_request_id,
|
||||
shared,
|
||||
} = args;
|
||||
|
||||
let db_result = insert_generic_inference_job_for_gmicloud(
|
||||
InsertGmiCloudInferenceJobArgs {
|
||||
apriori_job_token: shared.apriori_job_token,
|
||||
uuid_idempotency_token: shared.idempotency_token,
|
||||
external_request_id,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: shared.maybe_cost_estimates,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => Ok(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting GmiCloud inference job: {:?}", err);
|
||||
Err(CommonWebError::from_error(err))
|
||||
}
|
||||
}
|
||||
}
|
||||
-48
@@ -1,48 +0,0 @@
|
||||
use log::warn;
|
||||
|
||||
use mysql_queries::queries::generic_inference::api_providers::grok_api::insert_generic_inference_job_for_grok_api::{
|
||||
insert_generic_inference_job_for_grok_api, InsertGrokApiInferenceJobArgs,
|
||||
};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertGrokApiJobArgs<'a, 'tx> {
|
||||
pub external_request_id: &'a str,
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
pub async fn insert_grok_api_job(args: InsertGrokApiJobArgs<'_, '_>) -> Result<InferenceJobToken, CommonWebError> {
|
||||
let InsertGrokApiJobArgs {
|
||||
external_request_id,
|
||||
shared,
|
||||
} = args;
|
||||
|
||||
let db_result = insert_generic_inference_job_for_grok_api(
|
||||
InsertGrokApiInferenceJobArgs {
|
||||
apriori_job_token: shared.apriori_job_token,
|
||||
uuid_idempotency_token: shared.idempotency_token,
|
||||
external_request_id,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: shared.maybe_cost_estimates,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => Ok(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting Grok (xAI) API inference job: {:?}", err);
|
||||
Err(CommonWebError::from_error(err))
|
||||
}
|
||||
}
|
||||
}
|
||||
-95
@@ -1,95 +0,0 @@
|
||||
use log::{error, warn};
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use enums::common::visibility::Visibility;
|
||||
use mysql_queries::queries::generic_inference::api_providers::seedance2pro::insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token::{insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token, InsertGenericInferenceForSeedance2ProWithAprioriJobTokenArgs, KinoviVersion};
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::wallet_ledger_entries::WalletLedgerEntryToken;
|
||||
use uuid_utils::uuid::generate_random_uuid;
|
||||
|
||||
use super::shared_job_args::SharedJobArgs;
|
||||
|
||||
pub struct InsertSeedance2proJobsArgs<'a, 'tx> {
|
||||
pub primary_order_id: &'a str,
|
||||
pub maybe_additional_order_ids: Option<&'a [String]>,
|
||||
pub maybe_wallet_ledger_entry_token: Option<&'a WalletLedgerEntryToken>,
|
||||
pub kinovi_version: KinoviVersion,
|
||||
pub shared: SharedJobArgs<'a, 'tx>,
|
||||
}
|
||||
|
||||
pub struct InsertSeedance2proJobsResult {
|
||||
pub primary_job_token: InferenceJobToken,
|
||||
pub all_job_tokens: Vec<InferenceJobToken>,
|
||||
}
|
||||
|
||||
pub async fn insert_seedance2pro_jobs(args: InsertSeedance2proJobsArgs<'_, '_>) -> Result<InsertSeedance2proJobsResult, CommonWebError> {
|
||||
let InsertSeedance2proJobsArgs {
|
||||
primary_order_id,
|
||||
maybe_additional_order_ids,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
kinovi_version,
|
||||
mut shared,
|
||||
} = args;
|
||||
|
||||
// Build a deduplicated list with primary_order_id first.
|
||||
let mut all_order_ids = vec![primary_order_id.to_string()];
|
||||
|
||||
if let Some(additional) = maybe_additional_order_ids {
|
||||
for id in additional {
|
||||
if id != primary_order_id {
|
||||
all_order_ids.push(id.clone());
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let mut all_job_tokens: Vec<InferenceJobToken> = Vec::with_capacity(all_order_ids.len());
|
||||
|
||||
for (i, order_id) in all_order_ids.iter().enumerate() {
|
||||
let job_token = if i == 0 { shared.apriori_job_token.clone() } else { InferenceJobToken::generate() };
|
||||
|
||||
// TODO: These strings are too long for the database:
|
||||
//let idempotency_str = if i == 0 { shared.idempotency_token.to_string() } else { format!("{}-batch-{}", shared.idempotency_token, i) };
|
||||
|
||||
let idempotency_str = generate_random_uuid();
|
||||
|
||||
let db_result = insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token(
|
||||
InsertGenericInferenceForSeedance2ProWithAprioriJobTokenArgs {
|
||||
kinovi_version,
|
||||
apriori_job_token: &job_token,
|
||||
uuid_idempotency_token: &idempotency_str,
|
||||
maybe_external_third_party_id: order_id,
|
||||
maybe_model_type: shared.maybe_model_type,
|
||||
maybe_prompt_token: shared.maybe_prompt_token,
|
||||
maybe_wallet_ledger_entry_token,
|
||||
maybe_creator_user_token: Some(shared.user_token),
|
||||
maybe_avt_token: shared.maybe_avt_token,
|
||||
creator_ip_address: shared.ip_address,
|
||||
creator_set_visibility: Visibility::Public,
|
||||
maybe_platform_type: shared.maybe_platform_type,
|
||||
maybe_cost_estimates: shared.maybe_cost_estimates,
|
||||
maybe_debug_log_event_token: shared.maybe_debug_log_event_token,
|
||||
mysql_executor: &mut **shared.transaction,
|
||||
maybe_inference_args: None,
|
||||
phantom: Default::default(),
|
||||
}
|
||||
).await;
|
||||
|
||||
match db_result {
|
||||
Ok(token) => all_job_tokens.push(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting seedance2pro inference job (order_id={}): {:?}", order_id, err);
|
||||
if i == 0 { return Err(CommonWebError::from_error(err)); }
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let primary_job_token = all_job_tokens.first().cloned().ok_or_else(|| {
|
||||
error!("No inference job token was created");
|
||||
CommonWebError::server_error_with_message("No inference job token was created")
|
||||
})?;
|
||||
|
||||
Ok(InsertSeedance2proJobsResult {
|
||||
primary_job_token,
|
||||
all_job_tokens,
|
||||
})
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
pub mod insert_fal_job;
|
||||
pub mod insert_gmicloud_job;
|
||||
pub mod insert_grok_api_job;
|
||||
pub mod insert_seedance2pro_jobs;
|
||||
pub mod shared_job_args;
|
||||
-22
@@ -1,22 +0,0 @@
|
||||
use enums::common::generation::common_model_type::CommonModelType;
|
||||
use enums::common::platform_type::PlatformType;
|
||||
use mysql_queries::queries::generic_inference::common::job_cost_estimates::JobCostEstimates;
|
||||
use tokens::tokens::anonymous_visitor_tracking::AnonymousVisitorTrackingToken;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
|
||||
use tokens::tokens::prompts::PromptToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
pub struct SharedJobArgs<'a, 'tx> {
|
||||
pub apriori_job_token: &'a InferenceJobToken,
|
||||
pub idempotency_token: &'a str,
|
||||
pub user_token: &'a UserToken,
|
||||
pub maybe_avt_token: Option<&'a AnonymousVisitorTrackingToken>,
|
||||
pub maybe_prompt_token: Option<&'a PromptToken>,
|
||||
pub maybe_debug_log_event_token: Option<&'a DebugLogEventToken>,
|
||||
pub maybe_model_type: Option<CommonModelType>,
|
||||
pub maybe_platform_type: Option<PlatformType>,
|
||||
pub maybe_cost_estimates: Option<JobCostEstimates>,
|
||||
pub ip_address: &'a str,
|
||||
pub transaction: &'a mut sqlx::Transaction<'tx, sqlx::MySql>,
|
||||
}
|
||||
-7
@@ -1,7 +0,0 @@
|
||||
|
||||
#[derive(Debug, Clone, Copy)]
|
||||
pub enum KinoviAccount {
|
||||
Volcengine,
|
||||
BytePlus,
|
||||
BytePlusUltra,
|
||||
}
|
||||
-5
@@ -1,5 +0,0 @@
|
||||
pub mod omni_api_video_generate_handler;
|
||||
pub mod helpers;
|
||||
pub mod pipeline_v2;
|
||||
pub mod insert_db_job;
|
||||
pub mod kinovi_account;
|
||||
-474
@@ -1,474 +0,0 @@
|
||||
use std::collections::HashMap;
|
||||
use std::sync::Arc;
|
||||
|
||||
use actix_web::web::Json;
|
||||
use actix_web::{web, HttpRequest};
|
||||
use log::{error, info, warn};
|
||||
use sqlx::Acquire;
|
||||
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_video_cost_and_generate_request::OmniGenVideoCostAndGenerateRequest;
|
||||
use artcraft_api_defs::omni_gen::generate_response::omni_gen_video_generate_response::OmniGenVideoGenerateResponse;
|
||||
use artcraft_router::generate::generate_video::generate_video_response::GenerateVideoResponse;
|
||||
use enums::by_table::debug_logs::debug_log_type::DebugLogType;
|
||||
use enums::by_table::prompt_context_items::prompt_context_semantic_type::PromptContextSemanticType;
|
||||
use enums::by_table::prompts::prompt_type::PromptType;
|
||||
use enums::common::generation::common_generation_mode::CommonGenerationMode;
|
||||
use enums::common::generation::common_model_type::CommonModelType;
|
||||
use enums::common::generation::common_video_model::CommonVideoModel;
|
||||
use enums::common::generation_provider::GenerationProvider;
|
||||
use enums::common::platform_type::PlatformType;
|
||||
use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use mysql_queries::queries::debug_logs::insert_debug_log::{insert_debug_log, InsertDebugLogArgs};
|
||||
use mysql_queries::queries::generic_inference::api_providers::seedance2pro::insert_generic_inference_job_for_seedance2pro_queue_with_apriori_job_token::KinoviVersion;
|
||||
use mysql_queries::queries::idepotency_tokens::insert_idempotency_token::insert_idempotency_token;
|
||||
use mysql_queries::queries::prompt_context_items::insert_batch_prompt_context_items::{
|
||||
insert_batch_prompt_context_items, InsertBatchArgs, PromptContextItem,
|
||||
};
|
||||
use mysql_queries::queries::prompts::insert_prompt::{insert_prompt, InsertPromptArgs};
|
||||
use tokens::tokens::characters::CharacterToken;
|
||||
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
use tokens::tokens::non_unique::debug_logs_event_token::DebugLogEventToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::generate::common::payments_error_test::payments_error_test;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::hydrate_router_request::hydrate_to_router_request;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::resolve_kinovi_character_ids::resolve_kinovi_character_ids;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_fal_job::{insert_fal_job, InsertFalJobArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_gmicloud_job::{insert_gmicloud_job, InsertGmiCloudJobArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_grok_api_job::{insert_grok_api_job, InsertGrokApiJobArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::insert_seedance2pro_jobs::{insert_seedance2pro_jobs, InsertSeedance2proJobsArgs};
|
||||
use crate::http_server::endpoints::omni_api::generate::video::insert_db_job::shared_job_args::SharedJobArgs;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::pipeline_v2::run_pipeline_v2::{run_pipeline_v2, RunPipelineV2Args};
|
||||
use crate::http_server::endpoints::omni_api::shared_utils::video::validate_video_request::validate_video_request;
|
||||
use crate::http_server::session::lookup::user_session_feature_flags::UserSessionFeatureFlags;
|
||||
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
|
||||
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
|
||||
use crate::state::server_state::ServerState;
|
||||
use crate::util::lookup::lookup_media_files_as_cdn_url_list_and_map::lookup_media_files_as_cdn_url_list_and_map;
|
||||
|
||||
/// Generate a video using the omni-api unified endpoint (API-key authenticated).
|
||||
#[utoipa::path(
|
||||
post,
|
||||
tag = "Omni API",
|
||||
path = "/v1/omni_api/generate/video",
|
||||
request_body = OmniGenVideoCostAndGenerateRequest,
|
||||
responses(
|
||||
(status = 200, description = "Success", body = OmniGenVideoGenerateResponse),
|
||||
(status = 400, description = "Bad input"),
|
||||
(status = 401, description = "Unauthorized"),
|
||||
(status = 402, description = "Payment required"),
|
||||
(status = 500, description = "Server error"),
|
||||
),
|
||||
)]
|
||||
pub async fn omni_api_video_generate_handler(
|
||||
http_request: HttpRequest,
|
||||
request: Json<OmniGenVideoCostAndGenerateRequest>,
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
) -> Result<Json<OmniGenVideoGenerateResponse>, CommonWebError> {
|
||||
|
||||
info!("request: {:?}", request);
|
||||
|
||||
// Reject doomed combos (e.g. grok_imagine_video_1p5 without an image)
|
||||
// before any billable or DB-mutating work — see helper for the rules.
|
||||
validate_video_request(&request)?;
|
||||
|
||||
payments_error_test(&request.prompt.as_deref().unwrap_or(""))?;
|
||||
|
||||
let debug_log_event_token = DebugLogEventToken::generate();
|
||||
|
||||
let maybe_prompt_model_type: Option<CommonModelType> = request.model
|
||||
.as_ref()
|
||||
.map(|m| m.to_common_model_type());
|
||||
|
||||
// ==================== API KEY USER ==================== //
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
|
||||
|
||||
// API-key authentication (Authorization header) instead of a session cookie. Never cached.
|
||||
let api_session = require_api_key_user(&http_request, &mut *mysql_connection).await?;
|
||||
|
||||
let user_token = &api_session.user_token;
|
||||
|
||||
// ==================== MODEL ACCESS CHECK ==================== //
|
||||
|
||||
// API-key users have no per-user feature flags today.
|
||||
let user_feature_flags = UserSessionFeatureFlags::new(None);
|
||||
|
||||
// ==================== IDEMPOTENCY ==================== //
|
||||
|
||||
let idempotency_token = request.idempotency_token.as_deref()
|
||||
.unwrap_or("")
|
||||
.to_string();
|
||||
|
||||
if let Err(reason) = validate_idempotency_token_format(&idempotency_token) {
|
||||
return Err(CommonWebError::BadInputWithSimpleMessage(reason));
|
||||
}
|
||||
|
||||
insert_idempotency_token(&idempotency_token, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error inserting idempotency token: {:?}", err);
|
||||
CommonWebError::BadInputWithSimpleMessage("repeated idempotency token".to_string())
|
||||
})?;
|
||||
|
||||
// ==================== RESOLVE MEDIA TOKENS ==================== //
|
||||
|
||||
let mut all_tokens: Vec<MediaFileToken> = Vec::new();
|
||||
|
||||
if let Some(token) = &request.start_frame_image_media_token {
|
||||
all_tokens.push(token.clone());
|
||||
}
|
||||
if let Some(token) = &request.end_frame_image_media_token {
|
||||
all_tokens.push(token.clone());
|
||||
}
|
||||
if let Some(tokens) = &request.reference_image_media_tokens {
|
||||
all_tokens.extend(tokens.iter().cloned());
|
||||
}
|
||||
if let Some(tokens) = &request.reference_video_media_tokens {
|
||||
all_tokens.extend(tokens.iter().cloned());
|
||||
}
|
||||
if let Some(tokens) = &request.reference_audio_media_tokens {
|
||||
all_tokens.extend(tokens.iter().cloned());
|
||||
}
|
||||
|
||||
let media_file_to_url_map: Option<HashMap<MediaFileToken, String>> = if all_tokens.is_empty() {
|
||||
None
|
||||
} else {
|
||||
info!("Resolving {} media file tokens to CDN URLs", all_tokens.len());
|
||||
let resolved = lookup_media_files_as_cdn_url_list_and_map(
|
||||
&http_request,
|
||||
&mut mysql_connection,
|
||||
server_state.server_environment,
|
||||
&all_tokens,
|
||||
).await?;
|
||||
Some(resolved.token_to_url_map)
|
||||
};
|
||||
|
||||
// ==================== RESOLVE CHARACTERS ==================== //
|
||||
|
||||
let kinovi_character_id_map: Option<HashMap<CharacterToken, String>> =
|
||||
resolve_kinovi_character_ids(
|
||||
request.reference_character_tokens.as_deref(),
|
||||
&mut mysql_connection,
|
||||
).await?;
|
||||
|
||||
// ==================== HYDRATE ROUTER REQUEST ==================== //
|
||||
|
||||
let router_builder = hydrate_to_router_request(&request)?;
|
||||
|
||||
// ==================== PIPELINE DISPATCH ==================== //
|
||||
|
||||
let kinovi_account = match request.model {
|
||||
// BytePlus Ultra
|
||||
Some(CommonVideoModel::Seedance2p0BytePlusUltra) => KinoviAccount::BytePlusUltra,
|
||||
Some(CommonVideoModel::Seedance2p0BytePlusUltraFast) => KinoviAccount::BytePlusUltra,
|
||||
// BytePlus
|
||||
Some(CommonVideoModel::Seedance2p0BytePlus) => KinoviAccount::BytePlus,
|
||||
Some(CommonVideoModel::Seedance2p0BytePlusFast) => KinoviAccount::BytePlus,
|
||||
Some(CommonVideoModel::PreviewModel) => KinoviAccount::BytePlus,
|
||||
Some(CommonVideoModel::PreviewModelFast) => KinoviAccount::BytePlus,
|
||||
// Everything else goes through Volcengine
|
||||
_ => KinoviAccount::Volcengine,
|
||||
};
|
||||
|
||||
// ==================== DEBUG LOG: HTTP REQUEST ==================== //
|
||||
|
||||
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
|
||||
apriori_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
debug_log_type: DebugLogType::HttpRequest,
|
||||
message: &serde_json::to_string(&*request).unwrap_or_default(),
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
warn!("Failed to insert HTTP request debug log: {:?}", err);
|
||||
}
|
||||
|
||||
// NB: Release the pooled DB connection before the (slow, external) generation call so we don't
|
||||
// hold a pool slot idle while waiting on the provider — that's what starves the pool and causes
|
||||
// PoolTimedOut on unrelated endpoints. We re-acquire below to write the result.
|
||||
drop(mysql_connection);
|
||||
|
||||
let pipeline_result = run_pipeline_v2(RunPipelineV2Args {
|
||||
router_builder: &router_builder,
|
||||
server_state: &server_state,
|
||||
user_token,
|
||||
media_file_to_url_map: &media_file_to_url_map,
|
||||
kinovi_character_id_map: &kinovi_character_id_map,
|
||||
kinovi_account,
|
||||
}).await?;
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool.acquire().await?;
|
||||
|
||||
// ==================== DEBUG LOG: FAL REQUEST ==================== //
|
||||
|
||||
if let GenerateVideoResponse::Fal(ref fal_payload) = pipeline_result.response {
|
||||
if let Some(ref outbound_request) = fal_payload.maybe_outbound_request {
|
||||
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
|
||||
apriori_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
debug_log_type: DebugLogType::FalRequest,
|
||||
message: &format!("{:#?}", outbound_request),
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
warn!("Failed to insert Fal request debug log: {:?}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== DEBUG LOG: GROK API REQUEST ==================== //
|
||||
|
||||
if let GenerateVideoResponse::Grok(ref grok_payload) = pipeline_result.response {
|
||||
if let Some(ref outbound_request) = grok_payload.maybe_outbound_request {
|
||||
if let Err(err) = insert_debug_log(InsertDebugLogArgs {
|
||||
apriori_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
debug_log_type: DebugLogType::GrokApiRequest,
|
||||
message: &format!("{:#?}", outbound_request),
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
warn!("Failed to insert Grok API request debug log: {:?}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// ==================== WRITE RESULT ==================== //
|
||||
|
||||
let ip_address = get_request_ip(&http_request);
|
||||
// Omni API requests are always API-key authenticated; hardcode the platform type.
|
||||
let maybe_platform_type = Some(PlatformType::ApiKey);
|
||||
|
||||
let mut transaction = mysql_connection.begin().await.map_err(|err| {
|
||||
error!("Error starting MySQL transaction: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
|
||||
// -- Prompt --
|
||||
|
||||
let prompt_token = match insert_prompt(InsertPromptArgs {
|
||||
maybe_apriori_prompt_token: None,
|
||||
prompt_type: PromptType::ArtcraftApp,
|
||||
maybe_creator_user_token: Some(user_token),
|
||||
maybe_model_type: maybe_prompt_model_type,
|
||||
maybe_generation_provider: Some(GenerationProvider::Artcraft),
|
||||
maybe_positive_prompt: request.prompt.as_deref(),
|
||||
maybe_negative_prompt: request.negative_prompt.as_deref(),
|
||||
maybe_other_args: None,
|
||||
maybe_generation_mode: Some(determine_generation_mode(&request)),
|
||||
maybe_aspect_ratio: request.aspect_ratio,
|
||||
maybe_resolution: request.resolution,
|
||||
maybe_batch_count: request.video_batch_count.map(|c| c as u8),
|
||||
maybe_generate_audio: request.generate_audio,
|
||||
maybe_duration_seconds: request.duration_seconds.map(|d| d as u32),
|
||||
creator_ip_address: &ip_address,
|
||||
mysql_executor: &mut *transaction,
|
||||
phantom: Default::default(),
|
||||
}).await {
|
||||
Ok(token) => Some(token),
|
||||
Err(err) => {
|
||||
warn!("Error inserting prompt: {:?}", err);
|
||||
None
|
||||
}
|
||||
};
|
||||
|
||||
// -- Prompt context items --
|
||||
|
||||
if let Some(token) = prompt_token.as_ref() {
|
||||
let mut context_items = Vec::new();
|
||||
|
||||
if let Some(media_token) = &request.start_frame_image_media_token {
|
||||
context_items.push(PromptContextItem {
|
||||
media_token: media_token.clone(),
|
||||
context_semantic_type: PromptContextSemanticType::VidStartFrame,
|
||||
});
|
||||
}
|
||||
if let Some(media_token) = &request.end_frame_image_media_token {
|
||||
context_items.push(PromptContextItem {
|
||||
media_token: media_token.clone(),
|
||||
context_semantic_type: PromptContextSemanticType::VidEndFrame,
|
||||
});
|
||||
}
|
||||
if let Some(ref_tokens) = &request.reference_image_media_tokens {
|
||||
for media_token in ref_tokens {
|
||||
context_items.push(PromptContextItem {
|
||||
media_token: media_token.clone(),
|
||||
context_semantic_type: PromptContextSemanticType::Imgref,
|
||||
});
|
||||
}
|
||||
}
|
||||
if let Some(ref_tokens) = &request.reference_video_media_tokens {
|
||||
for media_token in ref_tokens {
|
||||
context_items.push(PromptContextItem {
|
||||
media_token: media_token.clone(),
|
||||
context_semantic_type: PromptContextSemanticType::VidRef,
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
if !context_items.is_empty() {
|
||||
if let Err(err) = insert_batch_prompt_context_items(InsertBatchArgs {
|
||||
prompt_token: token.clone(),
|
||||
items: context_items,
|
||||
transaction: &mut transaction,
|
||||
}).await {
|
||||
warn!("Error inserting batch prompt context items: {:?}", err);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// -- Inference job --
|
||||
|
||||
let (primary_job_token, all_job_tokens) = match &pipeline_result.response {
|
||||
GenerateVideoResponse::Seedance2Pro(payload) => {
|
||||
info!("Inserting seedance2pro job(s) with token: {:?}", pipeline_result.billing.apriori_job_token);
|
||||
|
||||
let kinovi_version = match kinovi_account {
|
||||
KinoviAccount::Volcengine => KinoviVersion::Volcengine,
|
||||
KinoviAccount::BytePlus => KinoviVersion::BytePlus,
|
||||
KinoviAccount::BytePlusUltra => KinoviVersion::BytePlusUltra,
|
||||
};
|
||||
|
||||
let result = insert_seedance2pro_jobs(InsertSeedance2proJobsArgs {
|
||||
primary_order_id: &payload.order_id,
|
||||
maybe_additional_order_ids: payload.maybe_order_ids.as_deref(),
|
||||
maybe_wallet_ledger_entry_token: pipeline_result.billing.maybe_wallet_ledger_entry_token.as_ref(),
|
||||
kinovi_version,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.billing.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
maybe_cost_estimates: Some(pipeline_result.cost_estimates),
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?;
|
||||
(result.primary_job_token, result.all_job_tokens)
|
||||
}
|
||||
GenerateVideoResponse::Fal(payload) => {
|
||||
let external_id = payload.request_id.as_deref().ok_or_else(|| {
|
||||
error!("Fal generation response missing request_id");
|
||||
CommonWebError::server_error_with_message("Fal generation response missing request_id")
|
||||
})?;
|
||||
info!("Inserting fal job with token: {:?}", pipeline_result.billing.apriori_job_token);
|
||||
let token = insert_fal_job(InsertFalJobArgs {
|
||||
external_job_id: external_id,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.billing.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
maybe_cost_estimates: Some(pipeline_result.cost_estimates),
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?;
|
||||
|
||||
(
|
||||
token.clone(),
|
||||
vec![token],
|
||||
)
|
||||
}
|
||||
GenerateVideoResponse::Artcraft(payload) => {
|
||||
(
|
||||
payload.inference_job_token.clone(),
|
||||
vec![payload.inference_job_token.clone()],
|
||||
)
|
||||
}
|
||||
GenerateVideoResponse::GmiCloud(payload) => {
|
||||
info!("Inserting GmiCloud job with token: {:?}", pipeline_result.billing.apriori_job_token);
|
||||
let token = insert_gmicloud_job(InsertGmiCloudJobArgs {
|
||||
external_request_id: &payload.request_id,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.billing.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
maybe_cost_estimates: Some(pipeline_result.cost_estimates),
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?;
|
||||
|
||||
(
|
||||
token.clone(),
|
||||
vec![token],
|
||||
)
|
||||
}
|
||||
GenerateVideoResponse::Grok(payload) => {
|
||||
info!("Inserting Grok (xAI) API job with token: {:?}", pipeline_result.billing.apriori_job_token);
|
||||
let token = insert_grok_api_job(InsertGrokApiJobArgs {
|
||||
external_request_id: &payload.request_id,
|
||||
shared: SharedJobArgs {
|
||||
apriori_job_token: &pipeline_result.billing.apriori_job_token,
|
||||
idempotency_token: &idempotency_token,
|
||||
user_token,
|
||||
maybe_avt_token: None, // AVT tokens are web-session only; API-key callers have none.
|
||||
maybe_model_type: request.model.map(|v| v.to_common_model_type()),
|
||||
maybe_prompt_token: prompt_token.as_ref(),
|
||||
maybe_debug_log_event_token: Some(&debug_log_event_token),
|
||||
maybe_platform_type,
|
||||
maybe_cost_estimates: Some(pipeline_result.cost_estimates),
|
||||
ip_address: &ip_address,
|
||||
transaction: &mut transaction,
|
||||
},
|
||||
}).await?;
|
||||
|
||||
(
|
||||
token.clone(),
|
||||
vec![token],
|
||||
)
|
||||
}
|
||||
other => {
|
||||
error!("Unexpected generation response variant: {:?}", other);
|
||||
return Err(CommonWebError::server_error_with_message("Unexpected generation response"));
|
||||
}
|
||||
};
|
||||
|
||||
transaction.commit().await.map_err(|err| {
|
||||
error!("Error committing transaction: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
|
||||
Ok(Json(OmniGenVideoGenerateResponse {
|
||||
success: true,
|
||||
inference_job_token: primary_job_token,
|
||||
all_job_tokens,
|
||||
}))
|
||||
}
|
||||
|
||||
fn determine_generation_mode(request: &OmniGenVideoCostAndGenerateRequest) -> CommonGenerationMode {
|
||||
let has_keyframe = request.start_frame_image_media_token.is_some()
|
||||
|| request.end_frame_image_media_token.is_some();
|
||||
|
||||
if has_keyframe {
|
||||
return CommonGenerationMode::Keyframe;
|
||||
}
|
||||
|
||||
let has_reference = request.reference_image_media_tokens.as_ref().is_some_and(|t| !t.is_empty())
|
||||
|| request.reference_video_media_tokens.as_ref().is_some_and(|t| !t.is_empty())
|
||||
|| request.reference_audio_media_tokens.as_ref().is_some_and(|t| !t.is_empty())
|
||||
|| request.reference_character_tokens.as_ref().is_some_and(|t| !t.is_empty());
|
||||
|
||||
if has_reference {
|
||||
return CommonGenerationMode::Reference;
|
||||
}
|
||||
|
||||
CommonGenerationMode::Text
|
||||
}
|
||||
-1
@@ -1 +0,0 @@
|
||||
pub mod run_pipeline_v2;
|
||||
-227
@@ -1,227 +0,0 @@
|
||||
use std::convert::TryFrom;
|
||||
|
||||
use std::collections::HashMap;
|
||||
|
||||
use log::{error, info, warn};
|
||||
use artcraft_router::api::router_video_model::RouterVideoModel;
|
||||
use artcraft_router::api::router_provider::RouterProvider;
|
||||
use artcraft_router::generate::generate_video::generate_video_request_builder::GenerateVideoRequestBuilder;
|
||||
use artcraft_router::generate::generate_video::generate_video_response::GenerateVideoResponse;
|
||||
use artcraft_router::generate::generate_video::video_generation_draft_context::VideoGenerationDraftContext;
|
||||
use artcraft_router::generate::generate_video::video_generation_draft_or_request::VideoGenerationDraftOrRequest;
|
||||
use tokens::tokens::characters::CharacterToken;
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoint_helpers::refund_wallet_after_api_failure::refund_wallet_after_api_failure;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::bill_wallet::bill_wallet;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::build_router_client::build_router_client;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::pipeline_result::PipelineResult;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::helpers::resolve_media_tokens_to_urls::resolve_media_tokens_to_urls;
|
||||
use mysql_queries::queries::generic_inference::common::job_cost_estimates::JobCostEstimates;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::kinovi_account::KinoviAccount;
|
||||
use crate::http_server::endpoints::omni_api::shared_utils::map_seedance2pro_router_error::map_router_error_to_web_error;
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
pub struct RunPipelineV2Args<'a> {
|
||||
pub router_builder: &'a GenerateVideoRequestBuilder,
|
||||
pub server_state: &'a ServerState,
|
||||
pub user_token: &'a UserToken,
|
||||
pub media_file_to_url_map: &'a Option<HashMap<MediaFileToken, String>>,
|
||||
pub kinovi_character_id_map: &'a Option<HashMap<CharacterToken, String>>,
|
||||
pub kinovi_account: KinoviAccount,
|
||||
}
|
||||
|
||||
// NB: This pipeline does an external generation call (`upload_and_generate`) that can take many
|
||||
// seconds. It deliberately does NOT hold a pooled DB connection across that call — it acquires
|
||||
// short-lived connections only for the billing and (on failure) refund writes. Holding a pooled
|
||||
// connection across the external call is what starves the pool and causes `PoolTimedOut`.
|
||||
pub async fn run_pipeline_v2(args: RunPipelineV2Args<'_>) -> Result<PipelineResult, CommonWebError> {
|
||||
let RunPipelineV2Args {
|
||||
router_builder,
|
||||
server_state,
|
||||
user_token,
|
||||
media_file_to_url_map,
|
||||
kinovi_character_id_map,
|
||||
kinovi_account,
|
||||
} = args;
|
||||
|
||||
let mut router_builder = router_builder.clone();
|
||||
|
||||
match router_builder.model {
|
||||
RouterVideoModel::PreviewModel |
|
||||
RouterVideoModel::Seedance2p0BytePlus |
|
||||
RouterVideoModel::Seedance2p0BytePlusUltra => {
|
||||
router_builder.model = RouterVideoModel::Seedance2p0;
|
||||
},
|
||||
RouterVideoModel::PreviewModelFast |
|
||||
RouterVideoModel::Seedance2p0BytePlusFast |
|
||||
RouterVideoModel::Seedance2p0BytePlusUltraFast => {
|
||||
router_builder.model = RouterVideoModel::Seedance2p0Fast;
|
||||
},
|
||||
_ => {}, // Fall-through
|
||||
}
|
||||
|
||||
let provider = match router_builder.model {
|
||||
RouterVideoModel::HappyHorse1p0 => RouterProvider::Seedance2Pro,
|
||||
RouterVideoModel::Seedance2p0 => RouterProvider::Seedance2Pro,
|
||||
RouterVideoModel::Seedance2p0Fast => RouterProvider::Seedance2Pro,
|
||||
//RouterVideoModel::Seedance2p0Ultra => RouterProvider::GmiCloud,
|
||||
//RouterVideoModel::Seedance2p0UltraFast => RouterProvider::GmiCloud,
|
||||
RouterVideoModel::GrokImagineVideo => RouterProvider::GrokApi,
|
||||
RouterVideoModel::GrokImagineVideo1p5 => RouterProvider::GrokApi,
|
||||
_ => RouterProvider::Fal,
|
||||
};
|
||||
|
||||
// 1. Build execution request
|
||||
let mut exec_builder = router_builder.clone();
|
||||
exec_builder.provider = provider;
|
||||
|
||||
// Fal, GmiCloud, and Grok (xAI) take image URLs directly, not media file tokens.
|
||||
// Resolve tokens to URLs before building.
|
||||
if matches!(provider, RouterProvider::Fal | RouterProvider::GmiCloud | RouterProvider::GrokApi) {
|
||||
resolve_media_tokens_to_urls(&mut exec_builder, media_file_to_url_map.as_ref());
|
||||
}
|
||||
|
||||
let draft_or_request = exec_builder.build2()
|
||||
.map_err(|e| {
|
||||
warn!("Failed to build2 for v2 pipeline: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})?;
|
||||
|
||||
// 2. Calculate cost.
|
||||
// For Artcraft-billable models, swap provider to Artcraft so credits = cents.
|
||||
// For GmiCloud, use the execution request's cost directly (no Artcraft equivalent).
|
||||
let system_cost_estimate = {
|
||||
let mut cost_builder = router_builder.clone();
|
||||
cost_builder.provider = RouterProvider::Artcraft;
|
||||
|
||||
cost_builder.build2()
|
||||
.map_err(|e| {
|
||||
warn!("Failed to build2 cost estimate for v2: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})?
|
||||
.estimate_cost()
|
||||
.map_err(|e| {
|
||||
warn!("Failed to estimate cost for v2: {}", e);
|
||||
CommonWebError::from_error(e)
|
||||
})?
|
||||
};
|
||||
|
||||
let cost = system_cost_estimate.cost_in_credits.unwrap_or(0);
|
||||
|
||||
// Provider-side estimate (what the fulfilling provider charges us). The
|
||||
// router defers to the underlying provider crates (seedance2pro_client,
|
||||
// gmicloud_client, grok_api_client, the fal pricing modules, etc.) per
|
||||
// request variant. Bookkeeping only — failures must not block generation.
|
||||
let maybe_provider_cost_estimate = match draft_or_request.estimate_cost() {
|
||||
Ok(estimate) => Some(estimate),
|
||||
Err(err) => {
|
||||
warn!("Failed to estimate provider cost for v2 video: {}", err);
|
||||
None
|
||||
}
|
||||
};
|
||||
|
||||
let cost_estimates = JobCostEstimates {
|
||||
maybe_external_third_party_cost_credits: maybe_provider_cost_estimate.as_ref()
|
||||
.and_then(|e| e.cost_in_credits)
|
||||
.and_then(|v| u32::try_from(v).ok()),
|
||||
maybe_external_third_party_cost_usd_cents: maybe_provider_cost_estimate.as_ref()
|
||||
.and_then(|e| e.cost_in_usd_cents)
|
||||
.and_then(|v| u32::try_from(v).ok()),
|
||||
maybe_system_cost_credits: system_cost_estimate.cost_in_credits
|
||||
.and_then(|v| u32::try_from(v).ok()),
|
||||
maybe_system_cost_usd_cents: system_cost_estimate.cost_in_usd_cents
|
||||
.and_then(|v| u32::try_from(v).ok()),
|
||||
};
|
||||
|
||||
info!("v2 estimated cost: {} credits (estimates: {:?})", cost, cost_estimates);
|
||||
|
||||
// 3. Bill wallet (short-lived connection — released before the external call below).
|
||||
let billing = {
|
||||
let mut billing_connection = server_state.mysql_pool.acquire().await
|
||||
.map_err(|err| {
|
||||
error!("Failed to acquire MySQL connection for billing: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
bill_wallet(user_token, cost, &mut billing_connection).await?
|
||||
};
|
||||
|
||||
// 4. Upload media (if draft) and generate video.
|
||||
// The entire block is wrapped so Kinovi failures trigger a refund.
|
||||
// NB: No pooled DB connection is held across this call.
|
||||
let result = upload_and_generate(
|
||||
draft_or_request,
|
||||
server_state,
|
||||
media_file_to_url_map.as_ref(),
|
||||
kinovi_character_id_map.as_ref(),
|
||||
kinovi_account,
|
||||
).await;
|
||||
|
||||
// 5. On failure, refund wallet for Kinovi requests.
|
||||
if let Err(ref err) = result {
|
||||
if matches!(provider, RouterProvider::Seedance2Pro) {
|
||||
if let Some(ledger_entry_token) = billing.maybe_wallet_ledger_entry_token.as_ref() {
|
||||
warn!("Kinovi v2 generation failed, issuing refund for {}: {:?}", ledger_entry_token.as_str(), err);
|
||||
|
||||
match server_state.mysql_pool.acquire().await {
|
||||
Ok(mut refund_connection) => {
|
||||
if let Err(refund_err) = refund_wallet_after_api_failure(ledger_entry_token, &mut refund_connection).await {
|
||||
error!("Failed to refund wallet after Kinovi v2 failure: {:?}", refund_err);
|
||||
}
|
||||
}
|
||||
Err(acquire_err) => {
|
||||
error!("Failed to acquire MySQL connection to refund wallet after Kinovi v2 failure: {:?}", acquire_err);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
let response = result?;
|
||||
|
||||
info!("v2 generation response: {:?}", response);
|
||||
|
||||
Ok(PipelineResult { billing, response, cost_estimates })
|
||||
}
|
||||
|
||||
/// Finalize the draft (uploading media if needed), then send the generation request.
|
||||
///
|
||||
/// This is the block that gets refunded on failure for Kinovi providers.
|
||||
async fn upload_and_generate(
|
||||
draft_or_request: VideoGenerationDraftOrRequest,
|
||||
server_state: &ServerState,
|
||||
media_file_urls_by_token: Option<&HashMap<MediaFileToken, String>>,
|
||||
kinovi_character_ids: Option<&HashMap<CharacterToken, String>>,
|
||||
kinovi_account: KinoviAccount,
|
||||
) -> Result<GenerateVideoResponse, CommonWebError> {
|
||||
|
||||
let provider = draft_or_request.get_provider();
|
||||
let client = build_router_client(provider, server_state, kinovi_account)?;
|
||||
|
||||
let video_request = match draft_or_request {
|
||||
VideoGenerationDraftOrRequest::Request(request) => request,
|
||||
VideoGenerationDraftOrRequest::Draft(draft) => {
|
||||
let draft_context = VideoGenerationDraftContext {
|
||||
client: Some(&client),
|
||||
media_file_to_artcraft_url_map: media_file_urls_by_token,
|
||||
character_token_to_kinovi_id_map: kinovi_character_ids,
|
||||
};
|
||||
|
||||
draft.finalize(draft_context)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("Failed to finalize v2 draft: {:?}", err);
|
||||
map_router_error_to_web_error(err)
|
||||
})?
|
||||
}
|
||||
};
|
||||
|
||||
video_request.send_request(&client)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("v2 video generation failed: {:?}", err);
|
||||
map_router_error_to_web_error(err)
|
||||
})
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
pub mod generate;
|
||||
pub mod shared_utils;
|
||||
pub mod upload;
|
||||
-90
@@ -1,90 +0,0 @@
|
||||
//! Maps artcraft router errors to web errors for the omni_gen pipelines.
|
||||
//!
|
||||
//! Known Kinovi (Seedance2Pro) bad requests (content violations, over-long
|
||||
//! prompts, too many urls) become 400s with a user-facing message. Everything
|
||||
//! else is unanticipated and stays a 500.
|
||||
|
||||
use artcraft_router::errors::artcraft_router_error::ArtcraftRouterError;
|
||||
use artcraft_router::errors::provider_error::ProviderError;
|
||||
use seedance2pro_client::error::seedance2pro_bad_request_api_error::Seedance2ProBadRequestApiError;
|
||||
use seedance2pro_client::error::seedance2pro_error::Seedance2ProError;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
/// Map an artcraft router error to a web error, unwrapping Kinovi bad requests.
|
||||
pub fn map_router_error_to_web_error(error: ArtcraftRouterError) -> CommonWebError {
|
||||
match error {
|
||||
ArtcraftRouterError::Provider(
|
||||
ProviderError::Seedance2Pro(Seedance2ProError::ApiBadRequest(bad_request))
|
||||
) => {
|
||||
CommonWebError::BadInputWithSimpleMessage(user_facing_bad_request_message(&bad_request))
|
||||
}
|
||||
other => CommonWebError::from_error(other),
|
||||
}
|
||||
}
|
||||
|
||||
fn user_facing_bad_request_message(error: &Seedance2ProBadRequestApiError) -> String {
|
||||
match error {
|
||||
Seedance2ProBadRequestApiError::VideoGenerationViolation { .. } => {
|
||||
"The generation request was flagged as a content violation. \
|
||||
Please adjust your prompt or input media and try again.".to_string()
|
||||
}
|
||||
Seedance2ProBadRequestApiError::PromptIsTooLong { .. } => {
|
||||
"The prompt is too long. Please shorten it below 10,000 characters and try again.".to_string()
|
||||
}
|
||||
Seedance2ProBadRequestApiError::TooManyUrls { .. } => {
|
||||
"Too many input files were attached. \
|
||||
Please remove some and try again.".to_string()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use actix_web::ResponseError;
|
||||
|
||||
use super::*;
|
||||
|
||||
#[test]
|
||||
fn kinovi_bad_requests_become_400s_with_user_facing_messages() {
|
||||
let cases = [
|
||||
(
|
||||
Seedance2ProBadRequestApiError::VideoGenerationViolation { raw_body: "{}".to_string() },
|
||||
"content violation",
|
||||
),
|
||||
(
|
||||
Seedance2ProBadRequestApiError::PromptIsTooLong { raw_body: "{}".to_string() },
|
||||
"prompt is too long",
|
||||
),
|
||||
(
|
||||
Seedance2ProBadRequestApiError::TooManyUrls { raw_body: "{}".to_string() },
|
||||
"Too many input files",
|
||||
),
|
||||
];
|
||||
|
||||
for (bad_request, expected_message_fragment) in cases {
|
||||
let error = map_router_error_to_web_error(
|
||||
ArtcraftRouterError::Provider(
|
||||
ProviderError::Seedance2Pro(
|
||||
Seedance2ProError::ApiBadRequest(bad_request))));
|
||||
|
||||
assert_eq!(error.status_code().as_u16(), 400);
|
||||
match error {
|
||||
CommonWebError::BadInputWithSimpleMessage(message) => {
|
||||
assert!(
|
||||
message.contains(expected_message_fragment),
|
||||
"message {:?} should contain {:?}", message, expected_message_fragment);
|
||||
}
|
||||
other => panic!("Expected BadInputWithSimpleMessage, got: {:?}", other),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn other_router_errors_stay_500s() {
|
||||
let error = map_router_error_to_web_error(
|
||||
ArtcraftRouterError::UnsupportedModel("some_model".to_string()));
|
||||
|
||||
assert_eq!(error.status_code().as_u16(), 500);
|
||||
}
|
||||
}
|
||||
-2
@@ -1,2 +0,0 @@
|
||||
pub mod map_seedance2pro_router_error;
|
||||
pub mod video;
|
||||
-2
@@ -1,2 +0,0 @@
|
||||
pub mod validate_video_request;
|
||||
mod validate_when_image_required;
|
||||
-11
@@ -1,11 +0,0 @@
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::omni_api::shared_utils::video::validate_when_image_required::validate_when_image_required;
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_video_cost_and_generate_request::OmniGenVideoCostAndGenerateRequest;
|
||||
|
||||
/// Validate requests before they incur user costs or send API requests
|
||||
pub fn validate_video_request(
|
||||
request: &OmniGenVideoCostAndGenerateRequest,
|
||||
) -> Result<(), CommonWebError> {
|
||||
validate_when_image_required(request)?;
|
||||
Ok(())
|
||||
}
|
||||
-157
@@ -1,157 +0,0 @@
|
||||
use artcraft_api_defs::omni_gen::cost_and_generate_requests::omni_gen_video_cost_and_generate_request::OmniGenVideoCostAndGenerateRequest;
|
||||
use enums::common::generation::common_video_model::CommonVideoModel;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
|
||||
/// xAI's `grok-imagine-video-1.5-preview` model rejects text-to-video at
|
||||
/// the upstream API. Reject the request at the handler layer so callers see
|
||||
/// a clean 400 with an actionable message and we don't charge their wallet
|
||||
/// for an inference job we know will fail.
|
||||
pub (super) fn validate_when_image_required(
|
||||
request: &OmniGenVideoCostAndGenerateRequest,
|
||||
) -> Result<(), CommonWebError> {
|
||||
match request.model {
|
||||
Some(CommonVideoModel::GrokImagineVideo1p5) => {}, // Fall-through
|
||||
_ => return Ok(()),
|
||||
}
|
||||
|
||||
let requires_image = matches!(request.model, Some(CommonVideoModel::GrokImagineVideo1p5));
|
||||
|
||||
if !requires_image {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
let has_start_frame = request.start_frame_image_media_token.is_some();
|
||||
|
||||
let has_reference_images = request
|
||||
.reference_image_media_tokens
|
||||
.as_ref()
|
||||
.is_some_and(|v| !v.is_empty());
|
||||
|
||||
if has_start_frame || has_reference_images {
|
||||
return Ok(());
|
||||
}
|
||||
|
||||
Err(CommonWebError::BadInputWithSimpleMessage(
|
||||
"Image is required".to_string(),
|
||||
))
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
|
||||
fn base_request() -> OmniGenVideoCostAndGenerateRequest {
|
||||
OmniGenVideoCostAndGenerateRequest {
|
||||
idempotency_token: Some("a1b2c3d4-e5f6-7890-abcd-ef1234567890".to_string()),
|
||||
model: None,
|
||||
prompt: Some("test".to_string()),
|
||||
negative_prompt: None,
|
||||
start_frame_image_media_token: None,
|
||||
end_frame_image_media_token: None,
|
||||
reference_image_media_tokens: None,
|
||||
reference_video_media_tokens: None,
|
||||
reference_audio_media_tokens: None,
|
||||
reference_character_tokens: None,
|
||||
resolution: None,
|
||||
aspect_ratio: None,
|
||||
quality: None,
|
||||
duration_seconds: Some(5),
|
||||
video_batch_count: Some(1),
|
||||
generate_audio: None,
|
||||
}
|
||||
}
|
||||
|
||||
fn token(id: &str) -> MediaFileToken {
|
||||
MediaFileToken::new(id.to_string())
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn passes_when_model_is_none() {
|
||||
let req = base_request();
|
||||
assert!(validate_when_image_required(&req).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn passes_for_other_models_without_image() {
|
||||
// Only v1.5 needs the gate; other models can do text-to-video.
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::Seedance2p0),
|
||||
..base_request()
|
||||
};
|
||||
assert!(validate_when_image_required(&req).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_v1p5_with_no_image_inputs() {
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
..base_request()
|
||||
};
|
||||
let err = validate_when_image_required(&req).expect_err("should reject");
|
||||
match err {
|
||||
CommonWebError::BadInputWithSimpleMessage(msg) => {
|
||||
assert_eq!(msg, "Image is required");
|
||||
}
|
||||
other => panic!("expected BadInputWithSimpleMessage, got {:?}", other),
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn rejects_v1p5_with_empty_reference_list_and_no_start_frame() {
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
reference_image_media_tokens: Some(vec![]),
|
||||
..base_request()
|
||||
};
|
||||
assert!(matches!(
|
||||
validate_when_image_required(&req),
|
||||
Err(CommonWebError::BadInputWithSimpleMessage(_))
|
||||
));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_v1p5_with_start_frame_only() {
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
start_frame_image_media_token: Some(token("mf_start")),
|
||||
..base_request()
|
||||
};
|
||||
assert!(validate_when_image_required(&req).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_v1p5_with_reference_images_only() {
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
reference_image_media_tokens: Some(vec![token("mf_a"), token("mf_b")]),
|
||||
..base_request()
|
||||
};
|
||||
assert!(validate_when_image_required(&req).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn accepts_v1p5_with_both_start_frame_and_reference_images() {
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
start_frame_image_media_token: Some(token("mf_start")),
|
||||
reference_image_media_tokens: Some(vec![token("mf_a")]),
|
||||
..base_request()
|
||||
};
|
||||
assert!(validate_when_image_required(&req).is_ok());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn end_frame_alone_does_not_count_as_image_input() {
|
||||
// end_frame_image_media_token is technically an image, but xAI's v1.5
|
||||
// wire shape only uses start_frame / reference_images. end_frame
|
||||
// shouldn't satisfy the gate.
|
||||
let req = OmniGenVideoCostAndGenerateRequest {
|
||||
model: Some(CommonVideoModel::GrokImagineVideo1p5),
|
||||
end_frame_image_media_token: Some(token("mf_end")),
|
||||
..base_request()
|
||||
};
|
||||
assert!(validate_when_image_required(&req).is_err());
|
||||
}
|
||||
}
|
||||
@@ -1,3 +0,0 @@
|
||||
pub mod omni_upload_audio_media_file_handler;
|
||||
pub mod omni_upload_image_media_file_handler;
|
||||
pub mod omni_upload_video_media_file_handler;
|
||||
-327
@@ -1,327 +0,0 @@
|
||||
use std::collections::HashSet;
|
||||
use std::io::Read;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use actix_multipart::form::tempfile::TempFile;
|
||||
use actix_multipart::form::text::Text;
|
||||
use actix_multipart::form::MultipartForm;
|
||||
use actix_web::web::Json;
|
||||
use actix_web::{web, HttpRequest};
|
||||
use log::{error, info, warn};
|
||||
use once_cell::sync::Lazy;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use artcraft_api_defs::omni_api::omni_upload_audio::OmniUploadAudioMediaFileSuccessResponse;
|
||||
use bucket_paths::legacy::typified_paths::public::media_files::bucket_file_path::MediaFileBucketPath;
|
||||
use enums::by_table::media_files::media_file_class::MediaFileClass;
|
||||
use enums::by_table::media_files::media_file_type::MediaFileType;
|
||||
use enums::common::visibility::Visibility;
|
||||
use hashing::sha256::sha256_hash_bytes::sha256_hash_bytes;
|
||||
use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use media::decode_basic_audio_info::decode_basic_audio_bytes_info;
|
||||
use mimetypes::mimetype_for_bytes::get_mimetype_for_bytes;
|
||||
use mimetypes::mimetype_to_extension::mimetype_to_extension;
|
||||
use mysql_queries::queries::idepotency_tokens::insert_idempotency_token::insert_idempotency_token;
|
||||
use mysql_queries::queries::media_files::create::specialized_insert::insert_media_file_from_file_upload::{insert_media_file_from_file_upload, InsertMediaFileFromUploadArgs, UploadType};
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::media_files::upload::upload_error::MediaFileUploadError;
|
||||
use crate::http_server::endpoints::media_files::upload::common_utils::try_parse_generation_provider::try_parse_generation_provider;
|
||||
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
|
||||
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
/// Form-multipart request fields.
|
||||
///
|
||||
/// IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadAudioMediaFileForm` (Under "Schema") FOR DETAILS ON FIELDS AND NULLABILITY.
|
||||
#[derive(MultipartForm, ToSchema)]
|
||||
#[multipart(duplicate_field = "deny")]
|
||||
pub struct OmniUploadAudioMediaFileForm {
|
||||
/// UUID for request idempotency
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = String, format = Binary)]
|
||||
uuid_idempotency_token: Text<String>,
|
||||
|
||||
/// The uploaded file
|
||||
#[multipart(limit = "512 MiB")]
|
||||
#[schema(value_type = Vec<u8>, format = Binary)]
|
||||
file: TempFile,
|
||||
|
||||
/// Optional: Title (name) of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_title: Option<Text<String>>,
|
||||
|
||||
/// Optional: Visibility of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<Visibility>, format = Binary)]
|
||||
maybe_visibility: Option<Text<Visibility>>,
|
||||
|
||||
/// Optional: The third-party generation provider (e.g. "fal", "replicate").
|
||||
/// If set, `is_user_upload` will be false and `is_intermediate_system_file` will be forced false.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_generation_provider: Option<Text<String>>,
|
||||
}
|
||||
|
||||
static ALLOWED_MIME_TYPES : Lazy<HashSet<&'static str>> = Lazy::new(|| {
|
||||
HashSet::from([
|
||||
// Audio
|
||||
"audio/aac",
|
||||
"audio/m4a",
|
||||
"audio/mpeg",
|
||||
"audio/ogg",
|
||||
"audio/opus",
|
||||
"audio/x-flac",
|
||||
"audio/x-wav",
|
||||
// Mixed
|
||||
"audio/mp4", // iPhone seems to upload these as audio
|
||||
// Video
|
||||
"video/mp4",
|
||||
"video/webm",
|
||||
])
|
||||
});
|
||||
|
||||
/// API-key authenticated audio upload (Omni API). Identity is read from the `Authorization` header
|
||||
/// API key rather than a session cookie.
|
||||
#[utoipa::path(
|
||||
post,
|
||||
tag = "Omni API",
|
||||
path = "/v1/omni_api/upload/audio",
|
||||
responses(
|
||||
(status = 200, description = "Success Update", body = OmniUploadAudioMediaFileSuccessResponse),
|
||||
(status = 400, description = "Bad input", body = MediaFileUploadError),
|
||||
(status = 401, description = "Not authorized", body = MediaFileUploadError),
|
||||
(status = 429, description = "Too many requests", body = MediaFileUploadError),
|
||||
(status = 500, description = "Server error", body = MediaFileUploadError),
|
||||
),
|
||||
params(
|
||||
(
|
||||
"request" = OmniUploadAudioMediaFileForm,
|
||||
description = "IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadAudioMediaFileForm` (Under 'Schema') FOR DETAILS ON FIELDS AND NULLABILITY."
|
||||
),
|
||||
)
|
||||
)]
|
||||
pub async fn omni_upload_audio_media_file_handler(
|
||||
http_request: HttpRequest,
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
MultipartForm(mut form): MultipartForm<OmniUploadAudioMediaFileForm>,
|
||||
) -> Result<Json<OmniUploadAudioMediaFileSuccessResponse>, MediaFileUploadError> {
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool
|
||||
.acquire()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("MySql pool error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== API KEY USER ==================== //
|
||||
|
||||
// API-key authentication (Authorization header) instead of a session cookie. Never cached, and a
|
||||
// banned owner is rejected inside `require_api_key_user`.
|
||||
let api_session = require_api_key_user(&http_request, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(map_api_key_auth_error)?;
|
||||
|
||||
let maybe_user_token = Some(&api_session.user_token);
|
||||
|
||||
// ==================== RATE LIMIT ==================== //
|
||||
|
||||
let rate_limiter = &server_state.redis_rate_limiters.file_upload_logged_in;
|
||||
|
||||
if let Err(_err) = rate_limiter.rate_limit_request(&http_request).await {
|
||||
return Err(MediaFileUploadError::RateLimited);
|
||||
}
|
||||
|
||||
// ==================== HANDLE IDEMPOTENCY ==================== //
|
||||
|
||||
let uuid_idempotency_token = form.uuid_idempotency_token.as_ref();
|
||||
|
||||
if let Err(reason) = validate_idempotency_token_format(uuid_idempotency_token) {
|
||||
return Err(MediaFileUploadError::BadInput(reason));
|
||||
}
|
||||
|
||||
insert_idempotency_token(uuid_idempotency_token, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error inserting idempotency token: {:?}", err);
|
||||
MediaFileUploadError::BadInput("invalid idempotency token".to_string())
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD METADATA ==================== //
|
||||
|
||||
let maybe_title = form.maybe_title
|
||||
.map(|title| title.trim().to_string())
|
||||
.filter(|title| !title.is_empty());
|
||||
|
||||
let creator_set_visibility = form.maybe_visibility
|
||||
.map(|visibility| visibility.0)
|
||||
.unwrap_or(Visibility::default());
|
||||
|
||||
// ==================== USER DATA ==================== //
|
||||
|
||||
let ip_address = get_request_ip(&http_request);
|
||||
|
||||
// ==================== FILE VALIDATION ==================== //
|
||||
|
||||
let mut file_bytes = Vec::new();
|
||||
form.file.file.read_to_end(&mut file_bytes)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading file: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let mimetype = get_mimetype_for_bytes(file_bytes.as_ref())
|
||||
.map(|mimetype| mimetype.to_string())
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine mimetype for file");
|
||||
MediaFileUploadError::BadInput("Could not determine mimetype for file".to_string())
|
||||
})?;
|
||||
|
||||
if !ALLOWED_MIME_TYPES.contains(mimetype.as_str()) {
|
||||
// NB: Don't let our error message inject malicious strings
|
||||
let filtered_mimetype = mimetype
|
||||
.chars()
|
||||
.filter(|c| c.is_ascii())
|
||||
.filter(|c| c.is_alphanumeric() || *c == '/')
|
||||
.collect::<String>();
|
||||
return Err(MediaFileUploadError::BadInput(format!("unpermitted mime type: {}", &filtered_mimetype)));
|
||||
}
|
||||
|
||||
// ==================== DURATION DETECTION ==================== //
|
||||
|
||||
let do_audio_decode = match mimetype.as_ref() {
|
||||
// TODO: Revisit when Safari can send us this metadata consistently
|
||||
"audio/mp4" | "video/mp4" => false,
|
||||
// Symphonia is currently broken for Firefox's opus; skip decoding it.
|
||||
"audio/opus" => false,
|
||||
// Also, don't decode images
|
||||
"image/jpeg" => false,
|
||||
"image/png" => false,
|
||||
"image/webp" => false,
|
||||
_ => true,
|
||||
};
|
||||
|
||||
let mut maybe_duration_millis = None;
|
||||
|
||||
if do_audio_decode {
|
||||
let basic_info = decode_basic_audio_bytes_info(
|
||||
file_bytes.as_ref(),
|
||||
Some(&mimetype),
|
||||
None
|
||||
).map_err(|e| {
|
||||
warn!("file decoding error: {:?}", e);
|
||||
MediaFileUploadError::BadInput("could not decode file".to_string())
|
||||
})?;
|
||||
|
||||
maybe_duration_millis = basic_info.duration_millis;
|
||||
}
|
||||
|
||||
// ==================== OTHER FILE METADATA ==================== //
|
||||
|
||||
let maybe_filename = form.file.file_name.as_deref()
|
||||
.as_deref()
|
||||
.map(|filename| PathBuf::from(filename));
|
||||
|
||||
let extension = mimetype_to_extension(&mimetype)
|
||||
.or_else(|| {
|
||||
maybe_filename
|
||||
.as_ref()
|
||||
.and_then(|filename| filename.extension())
|
||||
.and_then(|ext| ext.to_str())
|
||||
})
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine file extension for mimetype: {}", &mimetype);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let extension = format!(".{extension}"); // NB: needs dot prefix
|
||||
|
||||
let file_size_bytes = file_bytes.len();
|
||||
|
||||
let hash = sha256_hash_bytes(&file_bytes)
|
||||
.map_err(|io_error| {
|
||||
error!("Problem hashing bytes: {:?}", io_error);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD AND SAVE ==================== //
|
||||
|
||||
const PREFIX : Option<&str> = Some("aud_");
|
||||
|
||||
let public_upload_path = MediaFileBucketPath::generate_new(PREFIX, Some(&extension));
|
||||
|
||||
info!("Uploading media to bucket path: {}", public_upload_path.get_full_object_path_str());
|
||||
|
||||
server_state.public_bucket_client.upload_file_with_content_type(
|
||||
public_upload_path.get_full_object_path_str(),
|
||||
file_bytes.as_ref(),
|
||||
&mimetype)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
warn!("Upload media bytes to bucket error: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let maybe_generation_provider = form.maybe_generation_provider
|
||||
.as_ref()
|
||||
.and_then(|text| try_parse_generation_provider(text.as_ref()));
|
||||
|
||||
let upload_type = if maybe_generation_provider.is_some() {
|
||||
UploadType::ThirdPartyInference
|
||||
} else {
|
||||
UploadType::Filesystem
|
||||
};
|
||||
|
||||
let (token, record_id) = insert_media_file_from_file_upload(InsertMediaFileFromUploadArgs {
|
||||
maybe_media_class: Some(MediaFileClass::Audio),
|
||||
media_file_type: MediaFileType::Audio,
|
||||
maybe_creator_user_token: maybe_user_token,
|
||||
// NB: AVT (anonymous visitor) tokens are a web-session concept; API-key callers have none.
|
||||
maybe_creator_anonymous_visitor_token: None,
|
||||
creator_ip_address: &ip_address,
|
||||
creator_set_visibility,
|
||||
upload_type,
|
||||
maybe_engine_category: None,
|
||||
maybe_animation_type: None,
|
||||
maybe_prompt_token: None,
|
||||
maybe_batch_token: None,
|
||||
maybe_mime_type: Some(&mimetype),
|
||||
file_size_bytes: file_size_bytes as u64,
|
||||
maybe_duration_millis,
|
||||
sha256_checksum: &hash,
|
||||
maybe_title: maybe_title.as_deref(),
|
||||
maybe_scene_source_media_file_token: None,
|
||||
is_intermediate_system_file: false, // NB: is_user_upload = true
|
||||
maybe_generation_provider,
|
||||
public_bucket_directory_hash: public_upload_path.get_object_hash(),
|
||||
maybe_public_bucket_prefix: PREFIX,
|
||||
maybe_public_bucket_extension: Some(&extension),
|
||||
pool: &server_state.mysql_pool,
|
||||
})
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("New file creation DB error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
info!("new media file id: {} token: {:?}", record_id, &token);
|
||||
|
||||
Ok(Json(OmniUploadAudioMediaFileSuccessResponse {
|
||||
success: true,
|
||||
media_file_token: token,
|
||||
}))
|
||||
}
|
||||
|
||||
/// Map an API-key authentication failure onto this endpoint's error type. A genuine 401 stays a
|
||||
/// 401; anything else (e.g. a DB error during lookup) becomes a 500.
|
||||
fn map_api_key_auth_error(err: CommonWebError) -> MediaFileUploadError {
|
||||
match err {
|
||||
CommonWebError::NotAuthorized => {
|
||||
MediaFileUploadError::NotAuthorizedVerbose("invalid or missing API key".to_string())
|
||||
}
|
||||
_ => MediaFileUploadError::ServerError,
|
||||
}
|
||||
}
|
||||
-338
@@ -1,338 +0,0 @@
|
||||
use std::collections::HashSet;
|
||||
use std::io::Read;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
|
||||
use actix_multipart::form::tempfile::TempFile;
|
||||
use actix_multipart::form::text::Text;
|
||||
use actix_multipart::form::MultipartForm;
|
||||
use actix_web::web::Json;
|
||||
use actix_web::{web, HttpRequest};
|
||||
use log::{error, info, warn};
|
||||
use once_cell::sync::Lazy;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use artcraft_api_defs::omni_api::omni_upload_image::OmniUploadImageMediaFileSuccessResponse;
|
||||
use bucket_paths::legacy::typified_paths::public::media_files::bucket_file_path::MediaFileBucketPath;
|
||||
use enums::by_table::media_files::media_file_class::MediaFileClass;
|
||||
use enums::by_table::media_files::media_file_type::MediaFileType;
|
||||
use enums::common::visibility::Visibility;
|
||||
use hashing::sha256::sha256_hash_bytes::sha256_hash_bytes;
|
||||
use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use mimetypes::mimetype_for_bytes::get_mimetype_for_bytes;
|
||||
use mimetypes::mimetype_to_extension::mimetype_to_extension;
|
||||
use mysql_queries::queries::idepotency_tokens::insert_idempotency_token::insert_idempotency_token;
|
||||
use mysql_queries::queries::media_files::create::specialized_insert::insert_media_file_from_file_upload::{insert_media_file_from_file_upload, InsertMediaFileFromUploadArgs, UploadType};
|
||||
use tokens::tokens::batch_generations::BatchGenerationToken;
|
||||
use tokens::tokens::prompts::PromptToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::media_files::upload::upload_error::MediaFileUploadError;
|
||||
use crate::http_server::endpoints::media_files::upload::common_utils::try_parse_generation_provider::try_parse_generation_provider;
|
||||
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
|
||||
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
/// Form-multipart request fields.
|
||||
///
|
||||
/// IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadImageMediaFileForm` (Under "Schema") FOR DETAILS ON FIELDS AND NULLABILITY.
|
||||
#[derive(MultipartForm, ToSchema)]
|
||||
#[multipart(duplicate_field = "deny")]
|
||||
pub struct OmniUploadImageMediaFileForm {
|
||||
/// UUID for request idempotency
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = String, format = Binary)]
|
||||
uuid_idempotency_token: Text<String>,
|
||||
|
||||
/// The uploaded file
|
||||
#[multipart(limit = "512 MiB")]
|
||||
#[schema(value_type = Vec<u8>, format = Binary)]
|
||||
file: TempFile,
|
||||
|
||||
/// Optional: Title (name) of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_title: Option<Text<String>>,
|
||||
|
||||
/// Optional: Visibility of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<Visibility>, format = Binary)]
|
||||
maybe_visibility: Option<Text<Visibility>>,
|
||||
|
||||
/// Optional: Prompt associated with this image
|
||||
/// NOTE: Cannot set `is_intermediate_system_file = true` if this is set.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_prompt_token: Option<Text<PromptToken>>,
|
||||
|
||||
/// Optional: Batch associated with this image
|
||||
/// NOTE: Cannot set `is_intermediate_system_file = true` if this is set.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_batch_token: Option<Text<BatchGenerationToken>>,
|
||||
|
||||
/// Optional: Whether this is a system file (e.g. cover files we should hide)
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<bool>, format = Binary)]
|
||||
is_intermediate_system_file: Option<Text<bool>>,
|
||||
|
||||
/// Optional: The third-party generation provider (e.g. "fal", "replicate").
|
||||
/// If set, `is_user_upload` will be false and `is_intermediate_system_file` will be forced false.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_generation_provider: Option<Text<String>>,
|
||||
}
|
||||
|
||||
static ALLOWED_MIME_TYPES : Lazy<HashSet<&'static str>> = Lazy::new(|| {
|
||||
HashSet::from([
|
||||
"image/jpeg",
|
||||
"image/png",
|
||||
"image/gif",
|
||||
"image/webp",
|
||||
])
|
||||
});
|
||||
|
||||
/// API-key authenticated image upload (Omni API). Identity is read from the `Authorization` header
|
||||
/// API key rather than a session cookie.
|
||||
#[utoipa::path(
|
||||
post,
|
||||
tag = "Omni API",
|
||||
path = "/v1/omni_api/upload/image",
|
||||
responses(
|
||||
(status = 200, description = "Success Update", body = OmniUploadImageMediaFileSuccessResponse),
|
||||
(status = 400, description = "Bad input", body = MediaFileUploadError),
|
||||
(status = 401, description = "Not authorized", body = MediaFileUploadError),
|
||||
(status = 429, description = "Too many requests", body = MediaFileUploadError),
|
||||
(status = 500, description = "Server error", body = MediaFileUploadError),
|
||||
),
|
||||
params(
|
||||
(
|
||||
"request" = OmniUploadImageMediaFileForm,
|
||||
description = "IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadImageMediaFileForm` (Under 'Schema') FOR DETAILS ON FIELDS AND NULLABILITY."
|
||||
),
|
||||
)
|
||||
)]
|
||||
pub async fn omni_upload_image_media_file_handler(
|
||||
http_request: HttpRequest,
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
MultipartForm(mut form): MultipartForm<OmniUploadImageMediaFileForm>,
|
||||
) -> Result<Json<OmniUploadImageMediaFileSuccessResponse>, MediaFileUploadError> {
|
||||
|
||||
validate_form(&form)?;
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool
|
||||
.acquire()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("MySql pool error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== API KEY USER ==================== //
|
||||
|
||||
// API-key authentication (Authorization header) instead of a session cookie. Never cached, and a
|
||||
// banned owner is rejected inside `require_api_key_user`.
|
||||
let api_session = require_api_key_user(&http_request, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(map_api_key_auth_error)?;
|
||||
|
||||
let maybe_user_token = Some(&api_session.user_token);
|
||||
|
||||
// ==================== HANDLE IDEMPOTENCY ==================== //
|
||||
|
||||
let uuid_idempotency_token = form.uuid_idempotency_token.as_ref();
|
||||
|
||||
if let Err(reason) = validate_idempotency_token_format(uuid_idempotency_token) {
|
||||
return Err(MediaFileUploadError::BadInput(reason));
|
||||
}
|
||||
|
||||
insert_idempotency_token(uuid_idempotency_token, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error inserting idempotency token: {:?}", err);
|
||||
MediaFileUploadError::BadInput("invalid idempotency token".to_string())
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD METADATA ==================== //
|
||||
|
||||
let maybe_title = form.maybe_title.map(|title| title.to_string());
|
||||
|
||||
let creator_set_visibility = form.maybe_visibility
|
||||
.map(|visibility| visibility.0)
|
||||
.unwrap_or(Visibility::default());
|
||||
|
||||
// ==================== USER DATA ==================== //
|
||||
|
||||
let ip_address = get_request_ip(&http_request);
|
||||
|
||||
// ==================== FILE VALIDATION ==================== //
|
||||
|
||||
let mut file_bytes = Vec::new();
|
||||
form.file.file.read_to_end(&mut file_bytes)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading file: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let mimetype = get_mimetype_for_bytes(file_bytes.as_ref())
|
||||
.map(|mimetype| mimetype.to_string())
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine mimetype for file");
|
||||
MediaFileUploadError::BadInput("Could not determine mimetype for file".to_string())
|
||||
})?;
|
||||
|
||||
if !ALLOWED_MIME_TYPES.contains(mimetype.as_str()) {
|
||||
// NB: Don't let our error message inject malicious strings
|
||||
let filtered_mimetype = mimetype
|
||||
.chars()
|
||||
.filter(|c| c.is_ascii())
|
||||
.filter(|c| c.is_alphanumeric() || *c == '/')
|
||||
.collect::<String>();
|
||||
return Err(MediaFileUploadError::BadInput(format!("unpermitted mime type: {}", &filtered_mimetype)));
|
||||
}
|
||||
|
||||
// ==================== OTHER FILE METADATA ==================== //
|
||||
|
||||
let maybe_filename = form.file.file_name.as_deref()
|
||||
.as_deref()
|
||||
.map(|filename| PathBuf::from(filename));
|
||||
|
||||
let extension = mimetype_to_extension(&mimetype)
|
||||
.or_else(|| {
|
||||
maybe_filename
|
||||
.as_ref()
|
||||
.and_then(|filename| filename.extension())
|
||||
.and_then(|ext| ext.to_str())
|
||||
})
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine file extension for mimetype: {}", &mimetype);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let extension = format!(".{extension}"); // NB: needs dot prefix
|
||||
|
||||
let file_size_bytes = file_bytes.len();
|
||||
|
||||
let hash = sha256_hash_bytes(&file_bytes)
|
||||
.map_err(|io_error| {
|
||||
error!("Problem hashing bytes: {:?}", io_error);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD AND SAVE ==================== //
|
||||
|
||||
const PREFIX : Option<&str> = Some("image_");
|
||||
|
||||
let public_upload_path = MediaFileBucketPath::generate_new(PREFIX, Some(&extension));
|
||||
|
||||
info!("Uploading media to bucket path: {}", public_upload_path.get_full_object_path_str());
|
||||
|
||||
server_state.public_bucket_client.upload_file_with_content_type(
|
||||
public_upload_path.get_full_object_path_str(),
|
||||
file_bytes.as_ref(),
|
||||
&mimetype)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
warn!("Upload media bytes to bucket error: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let is_intermediate_system_file = form.is_intermediate_system_file
|
||||
.map(|b| b.0)
|
||||
.unwrap_or(false);
|
||||
|
||||
let maybe_prompt_token = form.maybe_prompt_token
|
||||
.as_ref()
|
||||
.map(|token| token.0.clone());
|
||||
|
||||
let maybe_batch_token = form.maybe_batch_token
|
||||
.as_ref()
|
||||
.map(|token| token.0.clone());
|
||||
|
||||
let maybe_generation_provider = form.maybe_generation_provider
|
||||
.as_ref()
|
||||
.and_then(|text| try_parse_generation_provider(text.as_ref()));
|
||||
|
||||
let upload_type = if maybe_generation_provider.is_some() {
|
||||
UploadType::ThirdPartyInference
|
||||
} else {
|
||||
UploadType::Filesystem
|
||||
};
|
||||
|
||||
let media_file_type = match mimetype.as_str() {
|
||||
"image/jpeg" => MediaFileType::Jpg,
|
||||
"image/png" => MediaFileType::Png,
|
||||
"image/gif" => MediaFileType::Gif,
|
||||
"image/webp" => MediaFileType::Image, // Fallback
|
||||
_ => MediaFileType::Image, // fallback
|
||||
};
|
||||
|
||||
let (token, record_id) = insert_media_file_from_file_upload(InsertMediaFileFromUploadArgs {
|
||||
maybe_media_class: Some(MediaFileClass::Image),
|
||||
media_file_type,
|
||||
maybe_creator_user_token: maybe_user_token,
|
||||
// NB: AVT (anonymous visitor) tokens are a web-session concept; API-key callers have none.
|
||||
maybe_creator_anonymous_visitor_token: None,
|
||||
creator_ip_address: &ip_address,
|
||||
creator_set_visibility,
|
||||
upload_type,
|
||||
maybe_engine_category: None,
|
||||
maybe_animation_type: None,
|
||||
maybe_mime_type: Some(&mimetype),
|
||||
maybe_prompt_token: maybe_prompt_token.as_ref(),
|
||||
maybe_batch_token: maybe_batch_token.as_ref(),
|
||||
file_size_bytes: file_size_bytes as u64,
|
||||
maybe_duration_millis: None,
|
||||
sha256_checksum: &hash,
|
||||
maybe_title: maybe_title.as_deref(),
|
||||
maybe_scene_source_media_file_token: None,
|
||||
is_intermediate_system_file,
|
||||
maybe_generation_provider,
|
||||
public_bucket_directory_hash: public_upload_path.get_object_hash(),
|
||||
maybe_public_bucket_prefix: PREFIX,
|
||||
maybe_public_bucket_extension: Some(&extension),
|
||||
pool: &server_state.mysql_pool,
|
||||
})
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("New file creation DB error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
info!("new media file id: {} token: {:?}", record_id, &token);
|
||||
|
||||
Ok(Json(OmniUploadImageMediaFileSuccessResponse {
|
||||
success: true,
|
||||
media_file_token: token,
|
||||
}))
|
||||
}
|
||||
|
||||
fn validate_form(form: &OmniUploadImageMediaFileForm) -> Result<(), MediaFileUploadError> {
|
||||
if form.uuid_idempotency_token.is_empty() {
|
||||
return Err(MediaFileUploadError::BadInput("Idempotency token is required".to_string()));
|
||||
}
|
||||
|
||||
let is_intermediate_system_file = form.is_intermediate_system_file
|
||||
.as_ref()
|
||||
.map(|b| b.0)
|
||||
.unwrap_or(false);
|
||||
|
||||
if is_intermediate_system_file && form.maybe_prompt_token.is_some() {
|
||||
return Err(MediaFileUploadError::BadInput(
|
||||
"Cannot set `is_intermediate_system_file` to true if `maybe_prompt_token` is provided."
|
||||
.to_string()));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Map an API-key authentication failure onto this endpoint's error type. A genuine 401 stays a
|
||||
/// 401; anything else (e.g. a DB error during lookup) becomes a 500.
|
||||
fn map_api_key_auth_error(err: CommonWebError) -> MediaFileUploadError {
|
||||
match err {
|
||||
CommonWebError::NotAuthorized => {
|
||||
MediaFileUploadError::NotAuthorizedVerbose("invalid or missing API key".to_string())
|
||||
}
|
||||
_ => MediaFileUploadError::ServerError,
|
||||
}
|
||||
}
|
||||
-557
@@ -1,557 +0,0 @@
|
||||
use std::collections::HashSet;
|
||||
use std::io::Read;
|
||||
use std::path::PathBuf;
|
||||
use std::sync::Arc;
|
||||
use std::time::Duration;
|
||||
|
||||
use actix_multipart::form::tempfile::TempFile;
|
||||
use actix_multipart::form::text::Text;
|
||||
use actix_multipart::form::MultipartForm;
|
||||
use actix_web::web::Json;
|
||||
use actix_web::{web, HttpRequest};
|
||||
use log::{debug, error, info, warn};
|
||||
use once_cell::sync::Lazy;
|
||||
use utoipa::ToSchema;
|
||||
|
||||
use artcraft_api_defs::omni_api::omni_upload_video::OmniUploadVideoMediaFileSuccessResponse;
|
||||
use bucket_paths::legacy::typified_paths::public::media_files::bucket_file_path::MediaFileBucketPath;
|
||||
use enums::by_table::media_files::media_file_class::MediaFileClass;
|
||||
use enums::by_table::media_files::media_file_type::MediaFileType;
|
||||
use enums::common::visibility::Visibility;
|
||||
use ffmpeg_utils::ffmpeg::ffmpeg_transcode_to_mp4::{ffmpeg_transcode_to_mp4, FfmpegTranscodeToMp4Args};
|
||||
use ffmpeg_utils::ffmpeg::ffmpeg_trim_and_resample::{ffmpeg_trim_and_resample, Args};
|
||||
use ffmpeg_utils::ffprobe::ffprobe_get_info::ffprobe_get_info;
|
||||
use filesys::file_read_bytes::file_read_bytes;
|
||||
use filesys::path_to_string::path_to_string;
|
||||
use hashing::sha256::sha256_hash_bytes::sha256_hash_bytes;
|
||||
use http_server_common::request::get_request_ip::get_request_ip;
|
||||
use mimetypes::mimetype_for_bytes::get_mimetype_for_bytes;
|
||||
use mimetypes::mimetype_to_extension::mimetype_to_extension;
|
||||
use mysql_queries::queries::idepotency_tokens::insert_idempotency_token::insert_idempotency_token;
|
||||
use mysql_queries::queries::media_files::create::specialized_insert::insert_media_file_from_file_upload::{insert_media_file_from_file_upload, InsertMediaFileFromUploadArgs, UploadType};
|
||||
use thumbnail_generator::task_client::thumbnail_task::{ThumbnailTaskBuilder, ThumbnailTaskInputMimeType};
|
||||
use tokens::tokens::media_files::MediaFileToken;
|
||||
use tokens::tokens::prompts::PromptToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::endpoints::media_files::upload::upload_error::MediaFileUploadError;
|
||||
use crate::http_server::endpoints::media_files::upload::common_utils::try_parse_generation_provider::try_parse_generation_provider;
|
||||
use crate::http_server::user_lookup::api_keys::require_api_key_user::require_api_key_user;
|
||||
use crate::http_server::validations::validate_idempotency_token_format::validate_idempotency_token_format;
|
||||
use crate::state::server_state::ServerState;
|
||||
|
||||
/// Form-multipart request fields.
|
||||
///
|
||||
/// IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadVideoMediaFileForm` (Under "Schema") FOR DETAILS ON FIELDS AND NULLABILITY.
|
||||
#[derive(MultipartForm, ToSchema)]
|
||||
#[multipart(duplicate_field = "deny")]
|
||||
pub struct OmniUploadVideoMediaFileForm {
|
||||
/// UUID for request idempotency
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = String, format = Binary)]
|
||||
uuid_idempotency_token: Text<String>,
|
||||
|
||||
/// The uploaded file
|
||||
#[multipart(limit = "512 MiB")]
|
||||
#[schema(value_type = Vec<u8>, format = Binary)]
|
||||
file: TempFile,
|
||||
|
||||
/// Optional: Title (name) of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_title: Option<Text<String>>,
|
||||
|
||||
/// Optional: Visibility of the scene
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<Visibility>, format = Binary)]
|
||||
maybe_visibility: Option<Text<Visibility>>,
|
||||
|
||||
/// Optional: Prompt associated with this image
|
||||
/// NOTE: Cannot set `is_intermediate_system_file = true` if this is set.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_prompt_token: Option<Text<PromptToken>>,
|
||||
|
||||
/// Optional: If an engine scene was used to generate this video, provide it here to create a link.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<MediaFileToken>, format = Binary)]
|
||||
maybe_scene_source_media_file_token: Option<Text<MediaFileToken>>,
|
||||
|
||||
/// Optional: Whether this is a system file (eg. cover files we should hide)
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<bool>, format = Binary)]
|
||||
is_intermediate_system_file: Option<Text<bool>>,
|
||||
|
||||
/// Optional: Trim start offset in milliseconds.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<u64>, format = Binary)]
|
||||
maybe_trim_start_millis: Option<Text<u64>>,
|
||||
|
||||
/// Optional: Trim end offset in milliseconds.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<u64>, format = Binary)]
|
||||
maybe_trim_end_millis: Option<Text<u64>>,
|
||||
|
||||
/// Optional: Resample the video to this FPS.
|
||||
/// Only certain values equal or under 24fps are allowed.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<u64>, format = Binary)]
|
||||
maybe_resample_fps: Option<Text<u8>>,
|
||||
|
||||
/// Optional: The third-party generation provider (e.g. "fal", "replicate").
|
||||
/// If set, `is_user_upload` will be false and `is_intermediate_system_file` will be forced false.
|
||||
#[multipart(limit = "2 KiB")]
|
||||
#[schema(value_type = Option<String>, format = Binary)]
|
||||
maybe_generation_provider: Option<Text<String>>,
|
||||
}
|
||||
|
||||
static ALLOWED_MIME_TYPES : Lazy<HashSet<&'static str>> = Lazy::new(|| {
|
||||
HashSet::from([
|
||||
// Video
|
||||
"video/mp4", // NB: Only mp4 for now.
|
||||
])
|
||||
});
|
||||
|
||||
static TRANSCODE_MIME_TYPES : Lazy<HashSet<&'static str>> = Lazy::new(|| {
|
||||
HashSet::from([
|
||||
// Video
|
||||
"video/quicktime", // .qt quicktime files
|
||||
])
|
||||
});
|
||||
|
||||
/// API-key authenticated video upload (Omni API). Identity is read from the `Authorization` header
|
||||
/// API key rather than a session cookie.
|
||||
#[utoipa::path(
|
||||
post,
|
||||
tag = "Omni API",
|
||||
path = "/v1/omni_api/upload/video",
|
||||
responses(
|
||||
(status = 200, description = "Success Update", body = OmniUploadVideoMediaFileSuccessResponse),
|
||||
(status = 400, description = "Bad input", body = MediaFileUploadError),
|
||||
(status = 401, description = "Not authorized", body = MediaFileUploadError),
|
||||
(status = 429, description = "Too many requests", body = MediaFileUploadError),
|
||||
(status = 500, description = "Server error", body = MediaFileUploadError),
|
||||
),
|
||||
params(
|
||||
(
|
||||
"request" = OmniUploadVideoMediaFileForm,
|
||||
description = "IF VIEWING DOCS, PLEASE SEE BOTTOM OF PAGE `OmniUploadVideoMediaFileForm` (Under 'Schema') FOR DETAILS ON FIELDS AND NULLABILITY."
|
||||
),
|
||||
)
|
||||
)]
|
||||
pub async fn omni_upload_video_media_file_handler(
|
||||
http_request: HttpRequest,
|
||||
server_state: web::Data<Arc<ServerState>>,
|
||||
MultipartForm(mut form): MultipartForm<OmniUploadVideoMediaFileForm>,
|
||||
) -> Result<Json<OmniUploadVideoMediaFileSuccessResponse>, MediaFileUploadError> {
|
||||
|
||||
fast_form_validations(&form)?;
|
||||
|
||||
let mut mysql_connection = server_state.mysql_pool
|
||||
.acquire()
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("MySql pool error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== API KEY USER ==================== //
|
||||
|
||||
// API-key authentication (Authorization header) instead of a session cookie. Never cached, and a
|
||||
// banned owner is rejected inside `require_api_key_user`.
|
||||
let api_session = require_api_key_user(&http_request, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(map_api_key_auth_error)?;
|
||||
|
||||
let maybe_user_token = Some(&api_session.user_token);
|
||||
|
||||
// ==================== RATE LIMIT ==================== //
|
||||
|
||||
let rate_limiter = &server_state.redis_rate_limiters.file_upload_logged_in;
|
||||
|
||||
if let Err(_err) = rate_limiter.rate_limit_request(&http_request).await {
|
||||
return Err(MediaFileUploadError::RateLimited);
|
||||
}
|
||||
|
||||
// ==================== HANDLE IDEMPOTENCY ==================== //
|
||||
|
||||
let uuid_idempotency_token = form.uuid_idempotency_token.as_ref();
|
||||
|
||||
if let Err(reason) = validate_idempotency_token_format(uuid_idempotency_token) {
|
||||
return Err(MediaFileUploadError::BadInput(reason));
|
||||
}
|
||||
|
||||
insert_idempotency_token(uuid_idempotency_token, &mut *mysql_connection)
|
||||
.await
|
||||
.map_err(|err| {
|
||||
error!("Error inserting idempotency token: {:?}", err);
|
||||
MediaFileUploadError::BadInput("invalid idempotency token".to_string())
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD METADATA ==================== //
|
||||
|
||||
let maybe_title = form.maybe_title
|
||||
.map(|title| title.trim().to_string())
|
||||
.filter(|title| !title.is_empty());
|
||||
|
||||
let creator_set_visibility = form.maybe_visibility
|
||||
.map(|visibility| visibility.0)
|
||||
.unwrap_or(Visibility::default());
|
||||
|
||||
// ==================== USER DATA ==================== //
|
||||
|
||||
let ip_address = get_request_ip(&http_request);
|
||||
|
||||
// ==================== FILE VALIDATION ==================== //
|
||||
|
||||
let mut file_bytes = Vec::new();
|
||||
form.file.file.read_to_end(&mut file_bytes)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading file: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let mut mimetype = get_mimetype_for_bytes(file_bytes.as_ref())
|
||||
.map(|mimetype| mimetype.to_string())
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine mimetype for file");
|
||||
MediaFileUploadError::BadInput("Could not determine mimetype for file".to_string())
|
||||
})?;
|
||||
|
||||
let needs_transcode = TRANSCODE_MIME_TYPES.contains(mimetype.as_str());
|
||||
|
||||
if !ALLOWED_MIME_TYPES.contains(mimetype.as_str()) && !needs_transcode {
|
||||
// NB: Don't let our error message inject malicious strings
|
||||
let filtered_mimetype = mimetype
|
||||
.chars()
|
||||
.filter(|c| c.is_ascii())
|
||||
.filter(|c| c.is_alphanumeric() || *c == '/')
|
||||
.collect::<String>();
|
||||
return Err(MediaFileUploadError::BadInput(format!("unpermitted mime type: {}", &filtered_mimetype)));
|
||||
}
|
||||
|
||||
// ==================== ORIGINAL FILE BOOKKEEPING ==================== //
|
||||
// Hold on to the user's original file (path + mimetype + extension) so we
|
||||
// can upload it as a `_original.{ext}` sibling later if we end up
|
||||
// transcoding and/or resampling it.
|
||||
|
||||
let original_file_path: PathBuf = form.file.file.path().to_path_buf();
|
||||
let original_mimetype: String = mimetype.clone();
|
||||
let original_filename_extension: Option<String> = form.file.file_name.as_deref()
|
||||
.and_then(|filename| std::path::Path::new(filename).extension())
|
||||
.and_then(|ext| ext.to_str())
|
||||
.map(|ext| ext.to_string());
|
||||
let original_extension: String = mimetype_to_extension(&original_mimetype)
|
||||
.map(|ext| ext.to_string())
|
||||
.or(original_filename_extension)
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine file extension for original mimetype: {}", &original_mimetype);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== OPTIONAL TRANSCODE TO MP4 ==================== //
|
||||
// Some accepted source mime types (e.g. QuickTime .mov) are not the canonical
|
||||
// distribution format we want for the media file record. Transcode them to
|
||||
// mp4 in a tempdir so we can resample / upload mp4 instead.
|
||||
|
||||
let mut transcode_tempdir_ref = None;
|
||||
let mut resample_tempdir_ref = None;
|
||||
|
||||
let mut final_upload_file_path = form.file.file.path().to_path_buf();
|
||||
|
||||
if needs_transcode {
|
||||
let transcode_tempdir = server_state.temp_dir_creator.new_tempdir("ffmpeg_transcode")
|
||||
.map_err(|err| {
|
||||
error!("Problem creating transcode temp dir: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let transcoded_path = transcode_tempdir.path().join("transcoded.mp4");
|
||||
|
||||
info!("Transcoding {} → mp4", &original_mimetype);
|
||||
ffmpeg_transcode_to_mp4(FfmpegTranscodeToMp4Args {
|
||||
video_input_path: form.file.file.path(),
|
||||
video_output_path: &transcoded_path,
|
||||
}).map_err(|err| {
|
||||
error!("Problem transcoding video to mp4: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
file_bytes = file_read_bytes(&transcoded_path)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading transcoded mp4: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
mimetype = "video/mp4".to_string();
|
||||
final_upload_file_path = transcoded_path;
|
||||
transcode_tempdir_ref = Some(transcode_tempdir);
|
||||
}
|
||||
|
||||
// ==================== OPTIONAL VIDEO RESAMPLE ==================== //
|
||||
|
||||
let should_resample = form.maybe_resample_fps.is_some()
|
||||
|| form.maybe_trim_start_millis.is_some()
|
||||
|| form.maybe_trim_end_millis.is_some();
|
||||
|
||||
if should_resample {
|
||||
let frame_temp_dir = server_state.temp_dir_creator.new_tempdir("ffmpeg")
|
||||
.map_err(|err| {
|
||||
error!("Problem creating temp dir: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let video_output_path = frame_temp_dir.path().join("output.mp4");
|
||||
|
||||
let maybe_new_frame_rate = form.maybe_resample_fps.map(|fps| fps.0);
|
||||
let maybe_start_offset = form.maybe_trim_start_millis.map(|millis| Duration::from_millis(millis.0));
|
||||
let maybe_end_offset = form.maybe_trim_end_millis.map(|millis| Duration::from_millis(millis.0));
|
||||
|
||||
ffmpeg_trim_and_resample(Args {
|
||||
video_input_path: &final_upload_file_path,
|
||||
video_output_path: &video_output_path,
|
||||
maybe_new_frame_rate,
|
||||
maybe_start_offset,
|
||||
maybe_end_offset,
|
||||
}).map_err(|err| {
|
||||
error!("Problem resampling video: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
file_bytes = file_read_bytes(&video_output_path)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading file: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
mimetype = get_mimetype_for_bytes(file_bytes.as_ref())
|
||||
.map(|mimetype| mimetype.to_string())
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine mimetype for file");
|
||||
MediaFileUploadError::BadInput("Could not determine mimetype for file".to_string())
|
||||
})?;
|
||||
|
||||
final_upload_file_path = video_output_path;
|
||||
resample_tempdir_ref = Some(frame_temp_dir); // NB: Keep from going out of scope
|
||||
}
|
||||
|
||||
// True if we touched the user's file in any way (transcoded, resampled, or both).
|
||||
let should_upload_original = needs_transcode || should_resample;
|
||||
|
||||
// ==================== OTHER FILE METADATA ==================== //
|
||||
|
||||
let mut maybe_duration_millis = None;
|
||||
|
||||
match ffprobe_get_info(&final_upload_file_path) {
|
||||
Ok(video_info) => {
|
||||
maybe_duration_millis = video_info.duration
|
||||
.map(|duration| duration.millis as u64);
|
||||
}
|
||||
Err(error) => {
|
||||
warn!("Error reading video dimensions with ffprobe: {:?}", error);
|
||||
}
|
||||
}
|
||||
|
||||
let maybe_filename = form.file.file_name.as_deref()
|
||||
.as_deref()
|
||||
.map(|filename| PathBuf::from(filename));
|
||||
|
||||
let extension = mimetype_to_extension(&mimetype)
|
||||
.or_else(|| {
|
||||
maybe_filename
|
||||
.as_ref()
|
||||
.and_then(|filename| filename.extension())
|
||||
.and_then(|ext| ext.to_str())
|
||||
})
|
||||
.ok_or_else(|| {
|
||||
warn!("Could not determine file extension for mimetype: {}", &mimetype);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
let extension = format!(".{extension}"); // NB: needs dot prefix
|
||||
|
||||
let file_size_bytes = file_bytes.len();
|
||||
|
||||
let hash = sha256_hash_bytes(&file_bytes)
|
||||
.map_err(|io_error| {
|
||||
error!("Problem hashing bytes: {:?}", io_error);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD AND SAVE ==================== //
|
||||
|
||||
const PREFIX : Option<&str> = Some("video_");
|
||||
|
||||
let public_upload_path = MediaFileBucketPath::generate_new(PREFIX, Some(&extension));
|
||||
|
||||
info!("Uploading media to bucket path: {}", public_upload_path.get_full_object_path_str());
|
||||
|
||||
server_state.public_bucket_client.upload_file_with_content_type(
|
||||
public_upload_path.get_full_object_path_str(),
|
||||
file_bytes.as_ref(),
|
||||
&mimetype)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
warn!("Upload media bytes to bucket error: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
// ==================== UPLOAD ORIGINAL SIBLING ==================== //
|
||||
// If we transcoded and/or resampled the user's upload, also upload the
|
||||
// unmodified original alongside the canonical file as `_original.{ext}`.
|
||||
|
||||
if should_upload_original {
|
||||
let original_suffix = format!("_original.{}", original_extension);
|
||||
let original_bucket_path = MediaFileBucketPath::from_object_hash(
|
||||
public_upload_path.get_object_hash(),
|
||||
PREFIX,
|
||||
Some(&original_suffix),
|
||||
);
|
||||
|
||||
info!(
|
||||
"Uploading original sibling to bucket path: {}",
|
||||
original_bucket_path.get_full_object_path_str(),
|
||||
);
|
||||
|
||||
let original_bytes = file_read_bytes(&original_file_path)
|
||||
.map_err(|e| {
|
||||
error!("Problem reading original file for sibling upload: {:?}", e);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
server_state.public_bucket_client.upload_file_with_content_type(
|
||||
original_bucket_path.get_full_object_path_str(),
|
||||
original_bytes.as_ref(),
|
||||
&original_mimetype,
|
||||
)
|
||||
.await
|
||||
.map_err(|e| {
|
||||
// NB: Sibling failure is non-fatal — the canonical upload already
|
||||
// succeeded and the user can still use the media. Log loudly.
|
||||
warn!("Upload original sibling to bucket error: {:?}", e);
|
||||
})
|
||||
.ok();
|
||||
}
|
||||
|
||||
let maybe_scene_source_media_file_token = form.maybe_scene_source_media_file_token
|
||||
.as_ref()
|
||||
.map(|token| &token.0);
|
||||
|
||||
let maybe_prompt_token = form.maybe_prompt_token
|
||||
.as_ref()
|
||||
.map(|token| token.0.clone());
|
||||
|
||||
// NB: If we're uploading a video file that references an engine scene, then this is an engine
|
||||
// render video, and we should mark it as a system (hidden) file.
|
||||
let is_intermediate_system_file =
|
||||
maybe_scene_source_media_file_token.is_some() ||
|
||||
form.is_intermediate_system_file.map(|b| b.0).unwrap_or(false);
|
||||
|
||||
let maybe_generation_provider = form.maybe_generation_provider
|
||||
.as_ref()
|
||||
.and_then(|text| try_parse_generation_provider(text.as_ref()));
|
||||
|
||||
let upload_type = if maybe_generation_provider.is_some() {
|
||||
UploadType::ThirdPartyInference
|
||||
} else {
|
||||
UploadType::Filesystem
|
||||
};
|
||||
|
||||
let (token, record_id) = insert_media_file_from_file_upload(InsertMediaFileFromUploadArgs {
|
||||
maybe_media_class: Some(MediaFileClass::Video),
|
||||
media_file_type: MediaFileType::Video,
|
||||
maybe_creator_user_token: maybe_user_token,
|
||||
// NB: AVT (anonymous visitor) tokens are a web-session concept; API-key callers have none.
|
||||
maybe_creator_anonymous_visitor_token: None,
|
||||
creator_ip_address: &ip_address,
|
||||
creator_set_visibility,
|
||||
upload_type,
|
||||
maybe_engine_category: None,
|
||||
maybe_animation_type: None,
|
||||
maybe_prompt_token: maybe_prompt_token.as_ref(),
|
||||
maybe_batch_token: None,
|
||||
maybe_mime_type: Some(&mimetype),
|
||||
file_size_bytes: file_size_bytes as u64,
|
||||
maybe_duration_millis,
|
||||
sha256_checksum: &hash,
|
||||
maybe_title: maybe_title.as_deref(),
|
||||
maybe_scene_source_media_file_token,
|
||||
is_intermediate_system_file, // NB: is_user_upload = TRUE
|
||||
maybe_generation_provider,
|
||||
public_bucket_directory_hash: public_upload_path.get_object_hash(),
|
||||
maybe_public_bucket_prefix: PREFIX,
|
||||
maybe_public_bucket_extension: Some(&extension),
|
||||
pool: &server_state.mysql_pool,
|
||||
})
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("New file creation DB error: {:?}", err);
|
||||
MediaFileUploadError::ServerError
|
||||
})?;
|
||||
|
||||
info!("new media file id: {} token: {:?}", record_id, &token);
|
||||
|
||||
let thumbnail_task_result = ThumbnailTaskBuilder::new_for_source_mimetype(ThumbnailTaskInputMimeType::MP4)
|
||||
.with_bucket(server_state.public_bucket_client.bucket_name().as_str())
|
||||
.with_path(&*path_to_string(public_upload_path.to_full_object_pathbuf()))
|
||||
.with_output_suffix("thumb")
|
||||
.with_event_id(&token.to_string())
|
||||
.send_all()
|
||||
.await;
|
||||
|
||||
match thumbnail_task_result {
|
||||
Ok(thumbnail_task) => {
|
||||
debug!("Thumbnail tasks sent: {:?}", thumbnail_task);
|
||||
},
|
||||
Err(e) => {
|
||||
error!("Failed to create some/all thumbnail tasks: {:?}", e);
|
||||
}
|
||||
}
|
||||
|
||||
Ok(Json(OmniUploadVideoMediaFileSuccessResponse {
|
||||
success: true,
|
||||
media_file_token: token,
|
||||
}))
|
||||
}
|
||||
|
||||
fn fast_form_validations(form: &OmniUploadVideoMediaFileForm) -> Result<(), MediaFileUploadError> {
|
||||
if let Some(resample_fps) = form.maybe_resample_fps.as_ref() {
|
||||
if **resample_fps > 24 {
|
||||
return Err(MediaFileUploadError::BadInput("Resample FPS must be 24 or lower".to_string()));
|
||||
} else if **resample_fps == 0 {
|
||||
return Err(MediaFileUploadError::BadInput("Resample FPS must be greater than 0".to_string()));
|
||||
}
|
||||
}
|
||||
|
||||
form.maybe_trim_start_millis.as_ref().zip(form.maybe_trim_end_millis.as_ref())
|
||||
.map(|(start, end)| {
|
||||
if **start >= **end {
|
||||
return Err(MediaFileUploadError::BadInput("Trim start must be less than trim end".to_string()));
|
||||
}
|
||||
Ok(())
|
||||
})
|
||||
.transpose()?;
|
||||
|
||||
let is_intermediate_system_file = form.is_intermediate_system_file
|
||||
.as_ref()
|
||||
.map(|b| b.0)
|
||||
.unwrap_or(false);
|
||||
|
||||
if is_intermediate_system_file && form.maybe_prompt_token.is_some() {
|
||||
return Err(MediaFileUploadError::BadInput(
|
||||
"Cannot set `is_intermediate_system_file` to true if `maybe_prompt_token` is provided."
|
||||
.to_string()));
|
||||
}
|
||||
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Map an API-key authentication failure onto this endpoint's error type. A genuine 401 stays a
|
||||
/// 401; anything else (e.g. a DB error during lookup) becomes a 500.
|
||||
fn map_api_key_auth_error(err: CommonWebError) -> MediaFileUploadError {
|
||||
match err {
|
||||
CommonWebError::NotAuthorized => {
|
||||
MediaFileUploadError::NotAuthorizedVerbose("invalid or missing API key".to_string())
|
||||
}
|
||||
_ => MediaFileUploadError::ServerError,
|
||||
}
|
||||
}
|
||||
@@ -7,6 +7,5 @@ pub mod middleware;
|
||||
pub mod requests;
|
||||
pub mod routes;
|
||||
pub mod session;
|
||||
pub mod user_lookup;
|
||||
pub mod validations;
|
||||
pub mod web_utils;
|
||||
-2
@@ -11,7 +11,6 @@ use crate::http_server::routes::application_routes::generate_routes::add_generat
|
||||
use crate::http_server::routes::application_routes::job_routes::add_job_routes;
|
||||
use crate::http_server::routes::application_routes::media_files_routes::add_media_file_routes;
|
||||
use crate::http_server::routes::application_routes::moderation_routes::add_moderator_routes;
|
||||
use crate::http_server::routes::application_routes::omni_api_routes::add_omni_api_routes;
|
||||
use crate::http_server::routes::application_routes::omni_gen_routes::add_omni_gen_routes;
|
||||
use crate::http_server::routes::application_routes::prompts_routes::add_prompts_routes;
|
||||
use crate::http_server::routes::application_routes::stripe_artcraft_routes::add_stripe_artcraft_routes;
|
||||
@@ -49,7 +48,6 @@ where
|
||||
// Artcraft surface area
|
||||
app = add_generate_routes(app); // /v1/generate/...
|
||||
app = add_omni_gen_routes(app); // /v1/omni_gen/...
|
||||
app = add_omni_api_routes(app); // /v1/omni_api/... (API-key authenticated generate)
|
||||
app = add_character_routes(app); // /v1/character/... and /v1/characters/...
|
||||
app = add_webhook_routes(app); // /v1/webhooks/... (fal)
|
||||
|
||||
|
||||
@@ -11,7 +11,6 @@ mod generate_routes;
|
||||
mod job_routes;
|
||||
mod media_files_routes;
|
||||
mod moderation_routes;
|
||||
mod omni_api_routes;
|
||||
mod omni_gen_routes;
|
||||
mod prompts_routes;
|
||||
mod stripe_artcraft_routes;
|
||||
|
||||
-53
@@ -1,53 +0,0 @@
|
||||
use actix_http::body::MessageBody;
|
||||
use actix_service::ServiceFactory;
|
||||
use actix_web::dev::{ServiceRequest, ServiceResponse};
|
||||
use actix_web::{web, App, Error, HttpResponse};
|
||||
|
||||
use crate::http_server::endpoints::omni_api::generate::image::omni_api_image_generate_handler::omni_api_image_generate_handler;
|
||||
use crate::http_server::endpoints::omni_api::generate::video::omni_api_video_generate_handler::omni_api_video_generate_handler;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_audio_media_file_handler::omni_upload_audio_media_file_handler;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_image_media_file_handler::omni_upload_image_media_file_handler;
|
||||
use crate::http_server::endpoints::omni_api::upload::omni_upload_video_media_file_handler::omni_upload_video_media_file_handler;
|
||||
|
||||
/// API-key authenticated generation endpoints. These mirror `/v1/omni_gen/generate/*` but read the
|
||||
/// caller's identity from the `Authorization` header API key instead of a session cookie. The
|
||||
/// `cost` and `models` endpoints are not duplicated here — they need no user and the existing
|
||||
/// `/v1/omni_gen/{cost,models}/*` routes suffice.
|
||||
pub fn add_omni_api_routes<T, B>(app: App<T>) -> App<T>
|
||||
where
|
||||
B: MessageBody,
|
||||
T: ServiceFactory<
|
||||
ServiceRequest,
|
||||
Config = (),
|
||||
Response = ServiceResponse<B>,
|
||||
Error = Error,
|
||||
InitError = (),
|
||||
>,
|
||||
{
|
||||
app.service(web::scope("/v1/omni_api")
|
||||
.service(web::scope("/generate")
|
||||
.service(web::resource("/video")
|
||||
.route(web::post().to(omni_api_video_generate_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok()))
|
||||
)
|
||||
.service(web::resource("/image")
|
||||
.route(web::post().to(omni_api_image_generate_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok()))
|
||||
)
|
||||
)
|
||||
.service(web::scope("/upload")
|
||||
.service(web::resource("/audio")
|
||||
.route(web::post().to(omni_upload_audio_media_file_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok()))
|
||||
)
|
||||
.service(web::resource("/image")
|
||||
.route(web::post().to(omni_upload_image_media_file_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok()))
|
||||
)
|
||||
.service(web::resource("/video")
|
||||
.route(web::post().to(omni_upload_video_media_file_handler))
|
||||
.route(web::head().to(|| HttpResponse::Ok()))
|
||||
)
|
||||
)
|
||||
)
|
||||
}
|
||||
-4
@@ -101,10 +101,6 @@ impl UserSessionFeatureFlags {
|
||||
pub fn can_use_referrals_program(&self) -> bool {
|
||||
self.feature_flags.contains(&UserFeatureFlag::ReferralsProgram)
|
||||
}
|
||||
|
||||
pub fn can_create_api_key(&self) -> bool {
|
||||
self.feature_flags.contains(&UserFeatureFlag::ApiKey)
|
||||
}
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
|
||||
@@ -1 +0,0 @@
|
||||
pub mod require_api_key_user;
|
||||
-83
@@ -1,83 +0,0 @@
|
||||
use std::marker::PhantomData;
|
||||
|
||||
use actix_web::HttpRequest;
|
||||
use log::{info, warn};
|
||||
use sqlx::MySqlConnection;
|
||||
|
||||
use mysql_queries::queries::api_keys::get_api_key_user_by_value::{
|
||||
get_api_key_user_by_value, GetApiKeyUserByValueArgs,
|
||||
};
|
||||
use tokens::tokens::api_keys::ApiKeyToken;
|
||||
use tokens::tokens::users::UserToken;
|
||||
|
||||
use crate::http_server::common_responses::common_web_error::CommonWebError;
|
||||
use crate::http_server::web_utils::get_authorization_header_api_key::get_authorization_header_api_key;
|
||||
|
||||
/// The authenticated identity behind an API-key request — the API-key analogue of a cookie
|
||||
/// `SessionUserRecord`, but deliberately minimal (just what the omni-api handlers need).
|
||||
///
|
||||
/// Unlike cookie sessions, this is **never** cached in Redis: every request performs a fresh MySQL
|
||||
/// lookup.
|
||||
pub struct UserApiSession {
|
||||
pub api_key_token: ApiKeyToken,
|
||||
pub user_token: UserToken,
|
||||
pub is_banned: bool,
|
||||
}
|
||||
|
||||
/// Authenticate a request using the API key in its `Authorization` header (see
|
||||
/// `get_authorization_header_api_key` for the accepted formats).
|
||||
///
|
||||
/// Returns `CommonWebError::NotAuthorized` (401) when the header is missing/unparseable, when the
|
||||
/// key does not match a live (non-soft-deleted) API key whose owning user still exists, or when
|
||||
/// that owner is banned. `mysql_executor` can be any sqlx executor — pass `&mut *connection` to
|
||||
/// reuse a connection the handler already holds.
|
||||
pub async fn require_api_key_user(
|
||||
http_request: &HttpRequest,
|
||||
mysql_connection: &mut MySqlConnection,
|
||||
) -> Result<UserApiSession, CommonWebError> {
|
||||
let api_key = match get_authorization_header_api_key(http_request) {
|
||||
Some(api_key) => api_key,
|
||||
None => {
|
||||
warn!("Missing or malformed Authorization header API key");
|
||||
return Err(CommonWebError::NotAuthorized);
|
||||
}
|
||||
};
|
||||
|
||||
let maybe_record = get_api_key_user_by_value(GetApiKeyUserByValueArgs {
|
||||
api_key: &api_key,
|
||||
mysql_executor: &mut *mysql_connection,
|
||||
phantom: PhantomData,
|
||||
})
|
||||
.await
|
||||
.map_err(|err| {
|
||||
warn!("API key user lookup error: {:?}", err);
|
||||
CommonWebError::from_error(err)
|
||||
})?;
|
||||
|
||||
// A missing or soft-deleted key (or a key whose owner no longer exists) is a 401, not a leak of
|
||||
// which case occurred.
|
||||
let record = match maybe_record {
|
||||
Some(record) => record,
|
||||
None => {
|
||||
warn!("No live API key user for presented key: {:?}", api_key);
|
||||
return Err(CommonWebError::NotAuthorized);
|
||||
}
|
||||
};
|
||||
|
||||
if record.is_banned {
|
||||
warn!("API key owner is banned: {:?}", record.user_token.as_str());
|
||||
return Err(CommonWebError::NotAuthorized);
|
||||
}
|
||||
|
||||
info!(
|
||||
"API key user authenticated: api_key_token={:?} user_token={:?}",
|
||||
record.api_key_token.as_str(),
|
||||
record.user_token.as_str(),
|
||||
);
|
||||
|
||||
Ok(UserApiSession {
|
||||
api_key_token: record.api_key_token,
|
||||
user_token: record.user_token,
|
||||
is_banned: record.is_banned,
|
||||
})
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
pub mod api_keys;
|
||||
@@ -16,7 +16,6 @@ import CreateImage from "../pages/create-image";
|
||||
import CreateVideo from "../pages/create-video";
|
||||
import CreateVFX from "../pages/create-vfx";
|
||||
import PageScene from "../pages/pagescene";
|
||||
import PageDraw from "../pages/pagedraw";
|
||||
import VideoEditorPage from "../pages/video-editor";
|
||||
import Pricing from "../pages/pricing";
|
||||
import Support from "../pages/support/support";
|
||||
@@ -150,7 +149,6 @@ export function App() {
|
||||
<Route path="/background-change" element={<CreateVFX />} />
|
||||
<Route path="/edit-3d" element={<PageScene />} />
|
||||
<Route path="/edit-3d/:sceneToken" element={<PageScene />} />
|
||||
<Route path="/edit-image" element={<PageDraw />} />
|
||||
<Route path="/video-editor" element={<VideoEditorPage />} />
|
||||
<Route
|
||||
path="/video-editor/:projectId"
|
||||
|
||||
@@ -75,7 +75,6 @@ export interface LightboxDetailsProps {
|
||||
onClose?: () => void;
|
||||
onRecreate?: () => void;
|
||||
onMakeVideo?: () => void;
|
||||
onEditOnCanvas?: () => void;
|
||||
onDelete?: () => void;
|
||||
showDownloadAppCta?: boolean;
|
||||
}
|
||||
@@ -91,7 +90,6 @@ export function LightboxDetails({
|
||||
onClose,
|
||||
onRecreate,
|
||||
onMakeVideo,
|
||||
onEditOnCanvas,
|
||||
onDelete,
|
||||
showDownloadAppCta,
|
||||
}: LightboxDetailsProps) {
|
||||
@@ -476,16 +474,6 @@ export function LightboxDetails({
|
||||
Make Video
|
||||
</Button>
|
||||
)}
|
||||
{onEditOnCanvas && (
|
||||
<Button
|
||||
icon={faPencil}
|
||||
className="w-full border border-ui-panel-border bg-ui-controls/40 hover:bg-ui-controls/60 text-white"
|
||||
variant="secondary"
|
||||
onClick={onEditOnCanvas}
|
||||
>
|
||||
Edit on Canvas
|
||||
</Button>
|
||||
)}
|
||||
{onRecreate && promptData.hasToken && (
|
||||
<Button
|
||||
icon={faArrowRotateRight}
|
||||
|
||||
@@ -23,7 +23,6 @@ import {
|
||||
type PromptData,
|
||||
} from "./shared";
|
||||
import { LightboxDetails } from "./LightboxDetails";
|
||||
import { applyEditOnCanvasFromImage } from "../../pages/pagedraw/edit-on-canvas";
|
||||
import {
|
||||
applyMakeVideoFromImage,
|
||||
applyRecreateFromMediaToken,
|
||||
@@ -335,13 +334,6 @@ export function Lightbox({
|
||||
applyMakeVideoFromImage(selectedMediaToken, selectedImageUrl, navigate);
|
||||
}, [selectedMediaToken, selectedImageUrl, canMakeVideo, navigate, onClose]);
|
||||
|
||||
const canEditOnCanvas = !isVideo && !is3D;
|
||||
const handleEditOnCanvas = useCallback(() => {
|
||||
if (!selectedMediaToken || !selectedImageUrl || !canEditOnCanvas) return;
|
||||
onClose();
|
||||
applyEditOnCanvasFromImage(selectedMediaToken, selectedImageUrl, navigate);
|
||||
}, [selectedMediaToken, selectedImageUrl, canEditOnCanvas, navigate, onClose]);
|
||||
|
||||
return (
|
||||
<>
|
||||
<Modal
|
||||
@@ -513,11 +505,6 @@ export function Lightbox({
|
||||
? handleMakeVideo
|
||||
: undefined
|
||||
}
|
||||
onEditOnCanvas={
|
||||
canEditOnCanvas && selectedMediaToken && selectedImageUrl
|
||||
? handleEditOnCanvas
|
||||
: undefined
|
||||
}
|
||||
onDelete={
|
||||
selectedMediaToken
|
||||
? () => setConfirmDeleteOpen(true)
|
||||
|
||||
@@ -63,9 +63,7 @@ export function ApiKeySection(_props: ApiKeySectionProps) {
|
||||
// The list includes soft-deleted keys; only show live ones.
|
||||
const live = fetched.filter((k) => k.maybe_deleted_at === null);
|
||||
setKeys((prev) => (reset ? live : [...prev, ...live]));
|
||||
setFetchedCount((prev) =>
|
||||
reset ? fetched.length : prev + fetched.length,
|
||||
);
|
||||
setFetchedCount((prev) => (reset ? fetched.length : prev + fetched.length));
|
||||
setHasMore(fetched.length === PAGE_SIZE);
|
||||
} else {
|
||||
toast.error(response.errorMessage ?? "Could not load API keys.");
|
||||
@@ -100,8 +98,8 @@ export function ApiKeySection(_props: ApiKeySectionProps) {
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="flex flex-col gap-3">
|
||||
<div className="flex items-start justify-between gap-3 mb-2">
|
||||
<div className="flex flex-col gap-6">
|
||||
<div className="flex items-start justify-between gap-3">
|
||||
<div className="flex flex-col gap-0.5">
|
||||
<p className="text-sm font-medium">API keys</p>
|
||||
<p className="text-xs opacity-70">
|
||||
@@ -135,6 +133,8 @@ export function ApiKeySection(_props: ApiKeySectionProps) {
|
||||
/>
|
||||
)}
|
||||
|
||||
<hr className="border-ui-panel-border" />
|
||||
|
||||
{loading ? (
|
||||
<div className="py-6 text-center text-xs opacity-60">
|
||||
<FontAwesomeIcon icon={faSpinnerThird} className="animate-spin" />
|
||||
@@ -299,20 +299,19 @@ function NewKeyReveal({
|
||||
</code>
|
||||
<Button
|
||||
type="button"
|
||||
variant="primary"
|
||||
className="h-8 px-3 shrink-0"
|
||||
variant="secondary"
|
||||
className="h-9 px-3 shrink-0"
|
||||
onClick={handleCopy}
|
||||
>
|
||||
<FontAwesomeIcon
|
||||
icon={copied ? faCheck : faCopy}
|
||||
className="mr-1.5"
|
||||
/>
|
||||
<FontAwesomeIcon icon={copied ? faCheck : faCopy} className="mr-1.5" />
|
||||
{copied ? "Copied" : "Copy"}
|
||||
</Button>
|
||||
</div>
|
||||
<div className="flex justify-end pt-0.5">
|
||||
<Button
|
||||
type="button"
|
||||
variant="secondary"
|
||||
className="h-8 px-3 shrink-0"
|
||||
className="h-8 px-3"
|
||||
onClick={onDismiss}
|
||||
>
|
||||
Done
|
||||
@@ -340,7 +339,7 @@ function ApiKeyRow({
|
||||
onRequestDelete: () => void;
|
||||
}) {
|
||||
return (
|
||||
<div className="flex flex-col gap-2 rounded-lg border border-white/10 bg-white/[0.03] p-3 transition-colors hover:border-white/[0.14]">
|
||||
<div className="flex flex-col gap-2 rounded-lg border border-ui-panel-border p-3">
|
||||
<div className="flex items-start justify-between gap-3">
|
||||
<div className="flex min-w-0 flex-col gap-0.5">
|
||||
<p className="truncate text-sm font-medium">{item.name}</p>
|
||||
@@ -350,10 +349,9 @@ function ApiKeyRow({
|
||||
</p>
|
||||
)}
|
||||
<div className="flex items-center gap-2 pt-1">
|
||||
{/* Truncated prefix, display-only — the full secret is only shown on
|
||||
create. The trailing ellipsis signals it's not the whole value. */}
|
||||
<code className="min-w-0 truncate rounded bg-black/40 px-2 py-1 font-mono text-[11px] text-white/70">
|
||||
{item.truncated_api_key}…
|
||||
{/* Truncated, display-only — the full secret is only shown on create. */}
|
||||
<code className="min-w-0 truncate rounded bg-black/30 px-2 py-1 font-mono text-[11px] text-white/70">
|
||||
{item.truncated_api_key}
|
||||
</code>
|
||||
<span className="text-[10px] uppercase tracking-wider text-white/40">
|
||||
{formatDate(item.created_at)}
|
||||
@@ -367,7 +365,6 @@ function ApiKeyRow({
|
||||
variant="secondary"
|
||||
className="h-8 px-3"
|
||||
onClick={onOpenEdit}
|
||||
title="Edit description"
|
||||
>
|
||||
Edit
|
||||
</Button>
|
||||
@@ -441,9 +438,6 @@ function EditDescriptionForm({
|
||||
|
||||
return (
|
||||
<form onSubmit={handleSubmit} className="flex flex-col gap-2 pt-1">
|
||||
<p className="text-xs opacity-70">
|
||||
Edit the description for "{item.name}". The key's name can't be changed.
|
||||
</p>
|
||||
<Input
|
||||
type="text"
|
||||
value={value}
|
||||
|
||||
@@ -167,7 +167,7 @@ function ApiKeysPanel() {
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="pb-4 pt-3">
|
||||
<div className="pt-3">
|
||||
<ApiKeySection user={user} />
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -6,7 +6,6 @@ import {
|
||||
faVideo,
|
||||
faImage,
|
||||
faCube,
|
||||
faPencil,
|
||||
faFilm,
|
||||
faWandMagicSparkles,
|
||||
faGraduationCap,
|
||||
@@ -54,7 +53,6 @@ const PRIMARY_ITEMS: NavItem[] = [{ label: "Home", href: "/", icon: faHouse }];
|
||||
const CREATE_ITEMS_STATIC: NavItem[] = [
|
||||
{ label: "Image", href: "/create-image", icon: faImage },
|
||||
{ label: "Video", href: "/create-video", icon: faVideo },
|
||||
{ label: "Edit Image", href: "/edit-image", icon: faPencil },
|
||||
{ label: "Edit 3D", href: "/edit-3d", icon: faCube },
|
||||
{ label: "Edit Video", href: "/video-editor", icon: faFilm, badge: "BETA" },
|
||||
{
|
||||
|
||||
@@ -1,223 +0,0 @@
|
||||
// The "no base image yet" state for the webapp's image editor.
|
||||
//
|
||||
// PageDraw renders this (via adapter.renderBaseImageSelector) until the user
|
||||
// picks a base image. It offers three entry points, mirroring the Tauri app's
|
||||
// BaseImageSelector but built from the webapp's own primitives:
|
||||
// - upload a local file → webapp uploadImage helper → media token
|
||||
// - pick from your library → GalleryModal in select mode
|
||||
// - start from a blank canvas → BlankCanvasModal (from the lib)
|
||||
|
||||
import { useState } from "react";
|
||||
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
|
||||
import {
|
||||
faExpand,
|
||||
faImages,
|
||||
faPencil,
|
||||
faSpinnerThird,
|
||||
} from "@fortawesome/pro-solid-svg-icons";
|
||||
import { UploaderStates } from "@storyteller/common";
|
||||
import { MediaFilesApi } from "@storyteller/api";
|
||||
import {
|
||||
GalleryModal,
|
||||
type GalleryItem,
|
||||
} from "@storyteller/ui-gallery-modal";
|
||||
import {
|
||||
BlankCanvasModal,
|
||||
type BaseSelectorImage,
|
||||
} from "@storyteller/ui-pagedraw";
|
||||
import { uploadImage } from "../../components/prompt-box/upload-image";
|
||||
import { showToast } from "../../components/toast/toast";
|
||||
|
||||
type BaseImageSelectorProps = {
|
||||
onImageSelect: (image: BaseSelectorImage) => void;
|
||||
showLoading?: boolean;
|
||||
};
|
||||
|
||||
export function BaseImageSelector({
|
||||
onImageSelect,
|
||||
showLoading = false,
|
||||
}: BaseImageSelectorProps) {
|
||||
const [isGalleryOpen, setIsGalleryOpen] = useState(false);
|
||||
const [gallerySelectedIds, setGallerySelectedIds] = useState<string[]>([]);
|
||||
const [isBlankCanvasOpen, setIsBlankCanvasOpen] = useState(false);
|
||||
const [isUploading, setIsUploading] = useState(false);
|
||||
const [isDragging, setIsDragging] = useState(false);
|
||||
|
||||
const busy = isUploading || showLoading;
|
||||
|
||||
const handleGallerySelect = (id: string) =>
|
||||
// Single-select: clicking always swaps to the latest pick.
|
||||
setGallerySelectedIds((prev) => (prev.includes(id) ? [] : [id]));
|
||||
|
||||
const handleUseGalleryImages = (items: GalleryItem[]) => {
|
||||
const item = items[0];
|
||||
if (!item) {
|
||||
showToast("error", "No image selected");
|
||||
return;
|
||||
}
|
||||
setIsGalleryOpen(false);
|
||||
setGallerySelectedIds([]);
|
||||
onImageSelect({
|
||||
mediaToken: item.id,
|
||||
url: item.fullImage || item.thumbnail || "",
|
||||
fullImageUrl: item.fullImage || undefined,
|
||||
thumbnailUrlTemplate: item.thumbnailUrlTemplate,
|
||||
});
|
||||
};
|
||||
|
||||
const handleBlankCanvasConfirm = (width: number, height: number) => {
|
||||
setIsBlankCanvasOpen(false);
|
||||
onImageSelect({
|
||||
url: "",
|
||||
mediaToken: `blank_canvas_${width}x${height}_${Math.random()
|
||||
.toString(36)
|
||||
.substring(2, 8)}`,
|
||||
isBlankCanvas: true,
|
||||
blankCanvasWidth: width,
|
||||
blankCanvasHeight: height,
|
||||
});
|
||||
};
|
||||
|
||||
const handleFile = (file: File) => {
|
||||
setIsUploading(true);
|
||||
const reader = new FileReader();
|
||||
reader.onloadend = async () => {
|
||||
const dataUrl = reader.result as string;
|
||||
await uploadImage({
|
||||
title: `edit-image-${Math.random().toString(36).substring(2, 15)}`,
|
||||
assetFile: file,
|
||||
progressCallback: (state) => {
|
||||
if (state.status === UploaderStates.success && state.data) {
|
||||
void resolveAndSelect(state.data, dataUrl);
|
||||
} else if (
|
||||
state.status === UploaderStates.assetError ||
|
||||
state.status === UploaderStates.imageCreateError
|
||||
) {
|
||||
showToast("error", "Upload failed. Please try again.");
|
||||
setIsUploading(false);
|
||||
}
|
||||
},
|
||||
});
|
||||
};
|
||||
reader.readAsDataURL(file);
|
||||
};
|
||||
|
||||
// The uploader hands back a media token immediately; fetch the media file to
|
||||
// swap the local data URL for a durable CDN URL (and grab the thumbnail
|
||||
// template) before handing the base image to the editor.
|
||||
const resolveAndSelect = async (mediaToken: string, fallbackUrl: string) => {
|
||||
let url = fallbackUrl;
|
||||
let thumbnailUrlTemplate: string | undefined;
|
||||
try {
|
||||
const result = await new MediaFilesApi().GetMediaFileByToken({
|
||||
mediaFileToken: mediaToken,
|
||||
});
|
||||
if (result.success && result.data) {
|
||||
url = result.data.media_links?.cdn_url || url;
|
||||
thumbnailUrlTemplate =
|
||||
result.data.media_links?.thumbnail_template ?? thumbnailUrlTemplate;
|
||||
}
|
||||
} catch {
|
||||
// Fall back to the data URL — the editor can still render it.
|
||||
}
|
||||
onImageSelect({ mediaToken, url, fullImageUrl: url, thumbnailUrlTemplate });
|
||||
setIsUploading(false);
|
||||
};
|
||||
|
||||
const onFileInput = (e: React.ChangeEvent<HTMLInputElement>) => {
|
||||
const file = e.target.files?.[0];
|
||||
if (file) handleFile(file);
|
||||
e.target.value = "";
|
||||
};
|
||||
|
||||
const onDrop = (e: React.DragEvent<HTMLLabelElement>) => {
|
||||
e.preventDefault();
|
||||
setIsDragging(false);
|
||||
const file = e.dataTransfer.files?.[0];
|
||||
if (file?.type.startsWith("image/")) handleFile(file);
|
||||
};
|
||||
|
||||
return (
|
||||
<>
|
||||
<label
|
||||
onDragOver={(e) => {
|
||||
e.preventDefault();
|
||||
setIsDragging(true);
|
||||
}}
|
||||
onDragLeave={() => setIsDragging(false)}
|
||||
onDrop={onDrop}
|
||||
className={`flex h-full w-full cursor-pointer flex-col items-center justify-center gap-4 rounded-2xl border-2 border-dashed p-8 text-center transition-colors ${
|
||||
isDragging
|
||||
? "border-blue-400 bg-blue-500/10"
|
||||
: "border-ui-panel-border bg-ui-background hover:border-blue-400/50"
|
||||
} ${busy ? "pointer-events-none opacity-60" : ""}`}
|
||||
>
|
||||
<input
|
||||
type="file"
|
||||
accept="image/*"
|
||||
className="hidden"
|
||||
onChange={onFileInput}
|
||||
disabled={busy}
|
||||
/>
|
||||
<div className="flex h-14 w-14 items-center justify-center rounded-2xl border border-blue-400/30 bg-blue-500/30 text-blue-300">
|
||||
<FontAwesomeIcon
|
||||
icon={busy ? faSpinnerThird : faPencil}
|
||||
className={`text-xl ${busy ? "animate-spin" : ""}`}
|
||||
/>
|
||||
</div>
|
||||
<div>
|
||||
<h2 className="text-lg font-semibold text-base-fg">Edit Image</h2>
|
||||
<p className="mt-1 text-sm text-base-fg/60">
|
||||
Click to upload or drag and drop an image here to edit
|
||||
</p>
|
||||
</div>
|
||||
<div className="mt-2 flex flex-wrap items-center justify-center gap-2">
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.preventDefault();
|
||||
setIsGalleryOpen(true);
|
||||
}}
|
||||
disabled={busy}
|
||||
className="flex items-center gap-2 rounded-lg border border-ui-controls-border bg-ui-controls px-3 py-2 text-sm font-medium text-base-fg transition-colors hover:bg-ui-controls/80"
|
||||
>
|
||||
<FontAwesomeIcon icon={faImages} />
|
||||
Pick from Library
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={(e) => {
|
||||
e.preventDefault();
|
||||
setIsBlankCanvasOpen(true);
|
||||
}}
|
||||
disabled={busy}
|
||||
className="flex items-center gap-2 rounded-lg border border-ui-controls-border bg-ui-controls px-3 py-2 text-sm font-medium text-base-fg transition-colors hover:bg-ui-controls/80"
|
||||
>
|
||||
<FontAwesomeIcon icon={faExpand} />
|
||||
Blank Canvas
|
||||
</button>
|
||||
</div>
|
||||
</label>
|
||||
|
||||
<GalleryModal
|
||||
mode="select"
|
||||
isOpen={isGalleryOpen}
|
||||
onClose={() => {
|
||||
setIsGalleryOpen(false);
|
||||
setGallerySelectedIds([]);
|
||||
}}
|
||||
selectedItemIds={gallerySelectedIds}
|
||||
onSelectItem={handleGallerySelect}
|
||||
maxSelections={1}
|
||||
onUseSelected={handleUseGalleryImages}
|
||||
forceFilter="image"
|
||||
/>
|
||||
|
||||
<BlankCanvasModal
|
||||
isOpen={isBlankCanvasOpen}
|
||||
onClose={() => setIsBlankCanvasOpen(false)}
|
||||
onConfirm={handleBlankCanvasConfirm}
|
||||
/>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -1,27 +0,0 @@
|
||||
// "Edit on Canvas" handoff: open an existing image in the PageDraw editor.
|
||||
//
|
||||
// Mirrors the Tauri app's handleEditFromGallery — RESET the editor, seed the
|
||||
// chosen image as the base (and into history), then route to /edit-image.
|
||||
// Seeding the lib's store before navigating means the editor mounts straight
|
||||
// into the image, same store-based handoff the create-image/create-video
|
||||
// "Recreate" and "Make Video" buttons use (see lib/recreate.ts).
|
||||
|
||||
import type { NavigateFunction } from "react-router-dom";
|
||||
import { useSceneStore } from "@storyteller/ui-pagedraw";
|
||||
|
||||
export function applyEditOnCanvasFromImage(
|
||||
mediaToken: string,
|
||||
mediaUrl: string,
|
||||
navigate: NavigateFunction,
|
||||
): void {
|
||||
const store = useSceneStore.getState();
|
||||
const baseImage = {
|
||||
url: mediaUrl,
|
||||
mediaToken,
|
||||
fullImageUrl: mediaUrl,
|
||||
};
|
||||
store.RESET();
|
||||
store.addHistoryImageBundle({ images: [baseImage] });
|
||||
store.setBaseImageInfo(baseImage);
|
||||
navigate("/edit-image");
|
||||
}
|
||||
@@ -1 +0,0 @@
|
||||
export { default } from "./pagedraw";
|
||||
@@ -1,141 +0,0 @@
|
||||
// Job polling for the PageDraw web adapter.
|
||||
//
|
||||
// Mirrors the create-image / create-vfx job loops: enqueue returns an
|
||||
// inference job token, then we poll `GET /v1/jobs/job/<token>` until the
|
||||
// job reaches a terminal state. On success we resolve the result media —
|
||||
// preferring the batch (so multi-image generations all land) and falling
|
||||
// back to the single media_links payload.
|
||||
//
|
||||
// Kept local to the pagedraw page (rather than imported from create-image)
|
||||
// so the editor owns its own result shape and isn't coupled to that page's
|
||||
// store types — same separation the artcraft-website pagedraw port used.
|
||||
|
||||
import { JobsApi, MediaFilesApi } from "@storyteller/api";
|
||||
|
||||
export interface PolledImage {
|
||||
media_token: string;
|
||||
cdn_url: string;
|
||||
maybe_thumbnail_template?: string;
|
||||
}
|
||||
|
||||
export type PollResult =
|
||||
| { status: "pending"; images: PolledImage[] }
|
||||
| { status: "complete"; images: PolledImage[] }
|
||||
| { status: "failed"; images: PolledImage[]; error: string };
|
||||
|
||||
export async function pollJobOnce(jobToken: string): Promise<PollResult> {
|
||||
const jobsApi = new JobsApi();
|
||||
const response = await jobsApi.GetJobByToken({ token: jobToken });
|
||||
|
||||
if (!response.success || !response.data) {
|
||||
return { status: "pending", images: [] };
|
||||
}
|
||||
|
||||
const state = response.data;
|
||||
const statusStr = state.status?.status?.toLowerCase() ?? "";
|
||||
|
||||
if (statusStr === "complete_success" || statusStr === "complete") {
|
||||
const result = state.maybe_result as Record<string, unknown> | undefined;
|
||||
const mediaLinks = (result as any)?.media_links;
|
||||
const entityToken = (result as any)?.entity_token as string | undefined;
|
||||
|
||||
if (entityToken) {
|
||||
try {
|
||||
const mediaApi = new MediaFilesApi();
|
||||
const mediaFile = await mediaApi.GetMediaFileByToken({
|
||||
mediaFileToken: entityToken,
|
||||
});
|
||||
const batchToken = mediaFile.data?.maybe_batch_token;
|
||||
if (batchToken) {
|
||||
const batchResponse = await mediaApi.GetMediaFilesByBatchToken({
|
||||
batchToken,
|
||||
});
|
||||
if (batchResponse.success && batchResponse.data?.length) {
|
||||
const images: PolledImage[] = batchResponse.data
|
||||
.map((file: any) => ({
|
||||
media_token: file.token,
|
||||
cdn_url: file.media_links?.cdn_url,
|
||||
maybe_thumbnail_template:
|
||||
file.media_links?.maybe_thumbnail_template,
|
||||
}))
|
||||
.filter((img: PolledImage) => img.cdn_url);
|
||||
if (images.length > 0) return { status: "complete", images };
|
||||
}
|
||||
}
|
||||
} catch {
|
||||
// Fall through to single-image extraction.
|
||||
}
|
||||
}
|
||||
|
||||
if (mediaLinks?.cdn_url) {
|
||||
return {
|
||||
status: "complete",
|
||||
images: [
|
||||
{
|
||||
media_token: entityToken ?? jobToken,
|
||||
cdn_url: mediaLinks.cdn_url,
|
||||
maybe_thumbnail_template: mediaLinks.maybe_thumbnail_template,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
return { status: "complete", images: [] };
|
||||
}
|
||||
|
||||
if (
|
||||
statusStr.includes("fail") ||
|
||||
statusStr.includes("error") ||
|
||||
statusStr === "dead"
|
||||
) {
|
||||
return {
|
||||
status: "failed",
|
||||
images: [],
|
||||
error:
|
||||
state.status?.maybe_failure_message ??
|
||||
state.status?.maybe_extra_status_description ??
|
||||
"Generation failed",
|
||||
};
|
||||
}
|
||||
|
||||
return { status: "pending", images: [] };
|
||||
}
|
||||
|
||||
export function startPolling(
|
||||
jobToken: string,
|
||||
onComplete: (images: PolledImage[]) => void,
|
||||
onError: (reason: string) => void,
|
||||
intervalMs = 3000,
|
||||
maxAttempts = 120,
|
||||
): () => void {
|
||||
let attempts = 0;
|
||||
let stopped = false;
|
||||
|
||||
const poll = async () => {
|
||||
if (stopped) return;
|
||||
attempts++;
|
||||
|
||||
try {
|
||||
const result = await pollJobOnce(jobToken);
|
||||
if (stopped) return;
|
||||
|
||||
if (result.status === "complete") return onComplete(result.images);
|
||||
if (result.status === "failed") return onError(result.error);
|
||||
if (attempts >= maxAttempts) return onError("Generation timed out");
|
||||
|
||||
setTimeout(poll, intervalMs);
|
||||
} catch {
|
||||
if (!stopped && attempts < maxAttempts) {
|
||||
setTimeout(poll, intervalMs * 2);
|
||||
} else if (!stopped) {
|
||||
onError("Network error during polling");
|
||||
}
|
||||
}
|
||||
};
|
||||
|
||||
setTimeout(poll, 2000);
|
||||
|
||||
return () => {
|
||||
stopped = true;
|
||||
};
|
||||
}
|
||||
@@ -1,25 +0,0 @@
|
||||
import { PageDraw as PageDrawLib } from "@storyteller/ui-pagedraw";
|
||||
import Seo from "../../components/seo";
|
||||
import { useWebPageDrawAdapter } from "./web-adapter";
|
||||
|
||||
export default function PageDraw() {
|
||||
const adapter = useWebPageDrawAdapter();
|
||||
return (
|
||||
<>
|
||||
<Seo
|
||||
title="Edit Image - ArtCraft"
|
||||
description="Edit and recompose images on an AI canvas, then generate new variations."
|
||||
/>
|
||||
{/* PageDraw lays its canvas and toolbars out with position:fixed.
|
||||
translateZ(0) makes this div their containing block so they scope
|
||||
to the SidebarInset area instead of covering the whole window —
|
||||
same technique the Edit 3D page uses for its lib overlays. */}
|
||||
<div
|
||||
className="relative h-full w-full overflow-hidden"
|
||||
style={{ transform: "translateZ(0)" }}
|
||||
>
|
||||
<PageDrawLib adapter={adapter} />
|
||||
</div>
|
||||
</>
|
||||
);
|
||||
}
|
||||
@@ -1,275 +0,0 @@
|
||||
// PageDrawAdapter implementation for the artcraft webapp.
|
||||
//
|
||||
// Plugs the shared @storyteller/ui-pagedraw lib into the webapp by speaking
|
||||
// REST instead of Tauri commands:
|
||||
// - enqueueEditImage → OmniGenApi.generateImage, then poll the job and feed
|
||||
// results back into the lib's Zustand store (addHistoryImageBundle /
|
||||
// resolvePendingGeneration), keyed on frontendSubscriberId.
|
||||
// - enqueueBgRemoval → upload base64 → media token → POST
|
||||
// /v1/generate/image/remove_background → poll → finishRemoveBackground.
|
||||
// - uploadImage → UploadImageMedia (shared REST uploader).
|
||||
// - renderBaseImageSelector → the webapp's own upload/library/blank picker.
|
||||
//
|
||||
// Inpaint is intentionally stubbed: the omni-gen REST endpoint doesn't yet
|
||||
// accept a raw mask payload, so we surface a clear toast until that path
|
||||
// exists. Mirrors the artcraft-website pagedraw port.
|
||||
|
||||
import { useMemo } from "react";
|
||||
import {
|
||||
type PageDrawAdapter,
|
||||
type ImageBundle,
|
||||
type BaseSelectorImage,
|
||||
useSceneStore,
|
||||
} from "@storyteller/ui-pagedraw";
|
||||
import {
|
||||
OmniGenApi,
|
||||
StorytellerApiHostStore,
|
||||
UploadImageMedia,
|
||||
type OmniGenImageRequest,
|
||||
} from "@storyteller/api";
|
||||
import { UploaderStates } from "@storyteller/common";
|
||||
import { showToast } from "../../components/toast/toast";
|
||||
import { useInsufficientCredits } from "../../components/insufficient-credits-modal";
|
||||
import { BaseImageSelector } from "./base-image-selector";
|
||||
import { startPolling, type PolledImage } from "./job-polling";
|
||||
|
||||
// ─── Local bg-removal call ──────────────────────────────────────────────────
|
||||
// Inlined (no dedicated method in @storyteller/api yet). Mirrors ApiManager's
|
||||
// auth: cookies via credentials:"include", plus the signed-session header
|
||||
// fallback for browsers that block third-party cookies.
|
||||
|
||||
const SESSION_STORAGE_KEY = "artcraft_signed_session";
|
||||
|
||||
interface RemoveBgRequest {
|
||||
uuid_idempotency_token: string;
|
||||
media_file_token: string;
|
||||
}
|
||||
|
||||
interface RemoveBgResponse {
|
||||
success: boolean;
|
||||
inference_job_token: string;
|
||||
}
|
||||
|
||||
const sessionHeaders = (): Record<string, string> => {
|
||||
const headers: Record<string, string> = {
|
||||
Accept: "application/json",
|
||||
"Content-Type": "application/json",
|
||||
};
|
||||
try {
|
||||
const session = localStorage.getItem(SESSION_STORAGE_KEY);
|
||||
if (session) headers.session = session;
|
||||
} catch {
|
||||
// localStorage unavailable — rely on the cookie.
|
||||
}
|
||||
return headers;
|
||||
};
|
||||
|
||||
const removeImageBackground = async (
|
||||
body: RemoveBgRequest,
|
||||
): Promise<RemoveBgResponse> => {
|
||||
const host = StorytellerApiHostStore.getInstance().getApiSchemeAndHost();
|
||||
const response = await fetch(`${host}/v1/generate/image/remove_background`, {
|
||||
method: "POST",
|
||||
headers: sessionHeaders(),
|
||||
credentials: "include",
|
||||
body: JSON.stringify(body),
|
||||
});
|
||||
if (!response.ok) {
|
||||
throw new Error(`Background removal HTTP ${response.status}`);
|
||||
}
|
||||
return response.json();
|
||||
};
|
||||
|
||||
// ─── Helpers ─────────────────────────────────────────────────────────────────
|
||||
|
||||
const polledImagesToBundle = (images: PolledImage[]): ImageBundle => ({
|
||||
images: images.map(
|
||||
(img) =>
|
||||
({
|
||||
url: img.cdn_url,
|
||||
mediaToken: img.media_token,
|
||||
thumbnailUrlTemplate: img.maybe_thumbnail_template,
|
||||
fullImageUrl: img.cdn_url,
|
||||
}) as BaseSelectorImage,
|
||||
),
|
||||
});
|
||||
|
||||
const startGenerationPoll = (jobToken: string, subscriberId: string) => {
|
||||
startPolling(
|
||||
jobToken,
|
||||
(images) => {
|
||||
const store = useSceneStore.getState();
|
||||
// The user may have dismissed the placeholder while the job ran.
|
||||
if (!store.pendingGenerations.some((p) => p.id === subscriberId)) return;
|
||||
store.addHistoryImageBundle(polledImagesToBundle(images));
|
||||
store.resolvePendingGeneration(subscriberId);
|
||||
},
|
||||
(reason) => {
|
||||
showToast("error", reason);
|
||||
useSceneStore.getState().resolvePendingGeneration(subscriberId);
|
||||
},
|
||||
);
|
||||
};
|
||||
|
||||
const base64ToBlob = (base64: string): Blob => {
|
||||
const commaIdx = base64.indexOf(",");
|
||||
const raw = commaIdx >= 0 ? base64.slice(commaIdx + 1) : base64;
|
||||
const headerMatch =
|
||||
commaIdx >= 0 ? base64.slice(0, commaIdx).match(/data:(.*?);base64/) : null;
|
||||
const mime = headerMatch?.[1] ?? "image/png";
|
||||
const binary = atob(raw);
|
||||
const bytes = new Uint8Array(binary.length);
|
||||
for (let i = 0; i < binary.length; i++) bytes[i] = binary.charCodeAt(i);
|
||||
return new Blob([bytes], { type: mime });
|
||||
};
|
||||
|
||||
const uploadBlobAsImage = async (
|
||||
blob: Blob,
|
||||
title: string,
|
||||
): Promise<string | undefined> => {
|
||||
let mediaToken: string | undefined;
|
||||
const file = new File([blob], `${title}.png`, { type: blob.type });
|
||||
await UploadImageMedia({
|
||||
title,
|
||||
assetFile: file,
|
||||
progressCallback: (state) => {
|
||||
if (state.status === UploaderStates.success && state.data) {
|
||||
mediaToken = state.data;
|
||||
}
|
||||
},
|
||||
});
|
||||
return mediaToken;
|
||||
};
|
||||
|
||||
// PageDrawEditRequest.model is an ImageModel object (or possibly a raw id);
|
||||
// the omni-gen REST body wants the string id.
|
||||
const modelIdOf = (model: unknown): string => {
|
||||
if (!model) return "";
|
||||
if (typeof model === "string") return model;
|
||||
const m = model as { id?: string; tauriId?: string };
|
||||
return m.id ?? m.tauriId ?? String(model);
|
||||
};
|
||||
|
||||
// ApiManager throws Error("HTTP error! status: 402") on a non-2xx response, and
|
||||
// our inlined bg-removal fetch throws "Background removal HTTP 402"; the JSON
|
||||
// body isn't surfaced either way. Pull the 3-digit status out of the message so
|
||||
// callers can special-case 402 Payment Required.
|
||||
const httpStatusFromError = (e: unknown): number | undefined => {
|
||||
const message = e instanceof Error ? e.message : String(e);
|
||||
const match = /(\d{3})/.exec(message);
|
||||
return match ? Number(match[1]) : undefined;
|
||||
};
|
||||
|
||||
// ─── Adapter ───────────────────────────────────────────────────────────────────
|
||||
|
||||
export const useWebPageDrawAdapter = (): PageDrawAdapter => {
|
||||
const openInsufficientCredits = useInsufficientCredits();
|
||||
|
||||
return useMemo<PageDrawAdapter>(
|
||||
() => ({
|
||||
enqueueEditImage: async (req) => {
|
||||
const refTokens = [
|
||||
...(req.canvasImageMediaToken ? [req.canvasImageMediaToken] : []),
|
||||
...(req.imageMediaTokens ?? []),
|
||||
].filter((t): t is string => !!t);
|
||||
|
||||
const body: OmniGenImageRequest = {
|
||||
model: modelIdOf(req.model),
|
||||
prompt: req.prompt ?? null,
|
||||
idempotency_token: crypto.randomUUID(),
|
||||
aspect_ratio: req.aspectRatio ?? null,
|
||||
resolution: req.imageResolution ?? null,
|
||||
image_batch_count: req.imageCount ?? 1,
|
||||
image_media_tokens: refTokens.length ? refTokens : null,
|
||||
};
|
||||
|
||||
try {
|
||||
const response = await new OmniGenApi().generateImage(body);
|
||||
if (
|
||||
response.success &&
|
||||
response.inference_job_token &&
|
||||
req.frontendSubscriberId
|
||||
) {
|
||||
startGenerationPoll(
|
||||
response.inference_job_token,
|
||||
req.frontendSubscriberId,
|
||||
);
|
||||
return { status: "success" };
|
||||
}
|
||||
showToast("error", "Generation failed to enqueue");
|
||||
return { status: "fail" };
|
||||
} catch (e: any) {
|
||||
// 402 Payment Required → the user is out of credits. Open the
|
||||
// upgrade/credits modal instead of a raw error toast, matching the
|
||||
// create-image / create-video generate handlers.
|
||||
if (httpStatusFromError(e) === 402) {
|
||||
openInsufficientCredits();
|
||||
return { status: "fail" };
|
||||
}
|
||||
showToast("error", `Generation error: ${e?.message ?? e}`);
|
||||
return { status: "fail" };
|
||||
}
|
||||
},
|
||||
|
||||
enqueueInpaint: async (req) => {
|
||||
// The omni-gen REST endpoint doesn't yet accept a raw mask payload;
|
||||
// surface clearly until a multipart/mask-upload path exists.
|
||||
if (!req.maskImageRawBytes) {
|
||||
showToast("error", "Inpaint mask missing");
|
||||
return { status: "fail" };
|
||||
}
|
||||
showToast("error", "Inpainting is not yet available on the web");
|
||||
return { status: "fail" };
|
||||
},
|
||||
|
||||
enqueueBgRemoval: async (base64Image, nodeId) => {
|
||||
try {
|
||||
const blob = base64ToBlob(base64Image);
|
||||
const mediaToken = await uploadBlobAsImage(
|
||||
blob,
|
||||
`bg-removal-${nodeId}`,
|
||||
);
|
||||
if (!mediaToken) {
|
||||
showToast("error", "Background removal upload failed");
|
||||
return;
|
||||
}
|
||||
const response = await removeImageBackground({
|
||||
uuid_idempotency_token: crypto.randomUUID(),
|
||||
media_file_token: mediaToken,
|
||||
});
|
||||
if (!response.success || !response.inference_job_token) {
|
||||
showToast("error", "Background removal failed to enqueue");
|
||||
return;
|
||||
}
|
||||
startPolling(
|
||||
response.inference_job_token,
|
||||
async (images) => {
|
||||
const out = images[0];
|
||||
if (!out) return;
|
||||
await useSceneStore
|
||||
.getState()
|
||||
.finishRemoveBackground(nodeId, out.media_token, out.cdn_url);
|
||||
},
|
||||
(reason) => showToast("error", reason),
|
||||
);
|
||||
} catch (e: any) {
|
||||
if (httpStatusFromError(e) === 402) {
|
||||
openInsufficientCredits();
|
||||
return;
|
||||
}
|
||||
showToast("error", `Background removal error: ${e?.message ?? e}`);
|
||||
}
|
||||
},
|
||||
|
||||
uploadImage: UploadImageMedia,
|
||||
|
||||
renderBaseImageSelector: ({ onImageSelect, showLoading }) => (
|
||||
<BaseImageSelector
|
||||
onImageSelect={onImageSelect}
|
||||
showLoading={showLoading}
|
||||
/>
|
||||
),
|
||||
}),
|
||||
[openInsufficientCredits],
|
||||
);
|
||||
};
|
||||
Reference in New Issue
Block a user