mirror of
https://github.com/storytold/artcraft.git
synced 2026-10-20 17:02:53 -04:00
artcraft-native: Create Audio page, track player and waveforms
Adds the Create Audio page (PromptBoxAudio): OmniGen audio models, style, instrumental, keep lyrics, loop, Beat & Key, Tuning, audio or image references with the 600 s cap, cost, and the audio feed. Audio cards and the lightbox play in place with a waveform, seeking and volume. The prompt box gains slots for rows above and below the prompt.
This commit is contained in:
@@ -7,8 +7,8 @@ The look follows the brutalist webapp (`artcraft-services/frontend/apps/artcraft
|
|||||||
means near-black surfaces with white/15 hairlines and 3 px corners on controls. Type is Archivo for
|
means near-black surfaces with white/15 hairlines and 3 px corners on controls. Type is Archivo for
|
||||||
display, Inter for body text and Geist Mono for labels. The brand blue marks the primary actions.
|
display, Inter for body text and Geist Mono for labels. The brand blue marks the primary actions.
|
||||||
|
|
||||||
Create Image and Create Video are fully built. The other tools show a "coming soon" page that
|
Create Image, Create Video and Create Audio are fully built. The other tools show a "coming soon"
|
||||||
links to the same tool in the web app.
|
page that links to the same tool in the web app.
|
||||||
|
|
||||||
## Running
|
## Running
|
||||||
|
|
||||||
@@ -32,9 +32,10 @@ The workspace's default member is the Tauri app, so pass `-p artcraft_native`.
|
|||||||
| `backend::wire` | Lenient response shapes; new server enum values can't break decoding |
|
| `backend::wire` | Lenient response shapes; new server enum values can't break decoding |
|
||||||
| `backend::session` | Credentials: own file, Tauri cookie import, env config |
|
| `backend::session` | Credentials: own file, Tauri cookie import, env config |
|
||||||
| `backend::media_cache` | Thumbnails and previews: downloaded and decoded off-thread, evicted when unused |
|
| `backend::media_cache` | Thumbnails and previews: downloaded and decoded off-thread, evicted when unused |
|
||||||
|
| `backend::audio` | One clip at a time: reference previews, feed tracks (pause, seek, volume), waveforms |
|
||||||
| `models` | Model catalog from OmniGen listings, plus names, families and limits |
|
| `models` | Model catalog from OmniGen listings, plus names, families and limits |
|
||||||
| `prompt_box` | The shared prompt box: editor, reference deck, keyframes, pickers, model selector |
|
| `prompt_box` | The shared prompt box: editor, reference deck, keyframes, pickers, model selector |
|
||||||
| `pages` | Create Image, Create Video, Home, Library, and the shared page shell |
|
| `pages` | Create Image, Video and Audio, Home, Library, and the shared page shell |
|
||||||
| `feed` | Generation feed: running, failed and finished jobs, library paging, grid and list |
|
| `feed` | Generation feed: running, failed and finished jobs, library paging, grid and list |
|
||||||
| `overlays` | Lightbox, library picker, sign-in and settings dialogs |
|
| `overlays` | Lightbox, library picker, sign-in and settings dialogs |
|
||||||
| `shell` | Custom title bar, sidebar flush to the window edge, content panel header |
|
| `shell` | Custom title bar, sidebar flush to the window edge, content panel header |
|
||||||
@@ -42,12 +43,13 @@ The workspace's default member is the Tauri app, so pass `-p artcraft_native`.
|
|||||||
|
|
||||||
How the backend works:
|
How the backend works:
|
||||||
|
|
||||||
- **Generation.** It goes through OmniGen: `/v1/omni_gen/{models,cost,generate}/{image,video}`.
|
- **Generation.** It goes through OmniGen: `/v1/omni_gen/{models,cost,generate}/{image,video,audio}`.
|
||||||
Model capabilities come from the server, so new models and options appear without an app update.
|
Model capabilities come from the server, so new models and options appear without an app update.
|
||||||
- **Typed client functions.** Model lists, uploads, credits, login and deletes use
|
- **Typed client functions.** Model lists, uploads, credits, login and deletes use
|
||||||
`artcraft_client`.
|
`artcraft_client`.
|
||||||
- **Lenient decoding.** Generate and cost requests are JSON maps, as in the Tauri app. Job polling
|
- **Lenient decoding.** Generate and cost requests are JSON maps, as in the Tauri app. Job polling
|
||||||
(`/v1/jobs/session`), the library list, batches and prompts are decoded with lenient structs.
|
(`/v1/jobs/session`), the audio model list, the library list, batches and prompts are decoded
|
||||||
|
with lenient structs.
|
||||||
- **Identity headers.** Every request sends the desktop identity headers: `Origin` and
|
- **Identity headers.** Every request sends the desktop identity headers: `Origin` and
|
||||||
`User-Agent`.
|
`User-Agent`.
|
||||||
|
|
||||||
@@ -74,19 +76,24 @@ How the backend works:
|
|||||||
| In-app video playback | Done with FFmpeg on the PATH; otherwise opens in the system player |
|
| In-app video playback | Done with FFmpeg on the PATH; otherwise opens in the system player |
|
||||||
| Audio reference playback (click to play/stop) | Done |
|
| Audio reference playback (click to play/stop) | Done |
|
||||||
| Animated video previews in the feed (with a stills toggle) | Done |
|
| Animated video previews in the feed (with a stills toggle) | Done |
|
||||||
|
| Audio: style, instrumental, keep lyrics, loop | Done |
|
||||||
|
| Audio: Beat & Key (BPM, musical key) and Tuning (sample rate, speed, volume, pitch) | Done |
|
||||||
|
| Audio references (play, 600 s cap) or one image, never both | Done |
|
||||||
|
|
||||||
Feed and lightbox:
|
Feed and lightbox:
|
||||||
|
|
||||||
- **Feed.** Pending cards show progress, time left and a batch banner. Failed cards show the reason
|
- **Feed.** Pending cards show progress, time left and a batch banner. Failed cards show the reason
|
||||||
and can be dismissed. The grid is masonry, with a list view alongside. Hover actions are Recreate,
|
and can be dismissed. The grid is masonry, with a list view alongside. Hover actions are Recreate,
|
||||||
Make Video, Share and Download. Select mode does batch downloads. The last viewed item is marked.
|
Make Video, Share and Download. Select mode does batch downloads. The last viewed item is marked.
|
||||||
|
- **Audio.** Audio cards play in place: a waveform (click to seek), the time and a volume flyout.
|
||||||
|
One track plays at a time and stops when you leave the page. Audio offers Share and Download.
|
||||||
- **Lightbox.** Prev/next with the arrow keys, prompt copy and details. Actions are Recreate, Make
|
- **Lightbox.** Prev/next with the arrow keys, prompt copy and details. Actions are Recreate, Make
|
||||||
Video, Download, Share and Delete.
|
Video, Download, Share and Delete. Audio plays in a panel with the full-size player.
|
||||||
|
|
||||||
## Diagnostics
|
## Diagnostics
|
||||||
|
|
||||||
Set `ARTCRAFT_SCREENSHOT=<file.png>` to save a screenshot once the window has settled.
|
Set `ARTCRAFT_SCREENSHOT=<file.png>` to save a screenshot once the window has settled.
|
||||||
`ARTCRAFT_PAGE` (`home`, `create-image`, `create-video` or `library`) picks the page, and
|
`ARTCRAFT_PAGE` (`home`, `create-image`, `create-video`, `create-audio` or `library`) picks the page, and
|
||||||
`ARTCRAFT_SCREENSHOT_EXIT=1` closes the app afterwards.
|
`ARTCRAFT_SCREENSHOT_EXIT=1` closes the app afterwards.
|
||||||
|
|
||||||
## Tests
|
## Tests
|
||||||
@@ -96,8 +103,8 @@ cargo test -p artcraft_native
|
|||||||
```
|
```
|
||||||
|
|
||||||
There are unit tests for the catalog, feed store, wire decoding and backend helpers. Headless UI
|
There are unit tests for the catalog, feed store, wire decoding and backend helpers. Headless UI
|
||||||
tests (`egui_kittest`) drive the prompt box: generate gating, Enter to generate, the pickers,
|
tests (`egui_kittest`) drive the prompt box (generate gating, Enter to generate, the pickers, `@`
|
||||||
`@` mentions and clear-all.
|
mentions, clear-all), the Characters dialog and the audio toolbar (toggles, Beat & Key, Tuning).
|
||||||
|
|
||||||
## Fonts and brand assets
|
## Fonts and brand assets
|
||||||
|
|
||||||
|
|||||||
+145
-99
@@ -21,10 +21,11 @@ use crate::overlays::characters::{CharactersAction, CharactersModal};
|
|||||||
use crate::overlays::library_picker::{LibraryPicker, PickerAction};
|
use crate::overlays::library_picker::{LibraryPicker, PickerAction};
|
||||||
use crate::overlays::lightbox::{Lightbox, LightboxAction, fit_inside};
|
use crate::overlays::lightbox::{Lightbox, LightboxAction, fit_inside};
|
||||||
use crate::pages::common::{self, AppRequest, Env, FeedKey};
|
use crate::pages::common::{self, AppRequest, Env, FeedKey};
|
||||||
|
use crate::pages::create_audio::{AudioSettings, CreateAudioPage};
|
||||||
use crate::pages::create_image::{CreateImagePage, ImageSettings};
|
use crate::pages::create_image::{CreateImagePage, ImageSettings};
|
||||||
use crate::pages::create_video::{CreateVideoPage, VideoSettings};
|
use crate::pages::create_video::{CreateVideoPage, VideoSettings};
|
||||||
use crate::pages::other;
|
use crate::pages::other;
|
||||||
use crate::prompt_box::refs::{RefKind, RefMedia};
|
use crate::prompt_box::refs::{RefKind, RefMedia, References};
|
||||||
use crate::shell::{self, Account, FeedToggles, Page, ShellAction};
|
use crate::shell::{self, Account, FeedToggles, Page, ShellAction};
|
||||||
use crate::theme;
|
use crate::theme;
|
||||||
use crate::ui::toast::Toasts;
|
use crate::ui::toast::Toasts;
|
||||||
@@ -34,7 +35,7 @@ use crate::ui::{creator_icons, widgets, window};
|
|||||||
/// borrowed mutably alongside it).
|
/// borrowed mutably alongside it).
|
||||||
macro_rules! env {
|
macro_rules! env {
|
||||||
($app:ident, $ctx:expr) => {
|
($app:ident, $ctx:expr) => {
|
||||||
Env { ctx: $ctx.clone(), backend: &$app.backend, cache: &mut $app.cache, toasts: &mut $app.toasts, catalog: &$app.catalog, signed_in: $app.user.is_some(), enter_to_generate: $app.enter_to_generate, view_mode: $app.view_mode, autoplay: $app.autoplay, ratios: &mut $app.ratios, prompts: &$app.prompts, requests: &mut $app.requests, characters: &$app.characters }
|
Env { ctx: $ctx.clone(), backend: &$app.backend, cache: &mut $app.cache, audio: &mut $app.player, toasts: &mut $app.toasts, catalog: &$app.catalog, signed_in: $app.user.is_some(), enter_to_generate: $app.enter_to_generate, view_mode: $app.view_mode, autoplay: $app.autoplay, ratios: &mut $app.ratios, prompts: &$app.prompts, requests: &mut $app.requests, characters: &$app.characters }
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -44,6 +45,8 @@ const CREDITS_INTERVAL: f64 = 60.0;
|
|||||||
const MODEL_RETRY_INTERVAL: f64 = 15.0;
|
const MODEL_RETRY_INTERVAL: f64 = 15.0;
|
||||||
/// List view: prompt records fetched per frame at most.
|
/// List view: prompt records fetched per frame at most.
|
||||||
const PROMPT_FETCHES_PER_FRAME: usize = 8;
|
const PROMPT_FETCHES_PER_FRAME: usize = 8;
|
||||||
|
/// Every create page's model listing.
|
||||||
|
const MODALITIES: [Modality; 3] = [Modality::Image, Modality::Video, Modality::Audio];
|
||||||
|
|
||||||
/// Preferences that survive restarts.
|
/// Preferences that survive restarts.
|
||||||
#[derive(Clone, Debug, Serialize, Deserialize)]
|
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||||
@@ -56,18 +59,19 @@ struct Saved {
|
|||||||
sidebar_open: bool,
|
sidebar_open: bool,
|
||||||
image: ImageSettings,
|
image: ImageSettings,
|
||||||
video: VideoSettings,
|
video: VideoSettings,
|
||||||
|
audio: AudioSettings,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Default for Saved {
|
impl Default for Saved {
|
||||||
fn default() -> Self {
|
fn default() -> Self {
|
||||||
Self { page: Page::CreateImage, enter_to_generate: false, view_mode: ViewMode::Grid, autoplay: true, sidebar_open: true, image: ImageSettings { count: 1, ..Default::default() }, video: VideoSettings::default() }
|
Self { page: Page::CreateImage, enter_to_generate: false, view_mode: ViewMode::Grid, autoplay: true, sidebar_open: true, image: ImageSettings { count: 1, ..Default::default() }, video: VideoSettings::default(), audio: AudioSettings::default() }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub struct ArtcraftApp {
|
pub struct ArtcraftApp {
|
||||||
backend: Backend,
|
backend: Backend,
|
||||||
cache: MediaCache,
|
cache: MediaCache,
|
||||||
audio: AudioPlayer,
|
player: AudioPlayer,
|
||||||
toasts: Toasts,
|
toasts: Toasts,
|
||||||
catalog: Catalog,
|
catalog: Catalog,
|
||||||
user: Option<SessionUser>,
|
user: Option<SessionUser>,
|
||||||
@@ -79,6 +83,7 @@ pub struct ArtcraftApp {
|
|||||||
sidebar_open: bool,
|
sidebar_open: bool,
|
||||||
image: CreateImagePage,
|
image: CreateImagePage,
|
||||||
video: CreateVideoPage,
|
video: CreateVideoPage,
|
||||||
|
audio: CreateAudioPage,
|
||||||
library: FeedStore,
|
library: FeedStore,
|
||||||
ratios: RatioCache,
|
ratios: RatioCache,
|
||||||
/// Prompt texts by prompt token (list view, lightbox).
|
/// Prompt texts by prompt token (list view, lightbox).
|
||||||
@@ -115,11 +120,12 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
let backend = Backend::new(cc.egui_ctx.clone());
|
let backend = Backend::new(cc.egui_ctx.clone());
|
||||||
let cache = MediaCache::new(backend.runtime(), backend.http());
|
let cache = MediaCache::new(backend.runtime(), backend.http());
|
||||||
let audio = AudioPlayer::new(backend.runtime(), backend.http());
|
let player = AudioPlayer::new(backend.runtime(), backend.http());
|
||||||
backend.load_models(Modality::Image);
|
for modality in MODALITIES {
|
||||||
backend.load_models(Modality::Video);
|
backend.load_models(modality);
|
||||||
|
}
|
||||||
backend.refresh_session();
|
backend.refresh_session();
|
||||||
Self { cache, audio, toasts: Toasts::default(), catalog: Catalog::default(), user: None, credits: None, page: saved.page, enter_to_generate: saved.enter_to_generate, view_mode: saved.view_mode, autoplay: saved.autoplay, sidebar_open: saved.sidebar_open, image: CreateImagePage::new(saved.image), video: CreateVideoPage::new(saved.video), library: FeedStore::default(), ratios: RatioCache::default(), prompts: HashMap::new(), prompt_records: HashMap::new(), requested_prompts: HashSet::new(), pending_recreate: None, batch_kinds: HashMap::new(), lightbox: None, picker: None, login: None, characters_modal: None, characters: Vec::new(), pending_characters: Vec::new(), settings_open: false, preview: None, requests: Vec::new(), screenshot: ScreenshotRequest::from_env(), next_poll: 0.0, next_credits: CREDITS_INTERVAL, models_retry_at: None, backend }
|
Self { cache, player, toasts: Toasts::default(), catalog: Catalog::default(), user: None, credits: None, page: saved.page, enter_to_generate: saved.enter_to_generate, view_mode: saved.view_mode, autoplay: saved.autoplay, sidebar_open: saved.sidebar_open, image: CreateImagePage::new(saved.image), video: CreateVideoPage::new(saved.video), audio: CreateAudioPage::new(saved.audio), library: FeedStore::default(), ratios: RatioCache::default(), prompts: HashMap::new(), prompt_records: HashMap::new(), requested_prompts: HashSet::new(), pending_recreate: None, batch_kinds: HashMap::new(), lightbox: None, picker: None, login: None, characters_modal: None, characters: Vec::new(), pending_characters: Vec::new(), settings_open: false, preview: None, requests: Vec::new(), screenshot: ScreenshotRequest::from_env(), next_poll: 0.0, next_credits: CREDITS_INTERVAL, models_retry_at: None, backend }
|
||||||
}
|
}
|
||||||
|
|
||||||
// --- Events from the backend -------------------------------------------------------------
|
// --- Events from the backend -------------------------------------------------------------
|
||||||
@@ -130,9 +136,9 @@ impl ArtcraftApp {
|
|||||||
let changed = self.user.as_ref().map(|u| &u.username) != user.as_ref().map(|u| &u.username);
|
let changed = self.user.as_ref().map(|u| &u.username) != user.as_ref().map(|u| &u.username);
|
||||||
self.user = user;
|
self.user = user;
|
||||||
if changed {
|
if changed {
|
||||||
self.image.feed.reset();
|
for feed in self.feeds_mut() {
|
||||||
self.video.feed.reset();
|
feed.reset();
|
||||||
self.library.reset();
|
}
|
||||||
self.next_poll = 0.0;
|
self.next_poll = 0.0;
|
||||||
}
|
}
|
||||||
if self.user.is_some() {
|
if self.user.is_some() {
|
||||||
@@ -163,6 +169,7 @@ impl ArtcraftApp {
|
|||||||
Event::Models(modality, Ok(models)) => match modality {
|
Event::Models(modality, Ok(models)) => match modality {
|
||||||
Modality::Image => self.catalog.set_image(models),
|
Modality::Image => self.catalog.set_image(models),
|
||||||
Modality::Video => self.catalog.set_video(models),
|
Modality::Video => self.catalog.set_video(models),
|
||||||
|
Modality::Audio => self.catalog.set_audio(models),
|
||||||
},
|
},
|
||||||
Event::Models(_, Err(message)) => {
|
Event::Models(_, Err(message)) => {
|
||||||
if self.models_retry_at.is_none() {
|
if self.models_retry_at.is_none() {
|
||||||
@@ -171,22 +178,25 @@ impl ArtcraftApp {
|
|||||||
self.models_retry_at = Some(ctx.input(|i| i.time) + MODEL_RETRY_INTERVAL);
|
self.models_retry_at = Some(ctx.input(|i| i.time) + MODEL_RETRY_INTERVAL);
|
||||||
},
|
},
|
||||||
Event::Cost { key, credits } => {
|
Event::Cost { key, credits } => {
|
||||||
self.image.cost.apply(&key, credits);
|
for cost in [&mut self.image.cost, &mut self.video.cost, &mut self.audio.cost] {
|
||||||
self.video.cost.apply(&key, credits);
|
cost.apply(&key, credits);
|
||||||
|
}
|
||||||
},
|
},
|
||||||
Event::Enqueued { modality, job_tokens, meta } => {
|
Event::Enqueued { modality, job_tokens, meta } => {
|
||||||
let mut env = env!(self, ctx);
|
let mut env = env!(self, ctx);
|
||||||
match modality {
|
match modality {
|
||||||
Modality::Image => self.image.on_enqueued(&mut env, &job_tokens, &meta),
|
Modality::Image => self.image.on_enqueued(&mut env, &job_tokens, &meta),
|
||||||
Modality::Video => self.video.on_enqueued(&mut env, &job_tokens, &meta),
|
Modality::Video => self.video.on_enqueued(&mut env, &job_tokens, &meta),
|
||||||
|
Modality::Audio => self.audio.on_enqueued(&mut env, &job_tokens, &meta),
|
||||||
}
|
}
|
||||||
self.next_poll = ctx.input(|i| i.time) + 2.0;
|
self.next_poll = ctx.input(|i| i.time) + 2.0;
|
||||||
},
|
},
|
||||||
Event::EnqueueFailed { modality, message } => {
|
Event::EnqueueFailed { modality, message, status } => {
|
||||||
let mut env = env!(self, ctx);
|
let mut env = env!(self, ctx);
|
||||||
match modality {
|
match modality {
|
||||||
Modality::Image => self.image.on_enqueue_failed(&mut env, &message),
|
Modality::Image => self.image.on_enqueue_failed(&mut env, &message),
|
||||||
Modality::Video => self.video.on_enqueue_failed(&mut env, &message),
|
Modality::Video => self.video.on_enqueue_failed(&mut env, &message),
|
||||||
|
Modality::Audio => self.audio.on_enqueue_failed(&mut env, &message, status),
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
Event::Uploaded { ref_id, result } => {
|
Event::Uploaded { ref_id, result } => {
|
||||||
@@ -202,6 +212,10 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
if self.image.refs.find_mut(ref_id).is_some() {
|
if self.image.refs.find_mut(ref_id).is_some() {
|
||||||
common::apply_upload(&mut self.toasts, &mut self.image.refs, ref_id, result);
|
common::apply_upload(&mut self.toasts, &mut self.image.refs, ref_id, result);
|
||||||
|
} else if self.audio.refs.find_mut(ref_id).is_some() {
|
||||||
|
if common::apply_upload(&mut self.toasts, &mut self.audio.refs, ref_id, result) {
|
||||||
|
self.audio.check_durations(&mut self.toasts, ref_id);
|
||||||
|
}
|
||||||
} else if common::apply_upload(&mut self.toasts, &mut self.video.refs, ref_id, result) {
|
} else if common::apply_upload(&mut self.toasts, &mut self.video.refs, ref_id, result) {
|
||||||
let mut env = env!(self, ctx);
|
let mut env = env!(self, ctx);
|
||||||
self.video.check_durations(&mut env, ref_id);
|
self.video.check_durations(&mut env, ref_id);
|
||||||
@@ -222,7 +236,7 @@ impl ArtcraftApp {
|
|||||||
if self.pending_characters.len() != before {
|
if self.pending_characters.len() != before {
|
||||||
self.backend.load_characters();
|
self.backend.load_characters();
|
||||||
}
|
}
|
||||||
for (kind, notices) in [(MediaKind::Image, self.image.feed.apply_jobs(MediaKind::Image, &jobs, &self.catalog)), (MediaKind::Video, self.video.feed.apply_jobs(MediaKind::Video, &jobs, &self.catalog))] {
|
for (kind, notices) in [(MediaKind::Image, self.image.feed.apply_jobs(MediaKind::Image, &jobs, &self.catalog)), (MediaKind::Video, self.video.feed.apply_jobs(MediaKind::Video, &jobs, &self.catalog)), (MediaKind::Audio, self.audio.feed.apply_jobs(MediaKind::Audio, &jobs, &self.catalog))] {
|
||||||
for notice in notices {
|
for notice in notices {
|
||||||
match notice {
|
match notice {
|
||||||
FeedNotice::Completed => {
|
FeedNotice::Completed => {
|
||||||
@@ -238,33 +252,30 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
Event::LibraryPage { key, page, items, total_pages } => match key.as_str() {
|
// The picker's pages come back under its own name.
|
||||||
"picker" => {
|
Event::LibraryPage { key, page, items, total_pages } => match FeedKey::from_name(&key) {
|
||||||
|
Some(feed) => self.feed_mut(feed).apply_library_page(feed.kind(), page, &items, total_pages),
|
||||||
|
None => {
|
||||||
if let Some(p) = &mut self.picker {
|
if let Some(p) = &mut self.picker {
|
||||||
p.apply_page(page, items, total_pages);
|
p.apply_page(page, items, total_pages);
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"image" => self.image.feed.apply_library_page(MediaKind::Image, page, &items, total_pages),
|
|
||||||
"video" => self.video.feed.apply_library_page(MediaKind::Video, page, &items, total_pages),
|
|
||||||
_ => self.library.apply_library_page(MediaKind::Image, page, &items, total_pages),
|
|
||||||
},
|
},
|
||||||
Event::LibraryFailed { key, message } => {
|
Event::LibraryFailed { key, message } => {
|
||||||
match key.as_str() {
|
match FeedKey::from_name(&key) {
|
||||||
"picker" => {
|
Some(feed) => self.feed_mut(feed).library_failed(),
|
||||||
|
None => {
|
||||||
if let Some(p) = &mut self.picker {
|
if let Some(p) = &mut self.picker {
|
||||||
p.loading = false;
|
p.loading = false;
|
||||||
p.has_more = false;
|
p.has_more = false;
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"image" => self.image.feed.library_failed(),
|
|
||||||
"video" => self.video.feed.library_failed(),
|
|
||||||
_ => self.library.library_failed(),
|
|
||||||
}
|
}
|
||||||
self.toasts.error(format!("Couldn't load your library: {message}"));
|
self.toasts.error(format!("Couldn't load your library: {message}"));
|
||||||
},
|
},
|
||||||
Event::Batch { job_token, items } => match self.batch_kinds.remove(&job_token) {
|
Event::Batch { job_token, items } => {
|
||||||
Some(MediaKind::Video) => self.video.feed.apply_batch(MediaKind::Video, &items),
|
let kind = self.batch_kinds.remove(&job_token).unwrap_or(MediaKind::Image);
|
||||||
_ => self.image.feed.apply_batch(MediaKind::Image, &items),
|
self.feed_mut(FeedKey::of(kind)).apply_batch(kind, &items);
|
||||||
},
|
},
|
||||||
Event::Prompt(prompt) => {
|
Event::Prompt(prompt) => {
|
||||||
if let Some(text) = &prompt.maybe_positive_prompt {
|
if let Some(text) = &prompt.maybe_positive_prompt {
|
||||||
@@ -285,7 +296,7 @@ impl ArtcraftApp {
|
|||||||
},
|
},
|
||||||
Event::CharacterChanged => self.backend.load_characters(),
|
Event::CharacterChanged => self.backend.load_characters(),
|
||||||
Event::Deleted(token) => {
|
Event::Deleted(token) => {
|
||||||
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
|
for feed in self.feeds_mut() {
|
||||||
feed.remove_item(&token);
|
feed.remove_item(&token);
|
||||||
}
|
}
|
||||||
if self.lightbox.as_ref().is_some_and(|l| l.token == token) {
|
if self.lightbox.as_ref().is_some_and(|l| l.token == token) {
|
||||||
@@ -319,7 +330,7 @@ impl ArtcraftApp {
|
|||||||
self.open_login();
|
self.open_login();
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let refs = if page == MediaKind::Image { &self.image.refs } else { &self.video.refs };
|
let refs = self.refs(page);
|
||||||
let attached: HashSet<String> = refs.list(kind).iter().chain(refs.first_frame.iter()).chain(refs.last_frame.iter()).filter_map(|r| r.token.clone()).collect();
|
let attached: HashSet<String> = refs.list(kind).iter().chain(refs.first_frame.iter()).chain(refs.last_frame.iter()).filter_map(|r| r.token.clone()).collect();
|
||||||
let mut picker = LibraryPicker::new(kind, slot, page, max, attached);
|
let mut picker = LibraryPicker::new(kind, slot, page, max, attached);
|
||||||
picker.loading = true;
|
picker.loading = true;
|
||||||
@@ -332,8 +343,7 @@ impl ArtcraftApp {
|
|||||||
};
|
};
|
||||||
let image = RefMedia::from_library(RefKind::Image, item.token.clone(), item.thumbnail.clone(), Some(item.full_url.clone()), 0.0);
|
let image = RefMedia::from_library(RefKind::Image, item.token.clone(), item.thumbnail.clone(), Some(item.full_url.clone()), 0.0);
|
||||||
self.video.set_start_image(image);
|
self.video.set_start_image(image);
|
||||||
self.lightbox = None;
|
self.navigate(Page::CreateVideo);
|
||||||
self.page = Page::CreateVideo;
|
|
||||||
},
|
},
|
||||||
AppRequest::Recreate { kind, token } => {
|
AppRequest::Recreate { kind, token } => {
|
||||||
let Some(prompt_token) = self.find_item(&token).and_then(|i| i.prompt_token.clone()) else {
|
let Some(prompt_token) = self.find_item(&token).and_then(|i| i.prompt_token.clone()) else {
|
||||||
@@ -359,7 +369,7 @@ impl ArtcraftApp {
|
|||||||
},
|
},
|
||||||
AppRequest::Download(tokens) => self.download(&tokens),
|
AppRequest::Download(tokens) => self.download(&tokens),
|
||||||
AppRequest::Preview(url) => self.preview = Some(url),
|
AppRequest::Preview(url) => self.preview = Some(url),
|
||||||
AppRequest::ToggleAudio { ref_id, url } => self.audio.toggle(ctx, &format!("ref:{ref_id}"), &url),
|
AppRequest::ToggleAudio { ref_id, url } => self.player.toggle(ctx, &format!("ref:{ref_id}"), &url),
|
||||||
AppRequest::OpenCharacters => {
|
AppRequest::OpenCharacters => {
|
||||||
if self.user.is_none() {
|
if self.user.is_none() {
|
||||||
self.open_login();
|
self.open_login();
|
||||||
@@ -377,13 +387,9 @@ impl ArtcraftApp {
|
|||||||
let Some(user) = &self.user else {
|
let Some(user) = &self.user else {
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
let (feed, name) = match key {
|
let username = user.username.clone();
|
||||||
FeedKey::Image => (&mut self.image.feed, "image"),
|
if let Some(page) = self.feed_mut(key).next_library_page() {
|
||||||
FeedKey::Video => (&mut self.video.feed, "video"),
|
self.backend.load_library(key.name().to_owned(), username, key.media_classes(), page);
|
||||||
FeedKey::Library => (&mut self.library, "library"),
|
|
||||||
};
|
|
||||||
if let Some(page) = feed.next_library_page() {
|
|
||||||
self.backend.load_library(name.to_owned(), user.username.clone(), key.media_classes(), page);
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -393,14 +399,15 @@ impl ArtcraftApp {
|
|||||||
match kind {
|
match kind {
|
||||||
MediaKind::Image => {
|
MediaKind::Image => {
|
||||||
self.image.apply_prompt(prompt);
|
self.image.apply_prompt(prompt);
|
||||||
self.page = Page::CreateImage;
|
self.navigate(Page::CreateImage);
|
||||||
},
|
},
|
||||||
MediaKind::Video => {
|
MediaKind::Video => {
|
||||||
self.video.apply_prompt(prompt);
|
self.video.apply_prompt(prompt);
|
||||||
self.page = Page::CreateVideo;
|
self.navigate(Page::CreateVideo);
|
||||||
},
|
},
|
||||||
|
// Audio isn't recreated yet (no Recreate button offers it).
|
||||||
|
MediaKind::Audio => {},
|
||||||
}
|
}
|
||||||
self.lightbox = None;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
fn download(&mut self, tokens: &[String]) {
|
fn download(&mut self, tokens: &[String]) {
|
||||||
@@ -418,7 +425,7 @@ impl ArtcraftApp {
|
|||||||
for item in many {
|
for item in many {
|
||||||
self.backend.save_url(item.full_url.clone(), dir.join(file_name_for(item)));
|
self.backend.save_url(item.full_url.clone(), dir.join(file_name_for(item)));
|
||||||
}
|
}
|
||||||
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
|
for feed in self.feeds_mut() {
|
||||||
feed.selecting = false;
|
feed.selecting = false;
|
||||||
feed.selected.clear();
|
feed.selected.clear();
|
||||||
}
|
}
|
||||||
@@ -428,7 +435,52 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn find_item(&self, token: &str) -> Option<&FeedItem> {
|
fn find_item(&self, token: &str) -> Option<&FeedItem> {
|
||||||
self.image.feed.find(token).or_else(|| self.video.feed.find(token)).or_else(|| self.library.find(token))
|
self.feeds().into_iter().find_map(|f| f.find(token))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn feeds(&self) -> [&FeedStore; 4] {
|
||||||
|
[&self.image.feed, &self.video.feed, &self.audio.feed, &self.library]
|
||||||
|
}
|
||||||
|
|
||||||
|
fn feeds_mut(&mut self) -> [&mut FeedStore; 4] {
|
||||||
|
[&mut self.image.feed, &mut self.video.feed, &mut self.audio.feed, &mut self.library]
|
||||||
|
}
|
||||||
|
|
||||||
|
fn feed(&self, key: FeedKey) -> &FeedStore {
|
||||||
|
match key {
|
||||||
|
FeedKey::Image => &self.image.feed,
|
||||||
|
FeedKey::Video => &self.video.feed,
|
||||||
|
FeedKey::Audio => &self.audio.feed,
|
||||||
|
FeedKey::Library => &self.library,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn feed_mut(&mut self, key: FeedKey) -> &mut FeedStore {
|
||||||
|
match key {
|
||||||
|
FeedKey::Image => &mut self.image.feed,
|
||||||
|
FeedKey::Video => &mut self.video.feed,
|
||||||
|
FeedKey::Audio => &mut self.audio.feed,
|
||||||
|
FeedKey::Library => &mut self.library,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The references of the page that makes `kind`.
|
||||||
|
fn refs(&self, kind: MediaKind) -> &References {
|
||||||
|
match kind {
|
||||||
|
MediaKind::Image => &self.image.refs,
|
||||||
|
MediaKind::Video => &self.video.refs,
|
||||||
|
MediaKind::Audio => &self.audio.refs,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Opens `page`, closing the lightbox. Leaving a page stops its audio, like the webapp's
|
||||||
|
/// players unmounting.
|
||||||
|
fn navigate(&mut self, page: Page) {
|
||||||
|
if page != self.page {
|
||||||
|
self.player.stop();
|
||||||
|
}
|
||||||
|
self.page = page;
|
||||||
|
self.lightbox = None;
|
||||||
}
|
}
|
||||||
|
|
||||||
fn request_prompt(&mut self, prompt_token: Option<String>) {
|
fn request_prompt(&mut self, prompt_token: Option<String>) {
|
||||||
@@ -445,12 +497,13 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The feed the top bar's toggles act on.
|
/// The feed the page shows (and the top bar's toggles act on).
|
||||||
fn current_feed(&mut self) -> Option<&mut FeedStore> {
|
fn page_feed(&self) -> Option<FeedKey> {
|
||||||
match self.page {
|
match self.page {
|
||||||
Page::CreateImage => Some(&mut self.image.feed),
|
Page::CreateImage => Some(FeedKey::Image),
|
||||||
Page::CreateVideo => Some(&mut self.video.feed),
|
Page::CreateVideo => Some(FeedKey::Video),
|
||||||
Page::Library => Some(&mut self.library),
|
Page::CreateAudio => Some(FeedKey::Audio),
|
||||||
|
Page::Library => Some(FeedKey::Library),
|
||||||
_ => None,
|
_ => None,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -469,11 +522,10 @@ impl ArtcraftApp {
|
|||||||
}
|
}
|
||||||
if self.models_retry_at.is_some_and(|t| now >= t) {
|
if self.models_retry_at.is_some_and(|t| now >= t) {
|
||||||
self.models_retry_at = None;
|
self.models_retry_at = None;
|
||||||
if self.catalog.image.is_empty() {
|
for (modality, loaded) in MODALITIES.into_iter().zip([&self.catalog.image, &self.catalog.video, &self.catalog.audio]) {
|
||||||
self.backend.load_models(Modality::Image);
|
if loaded.is_empty() {
|
||||||
}
|
self.backend.load_models(modality);
|
||||||
if self.catalog.video.is_empty() {
|
}
|
||||||
self.backend.load_models(Modality::Video);
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
ctx.request_repaint_after(Duration::from_secs(1));
|
ctx.request_repaint_after(Duration::from_secs(1));
|
||||||
@@ -500,10 +552,7 @@ impl ArtcraftApp {
|
|||||||
|
|
||||||
fn apply_shell_action(&mut self, action: ShellAction) {
|
fn apply_shell_action(&mut self, action: ShellAction) {
|
||||||
match action {
|
match action {
|
||||||
ShellAction::Navigate(page) => {
|
ShellAction::Navigate(page) => self.navigate(page),
|
||||||
self.page = page;
|
|
||||||
self.lightbox = None;
|
|
||||||
},
|
|
||||||
ShellAction::ToggleSidebar => self.sidebar_open = !self.sidebar_open,
|
ShellAction::ToggleSidebar => self.sidebar_open = !self.sidebar_open,
|
||||||
ShellAction::SignIn => self.open_login(),
|
ShellAction::SignIn => self.open_login(),
|
||||||
ShellAction::SignOut => {
|
ShellAction::SignOut => {
|
||||||
@@ -514,7 +563,8 @@ impl ArtcraftApp {
|
|||||||
ShellAction::SetViewMode(mode) => self.view_mode = mode,
|
ShellAction::SetViewMode(mode) => self.view_mode = mode,
|
||||||
ShellAction::SetAutoplay(on) => self.autoplay = on,
|
ShellAction::SetAutoplay(on) => self.autoplay = on,
|
||||||
ShellAction::ToggleSelect => {
|
ShellAction::ToggleSelect => {
|
||||||
if let Some(feed) = self.current_feed() {
|
if let Some(key) = self.page_feed() {
|
||||||
|
let feed = self.feed_mut(key);
|
||||||
feed.selecting = !feed.selecting;
|
feed.selecting = !feed.selecting;
|
||||||
feed.selected.clear();
|
feed.selected.clear();
|
||||||
}
|
}
|
||||||
@@ -528,13 +578,10 @@ impl ArtcraftApp {
|
|||||||
let ctx = ui.ctx().clone();
|
let ctx = ui.ctx().clone();
|
||||||
let gap = shell::PANEL_GAP;
|
let gap = shell::PANEL_GAP;
|
||||||
let margin = egui::Margin { left: if self.sidebar_open { 0 } else { gap }, right: gap, top: 0, bottom: gap };
|
let margin = egui::Margin { left: if self.sidebar_open { 0 } else { gap }, right: gap, top: 0, bottom: gap };
|
||||||
let toggles = match self.page {
|
// Only feeds with videos offer the preview autoplay toggle.
|
||||||
Page::CreateImage => Some(FeedToggles { mode: self.view_mode, selecting: self.image.feed.selecting, autoplay: None }),
|
let toggles = self.page_feed().map(|key| FeedToggles { mode: self.view_mode, selecting: self.feed(key).selecting, autoplay: matches!(key, FeedKey::Video | FeedKey::Library).then_some(self.autoplay) });
|
||||||
Page::CreateVideo => Some(FeedToggles { mode: self.view_mode, selecting: self.video.feed.selecting, autoplay: Some(self.autoplay) }),
|
|
||||||
Page::Library => Some(FeedToggles { mode: self.view_mode, selecting: self.library.selecting, autoplay: Some(self.autoplay) }),
|
|
||||||
_ => None,
|
|
||||||
};
|
|
||||||
let mut header_action = None;
|
let mut header_action = None;
|
||||||
|
let mut open_page = None;
|
||||||
egui::CentralPanel::default().frame(egui::Frame::NONE.fill(theme::CHROME).inner_margin(margin)).show(ui, |ui| {
|
egui::CentralPanel::default().frame(egui::Frame::NONE.fill(theme::CHROME).inner_margin(margin)).show(ui, |ui| {
|
||||||
let panel = egui::Frame::new().fill(theme::BG).stroke(theme::hairline()).corner_radius(shell::PANEL_RADIUS);
|
let panel = egui::Frame::new().fill(theme::BG).stroke(theme::hairline()).corner_radius(shell::PANEL_RADIUS);
|
||||||
panel.show(ui, |ui| {
|
panel.show(ui, |ui| {
|
||||||
@@ -545,13 +592,10 @@ impl ArtcraftApp {
|
|||||||
egui::CentralPanel::default().frame(egui::Frame::NONE).show(ui, |ui| {
|
egui::CentralPanel::default().frame(egui::Frame::NONE).show(ui, |ui| {
|
||||||
let mut env = env!(self, ctx);
|
let mut env = env!(self, ctx);
|
||||||
match self.page {
|
match self.page {
|
||||||
Page::Home => {
|
Page::Home => open_page = other::home(ui, env.signed_in, env.requests),
|
||||||
if let Some(page) = other::home(ui, env.signed_in, env.requests) {
|
|
||||||
self.page = page;
|
|
||||||
}
|
|
||||||
},
|
|
||||||
Page::CreateImage => self.image.ui(ui, &mut env),
|
Page::CreateImage => self.image.ui(ui, &mut env),
|
||||||
Page::CreateVideo => self.video.ui(ui, &mut env),
|
Page::CreateVideo => self.video.ui(ui, &mut env),
|
||||||
|
Page::CreateAudio => self.audio.ui(ui, &mut env),
|
||||||
Page::Library => other::library(ui, &mut env, &mut self.library),
|
Page::Library => other::library(ui, &mut env, &mut self.library),
|
||||||
page => other::coming_soon(ui, page),
|
page => other::coming_soon(ui, page),
|
||||||
}
|
}
|
||||||
@@ -561,39 +605,33 @@ impl ArtcraftApp {
|
|||||||
if let Some(action) = header_action {
|
if let Some(action) = header_action {
|
||||||
self.apply_shell_action(action);
|
self.apply_shell_action(action);
|
||||||
}
|
}
|
||||||
self.prefetch_list_prompts();
|
if let Some(page) = open_page {
|
||||||
|
self.navigate(page);
|
||||||
|
}
|
||||||
|
self.prefetch_prompts();
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The list view shows prompt texts: fetch the records for the visible feed, a few at a time.
|
/// The list view and audio cards show prompt texts: fetch the records for the visible feed, a
|
||||||
fn prefetch_list_prompts(&mut self) {
|
/// few at a time.
|
||||||
if self.view_mode != ViewMode::List {
|
fn prefetch_prompts(&mut self) {
|
||||||
|
let Some(key) = self.page_feed() else {
|
||||||
return;
|
return;
|
||||||
}
|
|
||||||
let feed = match self.page {
|
|
||||||
Page::CreateImage => &self.image.feed,
|
|
||||||
Page::CreateVideo => &self.video.feed,
|
|
||||||
Page::Library => &self.library,
|
|
||||||
_ => return,
|
|
||||||
};
|
};
|
||||||
let wanted: Vec<String> = feed.items.iter().filter_map(|i| i.prompt_token.clone()).filter(|t| !self.requested_prompts.contains(t)).take(PROMPT_FETCHES_PER_FRAME).collect();
|
let list = self.view_mode == ViewMode::List;
|
||||||
|
let wanted: Vec<String> = self.feed(key).items.iter().filter(|i| list || i.kind == MediaKind::Audio).filter_map(|i| i.prompt_token.clone()).filter(|t| !self.requested_prompts.contains(t)).take(PROMPT_FETCHES_PER_FRAME).collect();
|
||||||
for token in wanted {
|
for token in wanted {
|
||||||
self.request_prompt(Some(token));
|
self.request_prompt(Some(token));
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
fn overlays(&mut self, ctx: &egui::Context) {
|
fn overlays(&mut self, ctx: &egui::Context) {
|
||||||
if let Some(lb) = &mut self.lightbox {
|
if let Some((kind, token)) = self.lightbox.as_ref().map(|lb| (lb.kind, lb.token.clone())) {
|
||||||
let feed = match lb.kind {
|
let key = if self.page == Page::Library { FeedKey::Library } else { FeedKey::of(kind) };
|
||||||
_ if self.page == Page::Library => &self.library,
|
let order = self.feed(key).ordered_tokens();
|
||||||
MediaKind::Image => &self.image.feed,
|
match self.find_item(&token).cloned() {
|
||||||
MediaKind::Video => &self.video.feed,
|
|
||||||
};
|
|
||||||
let order = feed.ordered_tokens();
|
|
||||||
let item = self.image.feed.find(&lb.token).or_else(|| self.video.feed.find(&lb.token)).or_else(|| self.library.find(&lb.token)).cloned();
|
|
||||||
match item {
|
|
||||||
Some(item) => {
|
Some(item) => {
|
||||||
let prompt = item.prompt_token.as_ref().and_then(|t| self.prompt_records.get(t));
|
let prompt = item.prompt_token.as_ref().and_then(|t| self.prompt_records.get(t));
|
||||||
let actions = lb.show(ctx, &item, &order, prompt, &self.catalog, &mut self.cache);
|
let actions = self.lightbox.as_mut().map(|lb| lb.show(ctx, &item, &order, prompt, &self.catalog, &mut self.cache, &mut self.player)).unwrap_or_default();
|
||||||
for action in actions {
|
for action in actions {
|
||||||
self.lightbox_action(ctx, action, &item);
|
self.lightbox_action(ctx, action, &item);
|
||||||
}
|
}
|
||||||
@@ -615,8 +653,11 @@ impl ArtcraftApp {
|
|||||||
},
|
},
|
||||||
PickerAction::Confirm(picks) => {
|
PickerAction::Confirm(picks) => {
|
||||||
if let Some(p) = self.picker.take() {
|
if let Some(p) = self.picker.take() {
|
||||||
let refs = if p.page == MediaKind::Image { &mut self.image.refs } else { &mut self.video.refs };
|
match p.page {
|
||||||
common::attach_from_library(refs, p.kind, p.slot, picks);
|
MediaKind::Image => common::attach_from_library(&mut self.image.refs, p.kind, p.slot, picks),
|
||||||
|
MediaKind::Video => common::attach_from_library(&mut self.video.refs, p.kind, p.slot, picks),
|
||||||
|
MediaKind::Audio => self.audio.attach_from_library(&mut self.toasts, &self.catalog, p.kind, picks),
|
||||||
|
}
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
@@ -706,7 +747,7 @@ impl ArtcraftApp {
|
|||||||
if let Some(next) = self.find_item(&token).cloned() {
|
if let Some(next) = self.find_item(&token).cloned() {
|
||||||
self.request_prompt(next.prompt_token);
|
self.request_prompt(next.prompt_token);
|
||||||
}
|
}
|
||||||
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
|
for feed in self.feeds_mut() {
|
||||||
if feed.find(&token).is_some() {
|
if feed.find(&token).is_some() {
|
||||||
feed.last_viewed = Some(token.clone());
|
feed.last_viewed = Some(token.clone());
|
||||||
}
|
}
|
||||||
@@ -744,10 +785,10 @@ impl eframe::App for ArtcraftApp {
|
|||||||
fn ui(&mut self, ui: &mut Ui, _frame: &mut eframe::Frame) {
|
fn ui(&mut self, ui: &mut Ui, _frame: &mut eframe::Frame) {
|
||||||
let ctx = ui.ctx().clone();
|
let ctx = ui.ctx().clone();
|
||||||
self.cache.poll(&ctx);
|
self.cache.poll(&ctx);
|
||||||
if let Some(err) = self.audio.poll() {
|
if let Some(err) = self.player.poll() {
|
||||||
self.toasts.error(err);
|
self.toasts.error(err);
|
||||||
}
|
}
|
||||||
let playing = [&self.video.refs, &self.image.refs].iter().flat_map(|r| r.audios.iter()).map(|r| r.id).find(|id| self.audio.is_playing(&format!("ref:{id}")));
|
let playing = [&self.video.refs, &self.image.refs].iter().flat_map(|r| r.audios.iter()).map(|r| r.id).find(|id| self.player.is_playing(&format!("ref:{id}")));
|
||||||
crate::prompt_box::deck::set_playing_audio(&ctx, playing);
|
crate::prompt_box::deck::set_playing_audio(&ctx, playing);
|
||||||
self.shell(ui);
|
self.shell(ui);
|
||||||
self.page_ui(ui);
|
self.page_ui(ui);
|
||||||
@@ -763,7 +804,7 @@ impl eframe::App for ArtcraftApp {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn save(&mut self, storage: &mut dyn eframe::Storage) {
|
fn save(&mut self, storage: &mut dyn eframe::Storage) {
|
||||||
let saved = Saved { page: self.page, enter_to_generate: self.enter_to_generate, view_mode: self.view_mode, autoplay: self.autoplay, sidebar_open: self.sidebar_open, image: self.image.settings.clone(), video: self.video.settings.clone() };
|
let saved = Saved { page: self.page, enter_to_generate: self.enter_to_generate, view_mode: self.view_mode, autoplay: self.autoplay, sidebar_open: self.sidebar_open, image: self.image.settings.clone(), video: self.video.settings.clone(), audio: self.audio.settings.clone() };
|
||||||
eframe::set_value(storage, STORAGE_KEY, &saved);
|
eframe::set_value(storage, STORAGE_KEY, &saved);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -819,6 +860,11 @@ impl ScreenshotRequest {
|
|||||||
|
|
||||||
/// `artcraft_<token>.<ext>`, the extension taken from the URL.
|
/// `artcraft_<token>.<ext>`, the extension taken from the URL.
|
||||||
fn file_name_for(item: &FeedItem) -> String {
|
fn file_name_for(item: &FeedItem) -> String {
|
||||||
let ext = item.full_url.rsplit('.').next().filter(|e| e.len() <= 4 && e.chars().all(|c| c.is_ascii_alphanumeric())).unwrap_or(if item.kind == MediaKind::Video { "mp4" } else { "png" });
|
let fallback = match item.kind {
|
||||||
|
MediaKind::Image => "png",
|
||||||
|
MediaKind::Video => "mp4",
|
||||||
|
MediaKind::Audio => "mp3",
|
||||||
|
};
|
||||||
|
let ext = item.full_url.rsplit('.').next().filter(|e| e.len() <= 4 && e.chars().all(|c| c.is_ascii_alphanumeric())).unwrap_or(fallback);
|
||||||
format!("artcraft_{}.{ext}", item.token)
|
format!("artcraft_{}.{ext}", item.token)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,14 +1,26 @@
|
|||||||
//! Playing one audio clip at a time (reference cards play and stop on click, like the webapp's
|
//! Audio, one clip at a time (`audio-playback-controller`): reference cards play and stop on
|
||||||
//! `DeckCard`). The output device opens on first use; files download in the background.
|
//! click (the webapp's `DeckCard`), feed tracks pause, resume and seek (`WaveformAudioPlayer`).
|
||||||
|
//! Tracks also get waveforms, decoded off the UI thread. The output device opens on first use;
|
||||||
|
//! files download in the background.
|
||||||
|
|
||||||
|
use std::collections::HashMap;
|
||||||
use std::io::Cursor;
|
use std::io::Cursor;
|
||||||
|
use std::sync::Arc;
|
||||||
use std::sync::mpsc::{Receiver, Sender, channel};
|
use std::sync::mpsc::{Receiver, Sender, channel};
|
||||||
|
use std::time::Duration;
|
||||||
|
|
||||||
use log::warn;
|
use log::warn;
|
||||||
use rodio::{Decoder, OutputStream, OutputStreamHandle, Sink};
|
use rodio::{Decoder, OutputStream, OutputStreamHandle, Sink, Source};
|
||||||
|
use tokio::sync::Semaphore;
|
||||||
|
|
||||||
/// The webapp's preview volume.
|
/// The webapp's reference preview volume.
|
||||||
const PREVIEW_VOLUME: f32 = 0.2;
|
const PREVIEW_VOLUME: f32 = 0.2;
|
||||||
|
/// Peaks kept per waveform (resampled to however many bars are drawn).
|
||||||
|
const PEAK_COUNT: usize = 256;
|
||||||
|
/// Frames folded into one peak while decoding.
|
||||||
|
const WINDOW_FRAMES: usize = 1024;
|
||||||
|
/// Waveforms decoding at once (each downloads its whole file).
|
||||||
|
const MAX_CONCURRENT_WAVEFORMS: usize = 2;
|
||||||
|
|
||||||
pub struct AudioPlayer {
|
pub struct AudioPlayer {
|
||||||
rt: tokio::runtime::Handle,
|
rt: tokio::runtime::Handle,
|
||||||
@@ -16,36 +28,99 @@ pub struct AudioPlayer {
|
|||||||
/// Opened on first play (and kept: closing it would stop the sound).
|
/// Opened on first play (and kept: closing it would stop the sound).
|
||||||
output: Option<(OutputStream, OutputStreamHandle)>,
|
output: Option<(OutputStream, OutputStreamHandle)>,
|
||||||
sink: Option<Sink>,
|
sink: Option<Sink>,
|
||||||
/// What's playing (or loading), by the caller's key.
|
/// What's playing (or loading).
|
||||||
current: Option<String>,
|
current: Option<Clip>,
|
||||||
tx: Sender<(String, Result<Vec<u8>, String>)>,
|
tx: Sender<Loaded>,
|
||||||
rx: Receiver<(String, Result<Vec<u8>, String>)>,
|
rx: Receiver<Loaded>,
|
||||||
|
waveforms: HashMap<String, WaveformLookup>,
|
||||||
|
limiter: Arc<Semaphore>,
|
||||||
|
/// The tracks' volume slider, 0–1.
|
||||||
|
track_volume: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A track's shape: peaks (0–1, loudest = 1) and its length in seconds.
|
||||||
|
#[derive(Debug, PartialEq)]
|
||||||
|
pub struct Waveform {
|
||||||
|
pub peaks: Vec<f32>,
|
||||||
|
pub duration: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What [`AudioPlayer::waveform`] knows about a track.
|
||||||
|
#[derive(Clone)]
|
||||||
|
pub enum WaveformLookup {
|
||||||
|
Loading,
|
||||||
|
Ready(Arc<Waveform>),
|
||||||
|
Failed,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where the current track is.
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||||
|
pub struct Playback {
|
||||||
|
/// Still downloading.
|
||||||
|
pub loading: bool,
|
||||||
|
pub paused: bool,
|
||||||
|
/// Seconds in.
|
||||||
|
pub position: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The clip that's playing, by the caller's key (a track's key is its URL).
|
||||||
|
struct Clip {
|
||||||
|
key: String,
|
||||||
|
track: bool,
|
||||||
|
/// Where to start once it has downloaded (a seek before it loaded).
|
||||||
|
start_at: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
enum Loaded {
|
||||||
|
Clip { key: String, bytes: Result<Arc<[u8]>, String> },
|
||||||
|
Waveform { url: String, waveform: Option<Waveform> },
|
||||||
}
|
}
|
||||||
|
|
||||||
impl AudioPlayer {
|
impl AudioPlayer {
|
||||||
pub fn new(rt: tokio::runtime::Handle, http: reqwest::Client) -> Self {
|
pub fn new(rt: tokio::runtime::Handle, http: reqwest::Client) -> Self {
|
||||||
let (tx, rx) = channel();
|
let (tx, rx) = channel();
|
||||||
Self { rt, http, output: None, sink: None, current: None, tx, rx }
|
Self { rt, http, output: None, sink: None, current: None, tx, rx, waveforms: HashMap::new(), limiter: Arc::new(Semaphore::new(MAX_CONCURRENT_WAVEFORMS)), track_volume: 1.0 }
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Starts `url` under `key`, or stops it if it's what's playing.
|
/// Starts a reference clip under `key`, or stops it if it's what's playing.
|
||||||
pub fn toggle(&mut self, ctx: &egui::Context, key: &str, url: &str) {
|
pub fn toggle(&mut self, ctx: &egui::Context, key: &str, url: &str) {
|
||||||
if self.is_playing(key) {
|
if self.is_playing(key) {
|
||||||
self.stop();
|
self.stop();
|
||||||
|
} else {
|
||||||
|
self.start(ctx, key, url, false, 0.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Plays or pauses a feed track.
|
||||||
|
pub fn toggle_track(&mut self, ctx: &egui::Context, url: &str) {
|
||||||
|
match (&self.sink, self.is_playing(url)) {
|
||||||
|
(Some(sink), true) if sink.is_paused() => sink.play(),
|
||||||
|
(Some(sink), true) => sink.pause(),
|
||||||
|
// Still downloading: a second click cancels.
|
||||||
|
(None, true) => self.stop(),
|
||||||
|
_ => self.start(ctx, url, url, true, 0.0),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Jumps a feed track to `secs` (starting it there if it isn't the current one).
|
||||||
|
pub fn seek_track(&mut self, ctx: &egui::Context, url: &str, secs: f32) {
|
||||||
|
if !self.is_playing(url) {
|
||||||
|
self.start(ctx, url, url, true, secs);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
self.stop();
|
match &self.sink {
|
||||||
self.current = Some(key.to_owned());
|
Some(sink) => {
|
||||||
let (http, tx, ctx, key, url) = (self.http.clone(), self.tx.clone(), ctx.clone(), key.to_owned(), url.to_owned());
|
if let Err(err) = sink.try_seek(Duration::from_secs_f32(secs.max(0.0))) {
|
||||||
self.rt.spawn(async move {
|
warn!("Audio seek failed: {err}");
|
||||||
let bytes = async {
|
}
|
||||||
let response = http.get(&url).send().await.and_then(|r| r.error_for_status()).map_err(|e| e.to_string())?;
|
sink.play();
|
||||||
response.bytes().await.map(|b| b.to_vec()).map_err(|e| e.to_string())
|
},
|
||||||
}
|
None => {
|
||||||
.await;
|
if let Some(clip) = &mut self.current {
|
||||||
let _ = tx.send((key, bytes));
|
clip.start_at = secs;
|
||||||
ctx.request_repaint();
|
}
|
||||||
});
|
},
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn stop(&mut self) {
|
pub fn stop(&mut self) {
|
||||||
@@ -55,26 +130,78 @@ impl AudioPlayer {
|
|||||||
self.current = None;
|
self.current = None;
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether `key` is playing or about to.
|
/// Whether `key` is playing, paused or about to play.
|
||||||
pub fn is_playing(&self, key: &str) -> bool {
|
pub fn is_playing(&self, key: &str) -> bool {
|
||||||
self.current.as_deref() == Some(key)
|
self.current.as_ref().is_some_and(|c| c.key == key)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Starts downloads that finished; notices clips that ended. Call once per frame. Returns an
|
/// Where the track at `url` is, when it's the current one.
|
||||||
|
pub fn track(&self, url: &str) -> Option<Playback> {
|
||||||
|
let clip = self.current.as_ref().filter(|c| c.key == url)?;
|
||||||
|
Some(match &self.sink {
|
||||||
|
Some(sink) => Playback { loading: false, paused: sink.is_paused(), position: sink.get_pos().as_secs_f32() },
|
||||||
|
None => Playback { loading: true, paused: false, position: clip.start_at },
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn track_volume(&self) -> f32 {
|
||||||
|
self.track_volume
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn set_track_volume(&mut self, volume: f32) {
|
||||||
|
self.track_volume = volume.clamp(0.0, 1.0);
|
||||||
|
if let (Some(sink), Some(clip)) = (&self.sink, &self.current) {
|
||||||
|
if clip.track {
|
||||||
|
sink.set_volume(self.track_volume);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The waveform of the track at `url`, downloading and decoding it the first time it's asked
|
||||||
|
/// for.
|
||||||
|
pub fn waveform(&mut self, ctx: &egui::Context, url: &str) -> WaveformLookup {
|
||||||
|
if let Some(lookup) = self.waveforms.get(url) {
|
||||||
|
return lookup.clone();
|
||||||
|
}
|
||||||
|
self.waveforms.insert(url.to_owned(), WaveformLookup::Loading);
|
||||||
|
let (http, limiter, tx, ctx, url) = (self.http.clone(), self.limiter.clone(), self.tx.clone(), ctx.clone(), url.to_owned());
|
||||||
|
self.rt.spawn(async move {
|
||||||
|
let waveform = match limiter.acquire_owned().await {
|
||||||
|
Ok(_permit) => match download(&http, &url).await {
|
||||||
|
Ok(bytes) => tokio::task::spawn_blocking(move || decode_waveform(bytes)).await.ok().flatten(),
|
||||||
|
Err(err) => {
|
||||||
|
warn!("Waveform download failed for {url}: {err}");
|
||||||
|
None
|
||||||
|
},
|
||||||
|
},
|
||||||
|
Err(_) => None,
|
||||||
|
};
|
||||||
|
let _ = tx.send(Loaded::Waveform { url, waveform });
|
||||||
|
ctx.request_repaint();
|
||||||
|
});
|
||||||
|
WaveformLookup::Loading
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Starts downloads that finished and notices clips that ended. Call once per frame. Returns an
|
||||||
/// error message when a clip couldn't play.
|
/// error message when a clip couldn't play.
|
||||||
pub fn poll(&mut self) -> Option<String> {
|
pub fn poll(&mut self) -> Option<String> {
|
||||||
let mut error = None;
|
let mut error = None;
|
||||||
while let Ok((key, bytes)) = self.rx.try_recv() {
|
while let Ok(loaded) = self.rx.try_recv() {
|
||||||
// Superseded by another clip (or stopped) while downloading.
|
match loaded {
|
||||||
if self.current.as_deref() != Some(key.as_str()) {
|
Loaded::Waveform { url, waveform } => {
|
||||||
continue;
|
self.waveforms.insert(url, waveform.map_or(WaveformLookup::Failed, |w| WaveformLookup::Ready(Arc::new(w))));
|
||||||
}
|
},
|
||||||
match bytes.and_then(|b| self.play_bytes(b)) {
|
Loaded::Clip { key, bytes } => {
|
||||||
Ok(()) => {},
|
// Superseded by another clip (or stopped) while downloading.
|
||||||
Err(err) => {
|
let Some(clip) = self.current.as_ref().filter(|c| c.key == key) else {
|
||||||
warn!("Audio preview failed: {err}");
|
continue;
|
||||||
self.current = None;
|
};
|
||||||
error = Some(format!("Couldn't play the audio: {err}"));
|
let (volume, start_at) = (if clip.track { self.track_volume } else { PREVIEW_VOLUME }, clip.start_at);
|
||||||
|
if let Err(err) = bytes.and_then(|b| self.play_bytes(b, volume, start_at)) {
|
||||||
|
warn!("Audio playback failed: {err}");
|
||||||
|
self.current = None;
|
||||||
|
error = Some(format!("Couldn't play the audio: {err}"));
|
||||||
|
}
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -85,7 +212,18 @@ impl AudioPlayer {
|
|||||||
error
|
error
|
||||||
}
|
}
|
||||||
|
|
||||||
fn play_bytes(&mut self, bytes: Vec<u8>) -> Result<(), String> {
|
fn start(&mut self, ctx: &egui::Context, key: &str, url: &str, track: bool, start_at: f32) {
|
||||||
|
self.stop();
|
||||||
|
self.current = Some(Clip { key: key.to_owned(), track, start_at });
|
||||||
|
let (http, tx, ctx, key, url) = (self.http.clone(), self.tx.clone(), ctx.clone(), key.to_owned(), url.to_owned());
|
||||||
|
self.rt.spawn(async move {
|
||||||
|
let bytes = download(&http, &url).await;
|
||||||
|
let _ = tx.send(Loaded::Clip { key, bytes });
|
||||||
|
ctx.request_repaint();
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
fn play_bytes(&mut self, bytes: Arc<[u8]>, volume: f32, start_at: f32) -> Result<(), String> {
|
||||||
if self.output.is_none() {
|
if self.output.is_none() {
|
||||||
self.output = Some(OutputStream::try_default().map_err(|e| e.to_string())?);
|
self.output = Some(OutputStream::try_default().map_err(|e| e.to_string())?);
|
||||||
}
|
}
|
||||||
@@ -94,9 +232,112 @@ impl AudioPlayer {
|
|||||||
};
|
};
|
||||||
let sink = Sink::try_new(handle).map_err(|e| e.to_string())?;
|
let sink = Sink::try_new(handle).map_err(|e| e.to_string())?;
|
||||||
let source = Decoder::new(Cursor::new(bytes)).map_err(|e| e.to_string())?;
|
let source = Decoder::new(Cursor::new(bytes)).map_err(|e| e.to_string())?;
|
||||||
sink.set_volume(PREVIEW_VOLUME);
|
sink.set_volume(volume);
|
||||||
sink.append(source);
|
sink.append(source);
|
||||||
|
if start_at > 0.0 {
|
||||||
|
if let Err(err) = sink.try_seek(Duration::from_secs_f32(start_at)) {
|
||||||
|
warn!("Audio seek failed: {err}");
|
||||||
|
}
|
||||||
|
}
|
||||||
self.sink = Some(sink);
|
self.sink = Some(sink);
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async fn download(http: &reqwest::Client, url: &str) -> Result<Arc<[u8]>, String> {
|
||||||
|
let response = http.get(url).send().await.and_then(|r| r.error_for_status()).map_err(|e| e.to_string())?;
|
||||||
|
response.bytes().await.map(|b| Arc::from(b.as_ref())).map_err(|e| e.to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Decodes a whole file into its waveform: the loudest sample of each window, folded down to
|
||||||
|
/// [`PEAK_COUNT`] peaks and scaled so the loudest is 1.
|
||||||
|
fn decode_waveform(bytes: Arc<[u8]>) -> Option<Waveform> {
|
||||||
|
let decoder = match Decoder::new(Cursor::new(bytes)) {
|
||||||
|
Ok(d) => d,
|
||||||
|
Err(err) => {
|
||||||
|
warn!("Waveform decode failed: {err}");
|
||||||
|
return None;
|
||||||
|
},
|
||||||
|
};
|
||||||
|
let (channels, rate) = (usize::from(decoder.channels().max(1)), decoder.sample_rate().max(1));
|
||||||
|
let window = WINDOW_FRAMES * channels;
|
||||||
|
let (mut windows, mut loudest, mut count) = (Vec::new(), 0_u16, 0_usize);
|
||||||
|
for sample in decoder {
|
||||||
|
loudest = loudest.max(sample.unsigned_abs());
|
||||||
|
count += 1;
|
||||||
|
if count % window == 0 {
|
||||||
|
windows.push(f32::from(loudest));
|
||||||
|
loudest = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if count % window != 0 {
|
||||||
|
windows.push(f32::from(loudest));
|
||||||
|
}
|
||||||
|
if count == 0 {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
Some(Waveform { peaks: normalized(&resample(&windows, PEAK_COUNT)), duration: (count / channels) as f32 / rate as f32 })
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `values` folded (by maximum) or stretched to exactly `count` entries.
|
||||||
|
pub fn resample(values: &[f32], count: usize) -> Vec<f32> {
|
||||||
|
if values.is_empty() || count == 0 {
|
||||||
|
return vec![0.0; count];
|
||||||
|
}
|
||||||
|
(0..count)
|
||||||
|
.map(|i| {
|
||||||
|
let start = i * values.len() / count;
|
||||||
|
let end = ((i + 1) * values.len() / count).max(start + 1).min(values.len());
|
||||||
|
values[start..end].iter().copied().fold(0.0, f32::max)
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn normalized(values: &[f32]) -> Vec<f32> {
|
||||||
|
let max = values.iter().copied().fold(0.0, f32::max);
|
||||||
|
if max <= 0.0 {
|
||||||
|
return values.to_vec();
|
||||||
|
}
|
||||||
|
values.iter().map(|v| v / max).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resampling_keeps_the_loudest_of_each_span() {
|
||||||
|
assert_eq!(resample(&[0.1, 0.9, 0.2, 0.4], 2), [0.9, 0.4]);
|
||||||
|
assert_eq!(resample(&[0.5, 1.0], 4), [0.5, 0.5, 1.0, 1.0], "short inputs stretch");
|
||||||
|
assert_eq!(resample(&[], 3), [0.0, 0.0, 0.0]);
|
||||||
|
assert_eq!(normalized(&[0.0, 2.0, 1.0]), [0.0, 1.0, 0.5]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn decodes_a_wav_into_peaks_and_duration() {
|
||||||
|
// One second of 8 kHz mono: silence, then a loud half.
|
||||||
|
let rate = 8000_u32;
|
||||||
|
let samples: Vec<i16> = (0..rate).map(|i| if i < rate / 2 { 0 } else { 16000 }).collect();
|
||||||
|
let mut wav = Vec::new();
|
||||||
|
wav.extend_from_slice(b"RIFF");
|
||||||
|
wav.extend_from_slice(&(36 + samples.len() as u32 * 2).to_le_bytes());
|
||||||
|
wav.extend_from_slice(b"WAVEfmt ");
|
||||||
|
wav.extend_from_slice(&16_u32.to_le_bytes());
|
||||||
|
wav.extend_from_slice(&1_u16.to_le_bytes());
|
||||||
|
wav.extend_from_slice(&1_u16.to_le_bytes());
|
||||||
|
wav.extend_from_slice(&rate.to_le_bytes());
|
||||||
|
wav.extend_from_slice(&(rate * 2).to_le_bytes());
|
||||||
|
wav.extend_from_slice(&2_u16.to_le_bytes());
|
||||||
|
wav.extend_from_slice(&16_u16.to_le_bytes());
|
||||||
|
wav.extend_from_slice(b"data");
|
||||||
|
wav.extend_from_slice(&(samples.len() as u32 * 2).to_le_bytes());
|
||||||
|
for s in &samples {
|
||||||
|
wav.extend_from_slice(&s.to_le_bytes());
|
||||||
|
}
|
||||||
|
let waveform = decode_waveform(Arc::from(wav)).expect("decodes");
|
||||||
|
assert!((waveform.duration - 1.0).abs() < 0.01, "duration {}", waveform.duration);
|
||||||
|
assert_eq!(waveform.peaks.len(), PEAK_COUNT);
|
||||||
|
assert_eq!(waveform.peaks[0], 0.0, "starts silent");
|
||||||
|
assert_eq!(*waveform.peaks.last().unwrap(), 1.0, "ends loud");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|||||||
@@ -38,7 +38,7 @@ use tokens::tokens::media_files::MediaFileToken;
|
|||||||
use crate::feed::types::MediaKind;
|
use crate::feed::types::MediaKind;
|
||||||
use crate::models::{self, ModelInfo};
|
use crate::models::{self, ModelInfo};
|
||||||
use session::Credentials;
|
use session::Credentials;
|
||||||
use wire::{BatchMedia, CharactersPage, MediaFile, PromptResponse, SessionInfo, SessionJobs, SessionUser, UploadResponse, UserMediaList};
|
use wire::{AudioModels, BatchMedia, CharactersPage, MediaFile, PromptResponse, SessionInfo, SessionJobs, SessionUser, UploadResponse, UserMediaList};
|
||||||
|
|
||||||
/// Items per library page (`useGalleryData`'s `PAGE_SIZE`).
|
/// Items per library page (`useGalleryData`'s `PAGE_SIZE`).
|
||||||
pub const LIBRARY_PAGE_SIZE: u32 = 40;
|
pub const LIBRARY_PAGE_SIZE: u32 = 40;
|
||||||
@@ -57,6 +57,7 @@ const USER_AGENT: &str = "storyteller-client/1.0";
|
|||||||
pub enum Modality {
|
pub enum Modality {
|
||||||
Image,
|
Image,
|
||||||
Video,
|
Video,
|
||||||
|
Audio,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Modality {
|
impl Modality {
|
||||||
@@ -64,6 +65,7 @@ impl Modality {
|
|||||||
match self {
|
match self {
|
||||||
Modality::Image => "image",
|
Modality::Image => "image",
|
||||||
Modality::Video => "video",
|
Modality::Video => "video",
|
||||||
|
Modality::Audio => "audio",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -73,6 +75,7 @@ impl From<MediaKind> for Modality {
|
|||||||
match kind {
|
match kind {
|
||||||
MediaKind::Image => Modality::Image,
|
MediaKind::Image => Modality::Image,
|
||||||
MediaKind::Video => Modality::Video,
|
MediaKind::Video => Modality::Video,
|
||||||
|
MediaKind::Audio => Modality::Audio,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -102,6 +105,8 @@ pub enum Event {
|
|||||||
EnqueueFailed {
|
EnqueueFailed {
|
||||||
modality: Modality,
|
modality: Modality,
|
||||||
message: String,
|
message: String,
|
||||||
|
/// The HTTP status, when the server answered.
|
||||||
|
status: Option<u16>,
|
||||||
},
|
},
|
||||||
Uploaded {
|
Uploaded {
|
||||||
ref_id: u64,
|
ref_id: u64,
|
||||||
@@ -334,7 +339,8 @@ impl Backend {
|
|||||||
|
|
||||||
/// Fetches the OmniGen model listing for `modality`.
|
/// Fetches the OmniGen model listing for `modality`.
|
||||||
pub fn load_models(&self, modality: Modality) {
|
pub fn load_models(&self, modality: Modality) {
|
||||||
let host = self.host.clone();
|
let api = self.api();
|
||||||
|
let host = api.host.clone();
|
||||||
self.spawn(move |tx| async move {
|
self.spawn(move |tx| async move {
|
||||||
let mut last_err = String::new();
|
let mut last_err = String::new();
|
||||||
for attempt in 0..MODEL_LIST_ATTEMPTS {
|
for attempt in 0..MODEL_LIST_ATTEMPTS {
|
||||||
@@ -342,15 +348,16 @@ impl Backend {
|
|||||||
tokio::time::sleep(MODEL_LIST_BACKOFF).await;
|
tokio::time::sleep(MODEL_LIST_BACKOFF).await;
|
||||||
}
|
}
|
||||||
let result = match modality {
|
let result = match modality {
|
||||||
Modality::Image => omni_gen_list_image_models(OmniGenListImageModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::image_model).collect::<Vec<_>>()),
|
Modality::Image => omni_gen_list_image_models(OmniGenListImageModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::image_model).collect::<Vec<_>>()).map_err(|e| api_message(&e)),
|
||||||
Modality::Video => omni_gen_list_video_models(OmniGenListVideoModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::video_model).collect::<Vec<_>>()),
|
Modality::Video => omni_gen_list_video_models(OmniGenListVideoModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::video_model).collect::<Vec<_>>()).map_err(|e| api_message(&e)),
|
||||||
|
Modality::Audio => api.get_json::<AudioModels>("/v1/omni_gen/models/audio").await.map(|r| r.models.iter().map(models::audio_model).collect::<Vec<_>>()),
|
||||||
};
|
};
|
||||||
match result {
|
match result {
|
||||||
Ok(models) => {
|
Ok(models) => {
|
||||||
let _ = tx.send(Event::Models(modality, Ok(models)));
|
let _ = tx.send(Event::Models(modality, Ok(models)));
|
||||||
return;
|
return;
|
||||||
},
|
},
|
||||||
Err(err) => last_err = api_message(&err),
|
Err(err) => last_err = err,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
error!("Listing {modality:?} models failed: {last_err}");
|
error!("Listing {modality:?} models failed: {last_err}");
|
||||||
@@ -379,7 +386,7 @@ impl Backend {
|
|||||||
let api = self.api();
|
let api = self.api();
|
||||||
self.spawn(move |tx| async move {
|
self.spawn(move |tx| async move {
|
||||||
let Some(creds) = api.creds.as_ref() else {
|
let Some(creds) = api.creds.as_ref() else {
|
||||||
let _ = tx.send(Event::EnqueueFailed { modality, message: "Please sign in to generate.".to_owned() });
|
let _ = tx.send(Event::EnqueueFailed { modality, message: "Please sign in to generate.".to_owned(), status: None });
|
||||||
return;
|
return;
|
||||||
};
|
};
|
||||||
fields.entry("idempotency_token").or_insert_with(|| Value::String(uuid::Uuid::new_v4().to_string()));
|
fields.entry("idempotency_token").or_insert_with(|| Value::String(uuid::Uuid::new_v4().to_string()));
|
||||||
@@ -388,14 +395,14 @@ impl Backend {
|
|||||||
Ok(response) => {
|
Ok(response) => {
|
||||||
let tokens = job_tokens(&response);
|
let tokens = job_tokens(&response);
|
||||||
if tokens.is_empty() {
|
if tokens.is_empty() {
|
||||||
let _ = tx.send(Event::EnqueueFailed { modality, message: "The server didn't start a job.".to_owned() });
|
let _ = tx.send(Event::EnqueueFailed { modality, message: "The server didn't start a job.".to_owned(), status: None });
|
||||||
} else {
|
} else {
|
||||||
let _ = tx.send(Event::Enqueued { modality, job_tokens: tokens, meta });
|
let _ = tx.send(Event::Enqueued { modality, job_tokens: tokens, meta });
|
||||||
}
|
}
|
||||||
api.send_credits(&tx).await;
|
api.send_credits(&tx).await;
|
||||||
},
|
},
|
||||||
Err(err) => {
|
Err(err) => {
|
||||||
let _ = tx.send(Event::EnqueueFailed { modality, message: api_message(&err) });
|
let _ = tx.send(Event::EnqueueFailed { modality, message: api_message(&err), status: api_status(&err) });
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -750,7 +757,7 @@ impl Api {
|
|||||||
}
|
}
|
||||||
let info = self.get_json::<FileResponse>(&format!("/v1/media_files/file/{token}")).await.ok();
|
let info = self.get_json::<FileResponse>(&format!("/v1/media_files/file/{token}")).await.ok();
|
||||||
let file = info.map(|r| r.media_file);
|
let file = info.map(|r| r.media_file);
|
||||||
UploadedMedia { duration_secs: file.as_ref().and_then(|f| f.maybe_duration_millis).map_or(0.0, |ms| ms as f32 / 1000.0), thumbnail: file.as_ref().and_then(|f| f.media_links.thumbnail(256)), full_url: file.as_ref().map(|f| f.media_links.cdn_url.clone()).filter(|u| !u.is_empty()), token }
|
UploadedMedia { duration_secs: file.as_ref().and_then(|f| f.maybe_duration_millis).map_or(0.0, |ms| ms as f32 / 1000.0), thumbnail: file.as_ref().and_then(|f| f.thumbnail(256)), full_url: file.as_ref().map(|f| f.media_links.cdn_url.clone()).filter(|u| !u.is_empty()), token }
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -793,6 +800,24 @@ pub fn api_message(err: &StorytellerError) -> String {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// The HTTP status behind a client error, when there was a response.
|
||||||
|
fn api_status(err: &StorytellerError) -> Option<u16> {
|
||||||
|
let StorytellerError::Api(api) = err else {
|
||||||
|
return None;
|
||||||
|
};
|
||||||
|
Some(match api {
|
||||||
|
ApiError::InvalidRequest(_) => 400,
|
||||||
|
ApiError::Unauthorized(_) => 401,
|
||||||
|
ApiError::PaymentRequired(_) => 402,
|
||||||
|
ApiError::Forbidden(_) => 403,
|
||||||
|
ApiError::NotFound(_) => 404,
|
||||||
|
ApiError::TooManyRequests(_) => 429,
|
||||||
|
ApiError::InternalServerError { .. } => 500,
|
||||||
|
ApiError::UncategorizedBadResponseWithStatusAndBody { status_code, .. } => status_code.as_u16(),
|
||||||
|
_ => return None,
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
fn api_error_message(err: &ApiError) -> String {
|
fn api_error_message(err: &ApiError) -> String {
|
||||||
let body = match err {
|
let body = match err {
|
||||||
ApiError::InvalidRequest(b) | ApiError::Unauthorized(b) | ApiError::PaymentRequired(b) | ApiError::Forbidden(b) | ApiError::NotFound(b) | ApiError::TooManyRequests(b) => Some(b.as_str()),
|
ApiError::InvalidRequest(b) | ApiError::Unauthorized(b) | ApiError::PaymentRequired(b) | ApiError::Forbidden(b) | ApiError::NotFound(b) | ApiError::TooManyRequests(b) => Some(b.as_str()),
|
||||||
|
|||||||
@@ -81,10 +81,56 @@ pub struct MediaFile {
|
|||||||
pub maybe_model_type: Option<String>,
|
pub maybe_model_type: Option<String>,
|
||||||
pub maybe_origin_model_type: Option<String>,
|
pub maybe_origin_model_type: Option<String>,
|
||||||
pub maybe_duration_millis: Option<u64>,
|
pub maybe_duration_millis: Option<u64>,
|
||||||
|
pub maybe_title: Option<String>,
|
||||||
pub media_links: MediaLinks,
|
pub media_links: MediaLinks,
|
||||||
pub created_at: String,
|
pub created_at: String,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
impl MediaFile {
|
||||||
|
/// The file's still thumbnail; audio files have none (their CDN link is the audio itself).
|
||||||
|
pub fn thumbnail(&self, width: u32) -> Option<String> {
|
||||||
|
if self.media_class.as_deref() == Some("audio") {
|
||||||
|
return None;
|
||||||
|
}
|
||||||
|
self.media_links.thumbnail(width)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `GET /v1/omni_gen/models/audio`. The client has no binding for it (and the API's type only
|
||||||
|
/// serializes), so it's read here; capability flags are left out when false.
|
||||||
|
#[derive(Debug, Default, Deserialize)]
|
||||||
|
#[serde(default)]
|
||||||
|
pub struct AudioModels {
|
||||||
|
pub success: bool,
|
||||||
|
pub models: Vec<AudioModel>,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug, Default, Deserialize)]
|
||||||
|
#[serde(default)]
|
||||||
|
pub struct AudioModel {
|
||||||
|
pub model: String,
|
||||||
|
pub model_creator: Option<String>,
|
||||||
|
pub full_name: Option<String>,
|
||||||
|
pub extra_info_short: Option<String>,
|
||||||
|
pub text_prompt_supported: Option<bool>,
|
||||||
|
pub style_prompt_supported: bool,
|
||||||
|
pub audio_references_supported: bool,
|
||||||
|
pub audio_references_max: Option<u16>,
|
||||||
|
pub image_references_supported: bool,
|
||||||
|
pub image_references_max: Option<u16>,
|
||||||
|
pub keep_lyrics_supported: bool,
|
||||||
|
pub instrumental_toggle_supported: bool,
|
||||||
|
pub loopable_toggle_supported: bool,
|
||||||
|
pub bpm_supported: bool,
|
||||||
|
pub musical_key_supported: bool,
|
||||||
|
pub sample_rate_hz_options: Vec<u32>,
|
||||||
|
pub sample_rate_hz_default: Option<u32>,
|
||||||
|
pub speed_supported: bool,
|
||||||
|
pub volume_supported: bool,
|
||||||
|
pub pitch_supported: bool,
|
||||||
|
pub is_disabled: bool,
|
||||||
|
}
|
||||||
|
|
||||||
/// `GET /v1/media_files/batch_gen_redux/{token}`.
|
/// `GET /v1/media_files/batch_gen_redux/{token}`.
|
||||||
#[derive(Debug, Default, Deserialize)]
|
#[derive(Debug, Default, Deserialize)]
|
||||||
#[serde(default)]
|
#[serde(default)]
|
||||||
|
|||||||
@@ -1,16 +1,17 @@
|
|||||||
//! The generation feed as a masonry grid (`GenerationGridView`) or a list (`GenerationListView`):
|
//! The generation feed as a masonry grid (`GenerationGridView`) or a list (`GenerationListView`):
|
||||||
//! running jobs first-class alongside failures and finished media, newest first, loading more as
|
//! running jobs first-class alongside failures and finished media, newest first, loading more as
|
||||||
//! the user scrolls.
|
//! the user scrolls. Audio plays in place, with its waveform.
|
||||||
|
|
||||||
use std::collections::{HashMap, HashSet};
|
use std::collections::{HashMap, HashSet};
|
||||||
|
|
||||||
use egui::{Align2, Color32, CornerRadius, FontFamily, FontId, Id, Mesh, Rect, Sense, Stroke, Ui, pos2, vec2};
|
use egui::{Align2, Color32, CornerRadius, FontFamily, FontId, Id, Mesh, Rect, Sense, Stroke, Ui, pos2, vec2};
|
||||||
|
|
||||||
|
use crate::backend::audio::AudioPlayer;
|
||||||
use crate::backend::media_cache::{Lookup, MediaCache};
|
use crate::backend::media_cache::{Lookup, MediaCache};
|
||||||
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob, pending_status, time_ago};
|
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob, pending_status, time_ago};
|
||||||
use crate::theme;
|
use crate::theme;
|
||||||
use crate::ui::icons::{self, Icon};
|
use crate::ui::icons::{self, Icon};
|
||||||
use crate::ui::{creator_icons, widgets};
|
use crate::ui::{creator_icons, waveform, widgets};
|
||||||
|
|
||||||
/// `react-masonry-css` gutter and the cap on tall portraits (≈ 5:7).
|
/// `react-masonry-css` gutter and the cap on tall portraits (≈ 5:7).
|
||||||
const GAP: f32 = 8.0;
|
const GAP: f32 = 8.0;
|
||||||
@@ -18,6 +19,8 @@ const MAX_RATIO: f32 = 1.4;
|
|||||||
/// Start loading the next page this far before the end.
|
/// Start loading the next page this far before the end.
|
||||||
const LOAD_MORE_MARGIN: f32 = 400.0;
|
const LOAD_MORE_MARGIN: f32 = 400.0;
|
||||||
const LIST_THUMB: f32 = 100.0;
|
const LIST_THUMB: f32 = 100.0;
|
||||||
|
/// The list's compact audio player is at most `max-w-md` wide.
|
||||||
|
const LIST_PLAYER_MAX: f32 = 448.0;
|
||||||
|
|
||||||
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
|
||||||
pub enum ViewMode {
|
pub enum ViewMode {
|
||||||
@@ -86,7 +89,7 @@ impl Entry<'_> {
|
|||||||
#[derive(Clone, Default)]
|
#[derive(Clone, Default)]
|
||||||
pub struct RatioCache(HashMap<String, f32>);
|
pub struct RatioCache(HashMap<String, f32>);
|
||||||
|
|
||||||
pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &mut RatioCache, names: &dyn ModelNames) -> Vec<FeedAction> {
|
pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &mut RatioCache, audio: &mut AudioPlayer, names: &dyn ModelNames) -> Vec<FeedAction> {
|
||||||
let mut entries: Vec<Entry<'_>> = view.pending.iter().map(Entry::Pending).chain(view.failed.iter().map(Entry::Failed)).chain(view.items.iter().map(Entry::Item)).collect();
|
let mut entries: Vec<Entry<'_>> = view.pending.iter().map(Entry::Pending).chain(view.failed.iter().map(Entry::Failed)).chain(view.items.iter().map(Entry::Item)).collect();
|
||||||
entries.sort_by_key(|e| std::cmp::Reverse(e.time()));
|
entries.sort_by_key(|e| std::cmp::Reverse(e.time()));
|
||||||
let mut actions = Vec::new();
|
let mut actions = Vec::new();
|
||||||
@@ -113,10 +116,10 @@ pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &m
|
|||||||
match (entry, view.mode) {
|
match (entry, view.mode) {
|
||||||
(Entry::Pending(job), ViewMode::Grid) => pending_card(ui, rect, job, now, names),
|
(Entry::Pending(job), ViewMode::Grid) => pending_card(ui, rect, job, now, names),
|
||||||
(Entry::Failed(job), ViewMode::Grid) => failed_card(ui, view.id, rect, job, cache, names, &mut actions),
|
(Entry::Failed(job), ViewMode::Grid) => failed_card(ui, view.id, rect, job, cache, names, &mut actions),
|
||||||
(Entry::Item(item), ViewMode::Grid) => item_card(ui, view, rect, item, cache, ratios, names, &mut actions),
|
(Entry::Item(item), ViewMode::Grid) => item_card(ui, view, rect, item, cache, ratios, audio, names, &mut actions),
|
||||||
(Entry::Pending(job), ViewMode::List) => pending_row(ui, rect, job, now, names),
|
(Entry::Pending(job), ViewMode::List) => pending_row(ui, rect, job, now, names),
|
||||||
(Entry::Failed(job), ViewMode::List) => failed_row(ui, view.id, rect, job, names, &mut actions),
|
(Entry::Failed(job), ViewMode::List) => failed_row(ui, view.id, rect, job, names, &mut actions),
|
||||||
(Entry::Item(item), ViewMode::List) => item_row(ui, view, rect, item, cache, names, now, &mut actions),
|
(Entry::Item(item), ViewMode::List) => item_row(ui, view, rect, item, cache, audio, names, now, &mut actions),
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -228,31 +231,15 @@ fn failed_card(ui: &mut Ui, id: Id, rect: Rect, job: &FailedJob, cache: &mut Med
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[allow(clippy::too_many_arguments)]
|
#[allow(clippy::too_many_arguments)]
|
||||||
fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache, names: &dyn ModelNames, actions: &mut Vec<FeedAction>) {
|
fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache, audio: &mut AudioPlayer, names: &dyn ModelNames, actions: &mut Vec<FeedAction>) {
|
||||||
let resp = ui.interact(rect, view.id.with(("item", &item.token)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
|
let resp = ui.interact(rect, view.id.with(("item", &item.token)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
|
||||||
let hovered = resp.hovered();
|
// The pointer may be on an action button (or the audio player) rather than the card itself.
|
||||||
ui.painter().rect_filled(rect, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
|
let hovered = resp.hovered() || ui.rect_contains_pointer(rect);
|
||||||
// Videos play their animated preview when previews are on (the still shows while it loads).
|
let is_audio = item.kind == MediaKind::Audio;
|
||||||
let animated = item.animated.as_deref().filter(|_| view.autoplay && item.kind == MediaKind::Video).map(|u| cache.get_animated(ui.ctx(), u));
|
if is_audio {
|
||||||
let lookup = match animated {
|
audio_face(ui, view, rect, item, audio);
|
||||||
Some(Lookup::Ready(t)) => Some(Lookup::Ready(t)),
|
} else {
|
||||||
_ => item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)),
|
visual_face(ui, view, rect, item, cache, ratios);
|
||||||
};
|
|
||||||
match lookup {
|
|
||||||
Some(Lookup::Ready(t)) => {
|
|
||||||
let size = t.size_vec2();
|
|
||||||
if size.x > 0.0 && !ratios.0.contains_key(&item.token) {
|
|
||||||
ratios.0.insert(item.token.clone(), size.y / size.x);
|
|
||||||
}
|
|
||||||
egui::Image::new(&t).uv(cover_uv_for(size, rect.size())).corner_radius(theme::RADIUS).paint_at(ui, rect);
|
|
||||||
},
|
|
||||||
Some(Lookup::Loading) => shimmer(ui, rect),
|
|
||||||
_ => icons::paint(ui.painter(), Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), if item.kind == MediaKind::Video { Icon::Video } else { Icon::Image }, theme::FAINT),
|
|
||||||
}
|
|
||||||
if item.kind == MediaKind::Video && !(view.autoplay && item.animated.is_some()) {
|
|
||||||
let badge = Rect::from_center_size(rect.center(), vec2(40.0, 40.0));
|
|
||||||
ui.painter().circle_filled(badge.center(), 20.0, Color32::from_black_alpha(110));
|
|
||||||
icons::paint(ui.painter(), badge.shrink(12.0).translate(vec2(1.5, 0.0)), Icon::Play, theme::fade(Color32::WHITE, 0.9));
|
|
||||||
}
|
}
|
||||||
let selected = view.selected.contains(&item.token);
|
let selected = view.selected.contains(&item.token);
|
||||||
let ring = if selected {
|
let ring = if selected {
|
||||||
@@ -284,25 +271,17 @@ fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cach
|
|||||||
p.galley(pos2(badge.left() + 20.0, badge.center().y - galley.size().y / 2.0), galley, Color32::WHITE);
|
p.galley(pos2(badge.left() + 20.0, badge.center().y - galley.size().y / 2.0), galley, Color32::WHITE);
|
||||||
}
|
}
|
||||||
let mut action_clicked = false;
|
let mut action_clicked = false;
|
||||||
// The pointer may be on an action button rather than the card itself.
|
if hovered && !view.selecting {
|
||||||
if (hovered || ui.rect_contains_pointer(rect)) && !view.selecting {
|
// Audio keeps its player at the bottom, so its chips and actions go on top.
|
||||||
let footer = bottom_gradient(ui, rect, 56.0, 178);
|
let band = edge_fade(ui, rect, 56.0, 178, is_audio);
|
||||||
let chip_y = footer.bottom() - 18.0;
|
let chip_y = if is_audio { band.top() + 18.0 } else { band.bottom() - 18.0 };
|
||||||
let mut x = footer.left() + 8.0;
|
let mut x = band.left() + 8.0;
|
||||||
x = chip(ui, pos2(x, chip_y), Some(if item.kind == MediaKind::Video { Icon::Video } else { Icon::Image }), None, item.kind.label());
|
x = chip(ui, pos2(x, chip_y), Some(kind_icon(item.kind)), None, item.kind.label());
|
||||||
if let Some(model) = item.model_id.as_deref() {
|
if let Some(model) = item.model_id.as_deref() {
|
||||||
chip(ui, pos2(x + 6.0, chip_y), None, Some(&names.creator(model)), &truncate(&names.display_name(model), 16));
|
chip(ui, pos2(x + 6.0, chip_y), None, Some(&names.creator(model)), &truncate(&names.display_name(model), 16));
|
||||||
}
|
}
|
||||||
let mut buttons: Vec<(Icon, &str, FeedAction)> = Vec::new();
|
let buttons = item_actions(view, item, None);
|
||||||
if item.prompt_token.is_some() {
|
let pill = Rect::from_min_size(pos2(band.right() - 8.0 - 8.0 - 28.0 * buttons.len() as f32, chip_y - 18.0), vec2(8.0 + 28.0 * buttons.len() as f32, 36.0));
|
||||||
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
|
|
||||||
}
|
|
||||||
if view.make_video && item.kind == MediaKind::Image {
|
|
||||||
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
|
|
||||||
}
|
|
||||||
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
|
|
||||||
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
|
|
||||||
let pill = Rect::from_min_size(pos2(footer.right() - 8.0 - 8.0 - 28.0 * buttons.len() as f32, chip_y - 18.0), vec2(8.0 + 28.0 * buttons.len() as f32, 36.0));
|
|
||||||
ui.painter().rect_filled(pill, 0.0, Color32::from_black_alpha(153));
|
ui.painter().rect_filled(pill, 0.0, Color32::from_black_alpha(153));
|
||||||
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(pill.shrink(4.0)).layout(egui::Layout::left_to_right(egui::Align::Center)));
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(pill.shrink(4.0)).layout(egui::Layout::left_to_right(egui::Align::Center)));
|
||||||
child.spacing_mut().item_spacing.x = 0.0;
|
child.spacing_mut().item_spacing.x = 0.0;
|
||||||
@@ -318,6 +297,87 @@ fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cach
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// An image or video card's face: the thumbnail (or animated preview) and a video's play badge.
|
||||||
|
fn visual_face(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache) {
|
||||||
|
ui.painter().rect_filled(rect, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
|
||||||
|
// Videos play their animated preview when previews are on (the still shows while it loads).
|
||||||
|
let animated = item.animated.as_deref().filter(|_| view.autoplay && item.kind == MediaKind::Video).map(|u| cache.get_animated(ui.ctx(), u));
|
||||||
|
let lookup = match animated {
|
||||||
|
Some(Lookup::Ready(t)) => Some(Lookup::Ready(t)),
|
||||||
|
_ => item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)),
|
||||||
|
};
|
||||||
|
match lookup {
|
||||||
|
Some(Lookup::Ready(t)) => {
|
||||||
|
let size = t.size_vec2();
|
||||||
|
if size.x > 0.0 && !ratios.0.contains_key(&item.token) {
|
||||||
|
ratios.0.insert(item.token.clone(), size.y / size.x);
|
||||||
|
}
|
||||||
|
egui::Image::new(&t).uv(cover_uv_for(size, rect.size())).corner_radius(theme::RADIUS).paint_at(ui, rect);
|
||||||
|
},
|
||||||
|
Some(Lookup::Loading) => shimmer(ui, rect),
|
||||||
|
_ => icons::paint(ui.painter(), Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), kind_icon(item.kind), theme::FAINT),
|
||||||
|
}
|
||||||
|
if item.kind == MediaKind::Video && !(view.autoplay && item.animated.is_some()) {
|
||||||
|
let badge = Rect::from_center_size(rect.center(), vec2(40.0, 40.0));
|
||||||
|
ui.painter().circle_filled(badge.center(), 20.0, Color32::from_black_alpha(110));
|
||||||
|
icons::paint(ui.painter(), badge.shrink(12.0).translate(vec2(1.5, 0.0)), Icon::Play, theme::fade(Color32::WHITE, 0.9));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// An audio card's face (`GalleryCard`'s audio branch, a square tile): the title, a music note
|
||||||
|
/// and the waveform player along the bottom.
|
||||||
|
fn audio_face(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, audio: &mut AudioPlayer) {
|
||||||
|
let mut mesh = Mesh::default();
|
||||||
|
let (top, bottom) = (theme::fade(theme::ACCENT, 0.14), theme::fade(theme::CONTROLS, 0.6));
|
||||||
|
mesh.colored_vertex(rect.left_top(), top);
|
||||||
|
mesh.colored_vertex(rect.right_top(), top);
|
||||||
|
mesh.colored_vertex(rect.right_bottom(), bottom);
|
||||||
|
mesh.colored_vertex(rect.left_bottom(), bottom);
|
||||||
|
mesh.add_triangle(0, 1, 2);
|
||||||
|
mesh.add_triangle(0, 2, 3);
|
||||||
|
ui.painter().add(mesh);
|
||||||
|
// Clear of the selection checkbox when selecting.
|
||||||
|
let inset = if view.selecting { 36.0 } else { 12.0 };
|
||||||
|
let title = item.prompt_token.as_deref().and_then(|t| view.prompts.get(t)).cloned().unwrap_or_else(|| fallback_title(item));
|
||||||
|
let galley = clamp_lines(ui, &title, 13.0, theme::fade(Color32::WHITE, 0.85), rect.width() - inset - 12.0, 2);
|
||||||
|
ui.painter().galley(pos2(rect.left() + inset, rect.top() + 12.0), galley, Color32::WHITE);
|
||||||
|
waveform::music_tile(ui, Rect::from_center_size(rect.center() - vec2(0.0, 12.0), vec2(48.0, 48.0)), true);
|
||||||
|
let bar = Rect::from_min_max(pos2(rect.left() + 12.0, rect.bottom() - 56.0), pos2(rect.right() - 12.0, rect.bottom() - 12.0));
|
||||||
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar));
|
||||||
|
waveform::player(&mut child, view.id.with(("player", &item.token)), audio, &item.full_url, item.duration_secs, false);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The hover actions for an item: copy prompt (list rows, when the text is known), recreate,
|
||||||
|
/// make video, share and download. Audio only shares and downloads (`Audio v1`).
|
||||||
|
fn item_actions<'a>(view: &FeedView<'_>, item: &FeedItem, prompt: Option<&String>) -> Vec<(Icon, &'a str, FeedAction)> {
|
||||||
|
let mut buttons = Vec::new();
|
||||||
|
if let Some(text) = prompt {
|
||||||
|
buttons.push((Icon::Copy, "Copy prompt", FeedAction::CopyPrompt(text.clone())));
|
||||||
|
}
|
||||||
|
if item.prompt_token.is_some() && item.kind != MediaKind::Audio {
|
||||||
|
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
|
||||||
|
}
|
||||||
|
if view.make_video && item.kind == MediaKind::Image {
|
||||||
|
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
|
||||||
|
}
|
||||||
|
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
|
||||||
|
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
|
||||||
|
buttons
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What an item is called without its prompt: its title, else "Audio Generation" and the like.
|
||||||
|
fn fallback_title(item: &FeedItem) -> String {
|
||||||
|
item.title.clone().unwrap_or_else(|| format!("{} Generation", item.kind.label()))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn kind_icon(kind: MediaKind) -> Icon {
|
||||||
|
match kind {
|
||||||
|
MediaKind::Image => Icon::Image,
|
||||||
|
MediaKind::Video => Icon::Video,
|
||||||
|
MediaKind::Audio => Icon::Music,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
fn pending_row(ui: &mut Ui, rect: Rect, job: &PendingJob, now: i64, names: &dyn ModelNames) {
|
fn pending_row(ui: &mut Ui, rect: Rect, job: &PendingJob, now: i64, names: &dyn ModelNames) {
|
||||||
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
|
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
|
||||||
ui.painter().rect_filled(thumb, theme::RADIUS, Color32::from_white_alpha(8));
|
ui.painter().rect_filled(thumb, theme::RADIUS, Color32::from_white_alpha(8));
|
||||||
@@ -367,7 +427,7 @@ fn failed_row(ui: &mut Ui, id: Id, rect: Rect, job: &FailedJob, names: &dyn Mode
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[allow(clippy::too_many_arguments)]
|
#[allow(clippy::too_many_arguments)]
|
||||||
fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, names: &dyn ModelNames, now: i64, actions: &mut Vec<FeedAction>) {
|
fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, audio: &mut AudioPlayer, names: &dyn ModelNames, now: i64, actions: &mut Vec<FeedAction>) {
|
||||||
let resp = ui.interact(rect, view.id.with(("row", &item.token)), Sense::click());
|
let resp = ui.interact(rect, view.id.with(("row", &item.token)), Sense::click());
|
||||||
let hovered = resp.hovered();
|
let hovered = resp.hovered();
|
||||||
if hovered {
|
if hovered {
|
||||||
@@ -375,7 +435,10 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
|
|||||||
}
|
}
|
||||||
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
|
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
|
||||||
ui.painter().rect_filled(thumb, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
|
ui.painter().rect_filled(thumb, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
|
||||||
if let Some(Lookup::Ready(t)) = item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
|
let is_audio = item.kind == MediaKind::Audio;
|
||||||
|
if is_audio {
|
||||||
|
waveform::music_tile(ui, thumb, false);
|
||||||
|
} else if let Some(Lookup::Ready(t)) = item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
|
||||||
egui::Image::new(&t).uv(crate::prompt_box::deck::cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, thumb);
|
egui::Image::new(&t).uv(crate::prompt_box::deck::cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, thumb);
|
||||||
}
|
}
|
||||||
if view.selected.contains(&item.token) {
|
if view.selected.contains(&item.token) {
|
||||||
@@ -386,7 +449,8 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
|
|||||||
let prompt = item.prompt_token.as_deref().and_then(|t| view.prompts.get(t));
|
let prompt = item.prompt_token.as_deref().and_then(|t| view.prompts.get(t));
|
||||||
match prompt {
|
match prompt {
|
||||||
Some(text) => {
|
Some(text) => {
|
||||||
let g = clamp_lines(ui, text, 14.0, theme::INK, w, 3);
|
// Audio keeps room for its player under the text.
|
||||||
|
let g = clamp_lines(ui, text, 14.0, theme::INK, w, if is_audio { 2 } else { 3 });
|
||||||
ui.painter().galley(pos2(x, thumb.top()), g, theme::INK);
|
ui.painter().galley(pos2(x, thumb.top()), g, theme::INK);
|
||||||
},
|
},
|
||||||
None if item.prompt_token.is_some() => {
|
None if item.prompt_token.is_some() => {
|
||||||
@@ -395,9 +459,13 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
|
|||||||
}
|
}
|
||||||
},
|
},
|
||||||
None => {
|
None => {
|
||||||
ui.painter().text(pos2(x, thumb.top() + 8.0), Align2::LEFT_CENTER, format!("{} Generation", item.kind.label()), FontId::new(14.0, FontFamily::Proportional), theme::MUTED);
|
ui.painter().text(pos2(x, thumb.top() + 8.0), Align2::LEFT_CENTER, fallback_title(item), FontId::new(14.0, FontFamily::Proportional), theme::MUTED);
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
|
if is_audio {
|
||||||
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(Rect::from_min_size(pos2(x, thumb.top() + 44.0), vec2(w.min(LIST_PLAYER_MAX), 32.0))));
|
||||||
|
waveform::player(&mut child, view.id.with(("row-player", &item.token)), audio, &item.full_url, item.duration_secs, true);
|
||||||
|
}
|
||||||
let meta_y = thumb.bottom() - 10.0;
|
let meta_y = thumb.bottom() - 10.0;
|
||||||
let mut mx = x;
|
let mut mx = x;
|
||||||
if let Some(model) = item.model_id.as_deref() {
|
if let Some(model) = item.model_id.as_deref() {
|
||||||
@@ -408,18 +476,7 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
|
|||||||
ui.painter().text(pos2(rect.right(), thumb.top() + 8.0), Align2::RIGHT_CENTER, time_ago(item.created_at, now), FontId::new(11.0, FontFamily::Proportional), theme::FAINT);
|
ui.painter().text(pos2(rect.right(), thumb.top() + 8.0), Align2::RIGHT_CENTER, time_ago(item.created_at, now), FontId::new(11.0, FontFamily::Proportional), theme::FAINT);
|
||||||
let mut action_clicked = false;
|
let mut action_clicked = false;
|
||||||
if hovered && !view.selecting {
|
if hovered && !view.selecting {
|
||||||
let mut buttons: Vec<(Icon, &str, FeedAction)> = Vec::new();
|
let buttons = item_actions(view, item, prompt);
|
||||||
if let Some(text) = prompt {
|
|
||||||
buttons.push((Icon::Copy, "Copy prompt", FeedAction::CopyPrompt(text.clone())));
|
|
||||||
}
|
|
||||||
if item.prompt_token.is_some() {
|
|
||||||
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
|
|
||||||
}
|
|
||||||
if view.make_video && item.kind == MediaKind::Image {
|
|
||||||
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
|
|
||||||
}
|
|
||||||
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
|
|
||||||
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
|
|
||||||
let bar = Rect::from_min_size(pos2(rect.right() - 28.0 * buttons.len() as f32, thumb.bottom() - 28.0), vec2(28.0 * buttons.len() as f32, 28.0));
|
let bar = Rect::from_min_size(pos2(rect.right() - 28.0 * buttons.len() as f32, thumb.bottom() - 28.0), vec2(28.0 * buttons.len() as f32, 28.0));
|
||||||
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar).layout(egui::Layout::left_to_right(egui::Align::Center)));
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar).layout(egui::Layout::left_to_right(egui::Align::Center)));
|
||||||
child.spacing_mut().item_spacing.x = 0.0;
|
child.spacing_mut().item_spacing.x = 0.0;
|
||||||
@@ -472,9 +529,15 @@ fn model_chip(ui: &Ui, left_center: egui::Pos2, model_id: &str, names: &dyn Mode
|
|||||||
|
|
||||||
/// Darkens the bottom of `rect` (`bg-gradient-to-t from-black/70`), returning the faded band.
|
/// Darkens the bottom of `rect` (`bg-gradient-to-t from-black/70`), returning the faded band.
|
||||||
fn bottom_gradient(ui: &Ui, rect: Rect, height: f32, alpha: u8) -> Rect {
|
fn bottom_gradient(ui: &Ui, rect: Rect, height: f32, alpha: u8) -> Rect {
|
||||||
let band = Rect::from_min_max(pos2(rect.left(), (rect.bottom() - height).max(rect.top())), rect.max);
|
edge_fade(ui, rect, height, alpha, false)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Darkens `rect` toward its top or bottom edge over `height`, returning the faded band.
|
||||||
|
fn edge_fade(ui: &Ui, rect: Rect, height: f32, alpha: u8, top_edge: bool) -> Rect {
|
||||||
|
let band = if top_edge { Rect::from_min_max(rect.min, pos2(rect.right(), (rect.top() + height).min(rect.bottom()))) } else { Rect::from_min_max(pos2(rect.left(), (rect.bottom() - height).max(rect.top())), rect.max) };
|
||||||
let mut mesh = Mesh::default();
|
let mut mesh = Mesh::default();
|
||||||
let (top, bottom) = (Color32::TRANSPARENT, Color32::from_black_alpha(alpha));
|
let (clear, dark) = (Color32::TRANSPARENT, Color32::from_black_alpha(alpha));
|
||||||
|
let (top, bottom) = if top_edge { (dark, clear) } else { (clear, dark) };
|
||||||
mesh.colored_vertex(band.left_top(), top);
|
mesh.colored_vertex(band.left_top(), top);
|
||||||
mesh.colored_vertex(band.right_top(), top);
|
mesh.colored_vertex(band.right_top(), top);
|
||||||
mesh.colored_vertex(band.right_bottom(), bottom);
|
mesh.colored_vertex(band.right_bottom(), bottom);
|
||||||
|
|||||||
@@ -6,7 +6,7 @@ use std::collections::{HashMap, HashSet};
|
|||||||
use crate::backend::wire::{Job, JobPhase, MediaFile, unix_secs};
|
use crate::backend::wire::{Job, JobPhase, MediaFile, unix_secs};
|
||||||
use crate::backend::EnqueueMeta;
|
use crate::backend::EnqueueMeta;
|
||||||
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob};
|
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob};
|
||||||
use crate::models::Catalog;
|
use crate::models::{self, Catalog};
|
||||||
|
|
||||||
/// Failures older than this are history, not news.
|
/// Failures older than this are history, not news.
|
||||||
const FAILURE_WINDOW_SECS: i64 = 24 * 60 * 60;
|
const FAILURE_WINDOW_SECS: i64 = 24 * 60 * 60;
|
||||||
@@ -56,7 +56,7 @@ impl FeedStore {
|
|||||||
/// Shows the jobs a generate call just started, before the next poll catches up.
|
/// Shows the jobs a generate call just started, before the next poll catches up.
|
||||||
pub fn add_enqueued(&mut self, kind: MediaKind, job_tokens: &[String], meta: &EnqueueMeta, catalog: &Catalog) {
|
pub fn add_enqueued(&mut self, kind: MediaKind, job_tokens: &[String], meta: &EnqueueMeta, catalog: &Catalog) {
|
||||||
let now = chrono::Utc::now().timestamp();
|
let now = chrono::Utc::now().timestamp();
|
||||||
let expected = catalog.find(&meta.model_id).map_or(30.0, |m| m.expected_secs);
|
let expected = catalog.find(&meta.model_id).map_or_else(|| models::expected_secs(&meta.model_id, kind), |m| m.expected_secs);
|
||||||
for token in job_tokens {
|
for token in job_tokens {
|
||||||
self.meta.insert(token.clone(), meta.clone());
|
self.meta.insert(token.clone(), meta.clone());
|
||||||
self.seen_running.insert(token.clone());
|
self.seen_running.insert(token.clone());
|
||||||
@@ -81,7 +81,7 @@ impl FeedStore {
|
|||||||
match job.status.phase() {
|
match job.status.phase() {
|
||||||
JobPhase::Running => {
|
JobPhase::Running => {
|
||||||
self.seen_running.insert(job.job_token.clone());
|
self.seen_running.insert(job.job_token.clone());
|
||||||
let expected = model_id.as_deref().and_then(|id| catalog.find(id)).map_or(if kind == MediaKind::Video { 900.0 } else { 30.0 }, |m| m.expected_secs);
|
let expected = model_id.as_deref().and_then(|id| catalog.find(id)).map_or_else(|| models::expected_secs(model_id.as_deref().unwrap_or_default(), kind), |m| m.expected_secs);
|
||||||
pending.push(PendingJob { job_token: job.job_token.clone(), kind, prompt, model_id, created_at: unix_secs(&job.created_at), server_progress: Some(job.status.progress_percentage), expected_secs: expected, batch_count: meta.map_or(1, |m| m.batch_count) });
|
pending.push(PendingJob { job_token: job.job_token.clone(), kind, prompt, model_id, created_at: unix_secs(&job.created_at), server_progress: Some(job.status.progress_percentage), expected_secs: expected, batch_count: meta.map_or(1, |m| m.batch_count) });
|
||||||
},
|
},
|
||||||
JobPhase::Failed => {
|
JobPhase::Failed => {
|
||||||
@@ -105,7 +105,8 @@ impl FeedStore {
|
|||||||
};
|
};
|
||||||
// The finished card takes the pending card's place: date it by the job.
|
// The finished card takes the pending card's place: date it by the job.
|
||||||
let created_at = unix_secs(&job.created_at);
|
let created_at = unix_secs(&job.created_at);
|
||||||
self.push_item(FeedItem { token: result.entity_token.clone(), kind, thumbnail: result.media_links.thumbnail(THUMBNAIL_WIDTH), full_url: result.media_links.cdn_url.clone(), created_at, model_id: model_id.clone(), prompt_token: job.request.maybe_prompt_token.clone(), batch_token: result.maybe_batch_token.clone(), duration_secs: None, animated: result.media_links.animated_preview(ANIMATED_WIDTH) });
|
let thumbnail = result.media_links.thumbnail(THUMBNAIL_WIDTH).filter(|_| kind != MediaKind::Audio);
|
||||||
|
self.push_item(FeedItem { token: result.entity_token.clone(), kind, title: None, thumbnail, full_url: result.media_links.cdn_url.clone(), created_at, model_id: model_id.clone(), prompt_token: job.request.maybe_prompt_token.clone(), batch_token: result.maybe_batch_token.clone(), duration_secs: None, animated: result.media_links.animated_preview(ANIMATED_WIDTH) });
|
||||||
if let Some(batch) = &result.maybe_batch_token {
|
if let Some(batch) = &result.maybe_batch_token {
|
||||||
notices.push(FeedNotice::LoadBatch { job_token: job.job_token.clone(), batch_token: batch.clone() });
|
notices.push(FeedNotice::LoadBatch { job_token: job.job_token.clone(), batch_token: batch.clone() });
|
||||||
}
|
}
|
||||||
@@ -209,9 +210,11 @@ pub fn feed_item(kind: MediaKind, file: &MediaFile) -> FeedItem {
|
|||||||
kind: match file.media_class.as_deref() {
|
kind: match file.media_class.as_deref() {
|
||||||
Some("video") => MediaKind::Video,
|
Some("video") => MediaKind::Video,
|
||||||
Some("image") => MediaKind::Image,
|
Some("image") => MediaKind::Image,
|
||||||
|
Some("audio") => MediaKind::Audio,
|
||||||
_ => kind,
|
_ => kind,
|
||||||
},
|
},
|
||||||
thumbnail: file.media_links.thumbnail(THUMBNAIL_WIDTH),
|
title: file.maybe_title.clone().filter(|t| !t.is_empty()),
|
||||||
|
thumbnail: file.thumbnail(THUMBNAIL_WIDTH),
|
||||||
full_url: file.media_links.cdn_url.clone(),
|
full_url: file.media_links.cdn_url.clone(),
|
||||||
created_at: unix_secs(&file.created_at),
|
created_at: unix_secs(&file.created_at),
|
||||||
model_id: file.maybe_model_type.clone().or_else(|| file.maybe_origin_model_type.clone()),
|
model_id: file.maybe_model_type.clone().or_else(|| file.maybe_origin_model_type.clone()),
|
||||||
@@ -222,7 +225,8 @@ pub fn feed_item(kind: MediaKind, file: &MediaFile) -> FeedItem {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Whether a job made an image or a video: by its model, else by its category.
|
/// Whether a job made an image, a video or audio: by its model, else by its category
|
||||||
|
/// (`useGenerationJobs` checks video before image before audio).
|
||||||
fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
|
fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
|
||||||
if let Some(model) = job.request.maybe_model_type.as_deref() {
|
if let Some(model) = job.request.maybe_model_type.as_deref() {
|
||||||
if catalog.image.iter().any(|m| m.id == model) {
|
if catalog.image.iter().any(|m| m.id == model) {
|
||||||
@@ -231,12 +235,17 @@ fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
|
|||||||
if catalog.video.iter().any(|m| m.id == model) {
|
if catalog.video.iter().any(|m| m.id == model) {
|
||||||
return Some(MediaKind::Video);
|
return Some(MediaKind::Video);
|
||||||
}
|
}
|
||||||
|
if catalog.audio.iter().any(|m| m.id == model) {
|
||||||
|
return Some(MediaKind::Audio);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
let category = job.request.inference_category.as_str();
|
let category = job.request.inference_category.as_str();
|
||||||
if category.contains("video") {
|
if category.contains("video") {
|
||||||
Some(MediaKind::Video)
|
Some(MediaKind::Video)
|
||||||
} else if category.contains("image") {
|
} else if category.contains("image") {
|
||||||
Some(MediaKind::Image)
|
Some(MediaKind::Image)
|
||||||
|
} else if category.contains("audio") {
|
||||||
|
Some(MediaKind::Audio)
|
||||||
} else {
|
} else {
|
||||||
None
|
None
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -4,6 +4,7 @@
|
|||||||
pub enum MediaKind {
|
pub enum MediaKind {
|
||||||
Image,
|
Image,
|
||||||
Video,
|
Video,
|
||||||
|
Audio,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl MediaKind {
|
impl MediaKind {
|
||||||
@@ -11,6 +12,7 @@ impl MediaKind {
|
|||||||
match self {
|
match self {
|
||||||
MediaKind::Image => "Image",
|
MediaKind::Image => "Image",
|
||||||
MediaKind::Video => "Video",
|
MediaKind::Video => "Video",
|
||||||
|
MediaKind::Audio => "Audio",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -18,17 +20,20 @@ impl MediaKind {
|
|||||||
match self {
|
match self {
|
||||||
MediaKind::Image => "images",
|
MediaKind::Image => "images",
|
||||||
MediaKind::Video => "videos",
|
MediaKind::Video => "videos",
|
||||||
|
MediaKind::Audio => "audio clips",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// A finished image or video in the user's library.
|
/// A finished image, video or audio clip in the user's library.
|
||||||
#[derive(Clone, Debug)]
|
#[derive(Clone, Debug)]
|
||||||
pub struct FeedItem {
|
pub struct FeedItem {
|
||||||
/// The media file token.
|
/// The media file token.
|
||||||
pub token: String,
|
pub token: String,
|
||||||
pub kind: MediaKind,
|
pub kind: MediaKind,
|
||||||
/// A thumbnail URL (512 px wide where the CDN can resize).
|
/// The file's title (`maybe_title`), when it has one.
|
||||||
|
pub title: Option<String>,
|
||||||
|
/// A thumbnail URL (512 px wide where the CDN can resize); audio has none.
|
||||||
pub thumbnail: Option<String>,
|
pub thumbnail: Option<String>,
|
||||||
/// The original file.
|
/// The original file.
|
||||||
pub full_url: String,
|
pub full_url: String,
|
||||||
|
|||||||
@@ -8,10 +8,15 @@ use crate::feed::types::MediaKind;
|
|||||||
/// Default models (`defaultModelForPage.ts`).
|
/// Default models (`defaultModelForPage.ts`).
|
||||||
pub const DEFAULT_IMAGE_MODEL: &str = "nano_banana_pro";
|
pub const DEFAULT_IMAGE_MODEL: &str = "nano_banana_pro";
|
||||||
pub const DEFAULT_VIDEO_MODEL: &str = "seedance_2p0";
|
pub const DEFAULT_VIDEO_MODEL: &str = "seedance_2p0";
|
||||||
|
pub const DEFAULT_AUDIO_MODEL: &str = "suno_music";
|
||||||
/// The fallback prompt limit (`maxPromptLength ?? 1000`).
|
/// The fallback prompt limit (`maxPromptLength ?? 1000`).
|
||||||
pub const DEFAULT_PROMPT_MAX: usize = 1000;
|
pub const DEFAULT_PROMPT_MAX: usize = 1000;
|
||||||
/// The model pickers group by family once they list this many models.
|
/// The model pickers group by family once they list this many models.
|
||||||
const GROUP_THRESHOLD: usize = 8;
|
const GROUP_THRESHOLD: usize = 8;
|
||||||
|
/// Audio models that work from exactly one audio track (`AUDIO_MODELS_REQUIRING_AUDIO_REF`).
|
||||||
|
const AUDIO_MODELS_REQUIRING_AUDIO_REF: [&str; 2] = ["suno_remix", "suno_sample"];
|
||||||
|
/// The musical keys (`SoundsSettingsPopover`; there are no E keys): value, label, short label.
|
||||||
|
pub const MUSICAL_KEYS: [(&str, &str, &str); 13] = [("auto", "Auto", "Auto"), ("c_major", "C Major", "C"), ("c_minor", "C Minor", "Cm"), ("d_major", "D Major", "D"), ("d_minor", "D Minor", "Dm"), ("f_major", "F Major", "F"), ("f_minor", "F Minor", "Fm"), ("g_major", "G Major", "G"), ("g_minor", "G Minor", "Gm"), ("a_major", "A Major", "A"), ("a_minor", "A Minor", "Am"), ("b_major", "B Major", "B"), ("b_minor", "B Minor", "Bm")];
|
||||||
|
|
||||||
/// One generation model and what it accepts.
|
/// One generation model and what it accepts.
|
||||||
#[derive(Clone, Debug, Default)]
|
#[derive(Clone, Debug, Default)]
|
||||||
@@ -56,10 +61,42 @@ pub struct ModelInfo {
|
|||||||
pub bitrate_default: Option<String>,
|
pub bitrate_default: Option<String>,
|
||||||
pub output_formats: Vec<String>,
|
pub output_formats: Vec<String>,
|
||||||
pub output_format_default: Option<String>,
|
pub output_format_default: Option<String>,
|
||||||
|
/// Audio only.
|
||||||
|
pub audio: AudioCaps,
|
||||||
/// How long a generation usually takes (drives the estimated progress bar).
|
/// How long a generation usually takes (drives the estimated progress bar).
|
||||||
pub expected_secs: f32,
|
pub expected_secs: f32,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What an audio model takes besides the prompt and references.
|
||||||
|
#[derive(Clone, Debug, Default, PartialEq)]
|
||||||
|
pub struct AudioCaps {
|
||||||
|
pub style_prompt: bool,
|
||||||
|
pub instrumental: bool,
|
||||||
|
pub keep_lyrics: bool,
|
||||||
|
pub loopable: bool,
|
||||||
|
pub bpm: bool,
|
||||||
|
pub musical_key: bool,
|
||||||
|
pub sample_rates: Vec<u32>,
|
||||||
|
pub sample_rate_default: Option<u32>,
|
||||||
|
pub speed: bool,
|
||||||
|
pub volume: bool,
|
||||||
|
pub pitch: bool,
|
||||||
|
/// Remixes and samples work from exactly one audio track.
|
||||||
|
pub requires_audio_ref: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl AudioCaps {
|
||||||
|
/// Whether the Tuning popover shows.
|
||||||
|
pub fn has_tuning(&self) -> bool {
|
||||||
|
!self.sample_rates.is_empty() || self.speed || self.volume || self.pitch
|
||||||
|
}
|
||||||
|
|
||||||
|
/// `picked` if the model offers it, else its default, else its first rate.
|
||||||
|
pub fn sample_rate(&self, picked: Option<u32>) -> Option<u32> {
|
||||||
|
picked.filter(|hz| self.sample_rates.contains(hz)).or(self.sample_rate_default).or_else(|| self.sample_rates.first().copied())
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
impl ModelInfo {
|
impl ModelInfo {
|
||||||
/// Batch sizes the count picker offers (`predefined` or `1..=max`).
|
/// Batch sizes the count picker offers (`predefined` or `1..=max`).
|
||||||
pub fn batch_options(&self) -> Vec<u16> {
|
pub fn batch_options(&self) -> Vec<u16> {
|
||||||
@@ -185,6 +222,22 @@ pub fn video_model(d: &artcraft_client::endpoints::omni_gen::models::video::omni
|
|||||||
m
|
m
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Builds a model from the server's audio listing.
|
||||||
|
pub fn audio_model(d: &crate::backend::wire::AudioModel) -> ModelInfo {
|
||||||
|
let mut m = base(&d.model, MediaKind::Audio, d.full_name.as_deref(), d.model_creator.clone());
|
||||||
|
if let Some(short) = d.extra_info_short.as_deref().filter(|s| !s.is_empty()) {
|
||||||
|
m.description = short.to_owned();
|
||||||
|
}
|
||||||
|
m.disabled = d.is_disabled;
|
||||||
|
m.prompt_supported = d.text_prompt_supported.unwrap_or(true);
|
||||||
|
m.prompt_max = None;
|
||||||
|
let max = |supported: bool, max: Option<u16>| if supported { max.map_or(1, usize::from) } else { 0 };
|
||||||
|
m.audio_refs_max = max(d.audio_references_supported, d.audio_references_max);
|
||||||
|
m.image_refs_max = max(d.image_references_supported, d.image_references_max);
|
||||||
|
m.audio = AudioCaps { style_prompt: d.style_prompt_supported, instrumental: d.instrumental_toggle_supported, keep_lyrics: d.keep_lyrics_supported, loopable: d.loopable_toggle_supported, bpm: d.bpm_supported, musical_key: d.musical_key_supported, sample_rates: d.sample_rate_hz_options.clone(), sample_rate_default: d.sample_rate_hz_default, speed: d.speed_supported, volume: d.volume_supported, pitch: d.pitch_supported, requires_audio_ref: AUDIO_MODELS_REQUIRING_AUDIO_REF.contains(&d.model.as_str()) };
|
||||||
|
m
|
||||||
|
}
|
||||||
|
|
||||||
fn strings<T: Clone + Into<String>>(options: &Option<Vec<T>>) -> Vec<String> {
|
fn strings<T: Clone + Into<String>>(options: &Option<Vec<T>>) -> Vec<String> {
|
||||||
options.as_deref().unwrap_or_default().iter().cloned().map(Into::into).collect()
|
options.as_deref().unwrap_or_default().iter().cloned().map(Into::into).collect()
|
||||||
}
|
}
|
||||||
@@ -212,7 +265,7 @@ fn base(id: &str, kind: MediaKind, full_name: Option<&str>, creator: Option<Stri
|
|||||||
|
|
||||||
/// Families whose picker names the app curates (the static model lists' `selectorName`s).
|
/// Families whose picker names the app curates (the static model lists' `selectorName`s).
|
||||||
fn has_curated_name(id: &str) -> bool {
|
fn has_curated_name(id: &str) -> bool {
|
||||||
const PREFIXES: [&str; 11] = ["flux_", "nano_banana", "gpt_image", "seedream", "seedance", "kling", "veo_", "sora", "happy_horse", "grok_imagine", "qwen"];
|
const PREFIXES: [&str; 13] = ["flux_", "nano_banana", "gpt_image", "seedream", "seedance", "kling", "veo_", "sora", "happy_horse", "grok_imagine", "qwen", "suno", "seed_audio"];
|
||||||
PREFIXES.iter().any(|p| id.starts_with(p))
|
PREFIXES.iter().any(|p| id.starts_with(p))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -292,12 +345,17 @@ fn description(id: &str) -> &'static str {
|
|||||||
"grok_video" | "grok_imagine_video" => "Fastest video model",
|
"grok_video" | "grok_imagine_video" => "Fastest video model",
|
||||||
"sora_2" => "Smart video model",
|
"sora_2" => "Smart video model",
|
||||||
"veo_3" => "Slow, high-quality model",
|
"veo_3" => "Slow, high-quality model",
|
||||||
|
"suno_music" => "Full songs from a text prompt",
|
||||||
|
"suno_remix" => "Remix an existing track",
|
||||||
|
"suno_sounds" => "Sound effects with beat control",
|
||||||
|
"suno_sample" => "Build a song from a sample",
|
||||||
|
"seed_audio_1p0" => "Sound generation with fine tuning",
|
||||||
_ => "",
|
_ => "",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Typical generation time (the static list's `progressBarTime`).
|
/// Typical generation time (the static list's `progressBarTime`, else the feed's per-kind default).
|
||||||
fn expected_secs(id: &str, kind: MediaKind) -> f32 {
|
pub fn expected_secs(id: &str, kind: MediaKind) -> f32 {
|
||||||
match id {
|
match id {
|
||||||
"grok_image" | "grok_imagine_image" | "flux_pro_1p1" | "flux_1_dev" | "flux_1_schnell" => 10.0,
|
"grok_image" | "grok_imagine_image" | "flux_pro_1p1" | "flux_1_dev" | "flux_1_schnell" => 10.0,
|
||||||
"grok_video" | "grok_imagine_video" => 50.0,
|
"grok_video" | "grok_imagine_video" => 50.0,
|
||||||
@@ -307,19 +365,20 @@ fn expected_secs(id: &str, kind: MediaKind) -> f32 {
|
|||||||
"gpt_image_1" | "gpt_image_1p5" | "gpt_image_2p5_flare" | "seedream_4" | "seedream_4p5" | "seedream_5_lite" => 60.0,
|
"gpt_image_1" | "gpt_image_1p5" | "gpt_image_2p5_flare" | "seedream_4" | "seedream_4p5" | "seedream_5_lite" => 60.0,
|
||||||
"gpt_image_2" | "gpt_image_2p5_sunburst" => 120.0,
|
"gpt_image_2" | "gpt_image_2p5_sunburst" => 120.0,
|
||||||
_ if kind == MediaKind::Video => 900.0,
|
_ if kind == MediaKind::Video => 900.0,
|
||||||
|
_ if kind == MediaKind::Audio => 120.0,
|
||||||
_ => 30.0,
|
_ => 30.0,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The maker, by id prefix, when the server doesn't say.
|
/// The maker, by id prefix, when the server doesn't say.
|
||||||
pub fn creator_for(id: &str) -> &'static str {
|
pub fn creator_for(id: &str) -> &'static str {
|
||||||
const PREFIXES: [(&str, &str); 16] = [("flux", "black_forest_labs"), ("nano_banana", "google"), ("gemini", "google"), ("veo", "google"), ("imagen", "google"), ("gpt_image", "openai"), ("sora", "openai"), ("seedream", "bytedance"), ("seedance", "bytedance"), ("seededit", "bytedance"), ("kling", "kling"), ("grok", "grok"), ("midjourney", "midjourney"), ("qwen", "alibaba"), ("happy_horse", "alibaba"), ("minimax", "minimax")];
|
const PREFIXES: [(&str, &str); 18] = [("suno", "suno"), ("seed_audio", "bytedance"), ("flux", "black_forest_labs"), ("nano_banana", "google"), ("gemini", "google"), ("veo", "google"), ("imagen", "google"), ("gpt_image", "openai"), ("sora", "openai"), ("seedream", "bytedance"), ("seedance", "bytedance"), ("seededit", "bytedance"), ("kling", "kling"), ("grok", "grok"), ("midjourney", "midjourney"), ("qwen", "alibaba"), ("happy_horse", "alibaba"), ("minimax", "minimax")];
|
||||||
PREFIXES.iter().find(|(p, _)| id.starts_with(p)).map_or("artcraft", |(_, c)| c)
|
PREFIXES.iter().find(|(p, _)| id.starts_with(p)).map_or("artcraft", |(_, c)| c)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The picker family (`getModelFamilyName`).
|
/// The picker family (`getModelFamilyName`).
|
||||||
fn family(id: &str) -> &'static str {
|
fn family(id: &str) -> &'static str {
|
||||||
const FAMILIES: [(&str, &str); 22] = [("seedance", "Seedance"), ("kling", "Kling"), ("veo", "Veo"), ("sora", "Sora"), ("happy_horse", "Happy Horse"), ("wan", "Wan"), ("vidu", "Vidu"), ("preview", "Preview"), ("nano_banana", "Nano Banana"), ("gemini_25_flash", "Nano Banana"), ("seedream", "Seedream"), ("seededit", "SeedEdit"), ("gpt_image", "GPT Image"), ("flux", "Flux"), ("midjourney", "Midjourney"), ("grok", "Grok"), ("qwen", "Qwen"), ("recraft", "Recraft"), ("imagen", "Imagen"), ("minimax", "MiniMax"), ("marble", "World Labs"), ("suno", "Suno")];
|
const FAMILIES: [(&str, &str); 23] = [("seed_audio", "Seed Audio"), ("seedance", "Seedance"), ("kling", "Kling"), ("veo", "Veo"), ("sora", "Sora"), ("happy_horse", "Happy Horse"), ("wan", "Wan"), ("vidu", "Vidu"), ("preview", "Preview"), ("nano_banana", "Nano Banana"), ("gemini_25_flash", "Nano Banana"), ("seedream", "Seedream"), ("seededit", "SeedEdit"), ("gpt_image", "GPT Image"), ("flux", "Flux"), ("midjourney", "Midjourney"), ("grok", "Grok"), ("qwen", "Qwen"), ("recraft", "Recraft"), ("imagen", "Imagen"), ("minimax", "MiniMax"), ("marble", "World Labs"), ("suno", "Suno")];
|
||||||
FAMILIES.iter().find(|(p, _)| id.starts_with(p)).map_or("Other", |(_, f)| f)
|
FAMILIES.iter().find(|(p, _)| id.starts_with(p)).map_or("Other", |(_, f)| f)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -389,13 +448,16 @@ fn video_page_models(models: &[ModelInfo]) -> Vec<ModelInfo> {
|
|||||||
list
|
list
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Both catalogs, for looking up names in the feed, and each create page's picker list.
|
/// Every catalog, for looking up names in the feed, and each create page's picker list.
|
||||||
#[derive(Default)]
|
#[derive(Default)]
|
||||||
pub struct Catalog {
|
pub struct Catalog {
|
||||||
pub image: Vec<ModelInfo>,
|
pub image: Vec<ModelInfo>,
|
||||||
pub video: Vec<ModelInfo>,
|
pub video: Vec<ModelInfo>,
|
||||||
|
pub audio: Vec<ModelInfo>,
|
||||||
pub image_page: Vec<ModelInfo>,
|
pub image_page: Vec<ModelInfo>,
|
||||||
pub video_page: Vec<ModelInfo>,
|
pub video_page: Vec<ModelInfo>,
|
||||||
|
/// The audio page's list: enabled, in the server's order.
|
||||||
|
pub audio_page: Vec<ModelInfo>,
|
||||||
}
|
}
|
||||||
|
|
||||||
impl Catalog {
|
impl Catalog {
|
||||||
@@ -409,8 +471,13 @@ impl Catalog {
|
|||||||
self.video = models;
|
self.video = models;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
pub fn set_audio(&mut self, models: Vec<ModelInfo>) {
|
||||||
|
self.audio_page = models.iter().filter(|m| !m.disabled).cloned().collect();
|
||||||
|
self.audio = models;
|
||||||
|
}
|
||||||
|
|
||||||
pub fn find(&self, id: &str) -> Option<&ModelInfo> {
|
pub fn find(&self, id: &str) -> Option<&ModelInfo> {
|
||||||
self.image.iter().chain(&self.video).find(|m| m.id == id)
|
self.image.iter().chain(&self.video).chain(&self.audio).find(|m| m.id == id)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -505,6 +572,15 @@ pub fn quality_label(value: &str) -> String {
|
|||||||
.to_owned()
|
.to_owned()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Sample rate labels (`formatSampleRateHz`): 44100 → "44.1 kHz", 24000 → "24 kHz".
|
||||||
|
pub fn sample_rate_label(hz: u32) -> String {
|
||||||
|
if hz.is_multiple_of(1000) {
|
||||||
|
format!("{} kHz", hz / 1000)
|
||||||
|
} else {
|
||||||
|
format!("{:.1} kHz", hz as f32 / 1000.0)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// Bitrate labels.
|
/// Bitrate labels.
|
||||||
pub fn bitrate_label(value: &str) -> String {
|
pub fn bitrate_label(value: &str) -> String {
|
||||||
match value {
|
match value {
|
||||||
@@ -533,6 +609,8 @@ mod tests {
|
|||||||
assert_eq!(display_name("kling_3p0_pro"), "Kling 3.0 Pro");
|
assert_eq!(display_name("kling_3p0_pro"), "Kling 3.0 Pro");
|
||||||
assert_eq!(display_name("nano_banana"), "Nano Banana");
|
assert_eq!(display_name("nano_banana"), "Nano Banana");
|
||||||
assert_eq!(display_name("seedance_2p0_bpu"), "Seedance 2.0 Plus Ultra");
|
assert_eq!(display_name("seedance_2p0_bpu"), "Seedance 2.0 Plus Ultra");
|
||||||
|
assert_eq!(display_name("seed_audio_1p0"), "Seed Audio 1.0");
|
||||||
|
assert_eq!(display_name("suno_music"), "Suno Music");
|
||||||
}
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
@@ -547,6 +625,8 @@ mod tests {
|
|||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn falls_back_to_creator_by_prefix() {
|
fn falls_back_to_creator_by_prefix() {
|
||||||
|
assert_eq!(creator_for("seed_audio_1p0"), "bytedance");
|
||||||
|
assert_eq!(creator_for("suno_remix"), "suno");
|
||||||
assert_eq!(creator_for("seedream_4p5"), "bytedance");
|
assert_eq!(creator_for("seedream_4p5"), "bytedance");
|
||||||
assert_eq!(creator_for("veo_3"), "google");
|
assert_eq!(creator_for("veo_3"), "google");
|
||||||
assert_eq!(creator_for("something_new"), "artcraft");
|
assert_eq!(creator_for("something_new"), "artcraft");
|
||||||
@@ -576,6 +656,21 @@ mod tests {
|
|||||||
assert_eq!(ModelInfo::resolve(None, &[], None), None);
|
assert_eq!(ModelInfo::resolve(None, &[], None), None);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn audio_listing_maps_capabilities() {
|
||||||
|
let d: crate::backend::wire::AudioModel = serde_json::from_value(serde_json::json!({ "model": "seed_audio_1p0", "model_creator": "artcraft", "audio_references_supported": true, "audio_references_max": 3, "image_references_supported": true, "sample_rate_hz_options": [8000, 24000, 44100], "sample_rate_hz_default": 24000, "speed_supported": true })).unwrap();
|
||||||
|
let m = audio_model(&d);
|
||||||
|
assert_eq!((m.name.as_str(), m.creator.as_str(), m.description.as_str()), ("Seed Audio 1.0", "bytedance", "Sound generation with fine tuning"));
|
||||||
|
assert_eq!((m.audio_refs_max, m.image_refs_max, m.prompt_max), (3, 1, None));
|
||||||
|
assert!(m.audio.has_tuning() && !m.audio.requires_audio_ref && !m.audio.style_prompt);
|
||||||
|
assert_eq!(m.audio.sample_rate(Some(44100)), Some(44100));
|
||||||
|
assert_eq!(m.audio.sample_rate(Some(11025)), Some(24000), "unsupported picks fall back to the default");
|
||||||
|
let remix = audio_model(&serde_json::from_value(serde_json::json!({ "model": "suno_remix", "audio_references_supported": true })).unwrap());
|
||||||
|
assert!(remix.audio.requires_audio_ref && !remix.audio.has_tuning());
|
||||||
|
assert_eq!((remix.audio_refs_max, remix.audio.sample_rate(None)), (1, None));
|
||||||
|
assert_eq!((sample_rate_label(44100).as_str(), sample_rate_label(24000).as_str()), ("44.1 kHz", "24 kHz"));
|
||||||
|
}
|
||||||
|
|
||||||
#[test]
|
#[test]
|
||||||
fn batch_counts_snap_to_valid_values() {
|
fn batch_counts_snap_to_valid_values() {
|
||||||
let mut m = model("grok_imagine_image");
|
let mut m = model("grok_imagine_image");
|
||||||
|
|||||||
@@ -153,7 +153,7 @@ impl LibraryPicker {
|
|||||||
}
|
}
|
||||||
|
|
||||||
fn picks(&self) -> Vec<RefMedia> {
|
fn picks(&self) -> Vec<RefMedia> {
|
||||||
self.selected.iter().filter_map(|t| self.items.iter().find(|i| i.token == *t)).map(|f| RefMedia::from_library(self.kind, f.token.clone(), f.media_links.thumbnail(256), Some(f.media_links.cdn_url.clone()), f.maybe_duration_millis.map_or(0.0, |ms| ms as f32 / 1000.0))).collect()
|
self.selected.iter().filter_map(|t| self.items.iter().find(|i| i.token == *t)).map(|f| RefMedia::from_library(self.kind, f.token.clone(), f.thumbnail(256), Some(f.media_links.cdn_url.clone()), f.maybe_duration_millis.map_or(0.0, |ms| ms as f32 / 1000.0))).collect()
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -162,7 +162,7 @@ fn tile_ui(ui: &mut egui::Ui, rect: Rect, item: &MediaFile, kind: RefKind, attac
|
|||||||
let resp = ui.interact(rect, Id::new(("picker-tile", &item.token)), if attached { Sense::hover() } else { Sense::click() });
|
let resp = ui.interact(rect, Id::new(("picker-tile", &item.token)), if attached { Sense::hover() } else { Sense::click() });
|
||||||
let p = ui.painter();
|
let p = ui.painter();
|
||||||
p.rect_filled(rect, 0.0, theme::fade(theme::CONTROLS, 0.6));
|
p.rect_filled(rect, 0.0, theme::fade(theme::CONTROLS, 0.6));
|
||||||
match item.media_links.thumbnail(256).map(|u| cache.get(ui.ctx(), &u)) {
|
match item.thumbnail(256).map(|u| cache.get(ui.ctx(), &u)) {
|
||||||
Some(Lookup::Ready(t)) => {
|
Some(Lookup::Ready(t)) => {
|
||||||
let image = egui::Image::new(&t).uv(cover_uv(t.size_vec2()));
|
let image = egui::Image::new(&t).uv(cover_uv(t.size_vec2()));
|
||||||
let image = if attached { image.tint(Color32::from_gray(90)) } else { image };
|
let image = if attached { image.tint(Color32::from_gray(90)) } else { image };
|
||||||
|
|||||||
@@ -1,8 +1,9 @@
|
|||||||
//! The lightbox (`lightbox-modal`): the media large on black, prev/next through the feed, and a
|
//! The lightbox (`lightbox-modal`): the media large on black (audio as a player panel), prev/next
|
||||||
//! side column with the prompt, details and actions.
|
//! through the feed, and a side column with the prompt, details and actions.
|
||||||
|
|
||||||
use egui::{Align2, Color32, FontId, Id, Key, Rect, Sense, Ui, pos2, vec2};
|
use egui::{Align2, Color32, FontId, Id, Key, Rect, Sense, Ui, pos2, vec2};
|
||||||
|
|
||||||
|
use crate::backend::audio::AudioPlayer;
|
||||||
use crate::backend::media_cache::{Lookup, MediaCache};
|
use crate::backend::media_cache::{Lookup, MediaCache};
|
||||||
use crate::backend::video::{self, VideoPlayer};
|
use crate::backend::video::{self, VideoPlayer};
|
||||||
use crate::backend::wire::Prompt;
|
use crate::backend::wire::Prompt;
|
||||||
@@ -10,9 +11,12 @@ use crate::feed::grid::ModelNames;
|
|||||||
use crate::feed::types::{FeedItem, MediaKind};
|
use crate::feed::types::{FeedItem, MediaKind};
|
||||||
use crate::theme;
|
use crate::theme;
|
||||||
use crate::ui::icons::{self, Icon};
|
use crate::ui::icons::{self, Icon};
|
||||||
use crate::ui::{creator_icons, widgets};
|
use crate::ui::{creator_icons, waveform, widgets};
|
||||||
|
|
||||||
const SIDE_WIDTH: f32 = 280.0;
|
const SIDE_WIDTH: f32 = 280.0;
|
||||||
|
/// The audio panel's widest (`max-w-xl`) and its padding.
|
||||||
|
const AUDIO_PANEL_MAX: f32 = 576.0;
|
||||||
|
const AUDIO_PANEL_PAD: f32 = 24.0;
|
||||||
|
|
||||||
pub struct Lightbox {
|
pub struct Lightbox {
|
||||||
pub kind: MediaKind,
|
pub kind: MediaKind,
|
||||||
@@ -41,7 +45,8 @@ impl Lightbox {
|
|||||||
Self { kind, token, confirm_delete: false, prompt_expanded: false, player: None }
|
Self { kind, token, confirm_delete: false, prompt_expanded: false, player: None }
|
||||||
}
|
}
|
||||||
|
|
||||||
pub fn show(&mut self, ctx: &egui::Context, item: &FeedItem, order: &[String], prompt: Option<&Prompt>, names: &dyn ModelNames, cache: &mut MediaCache) -> Vec<LightboxAction> {
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
pub fn show(&mut self, ctx: &egui::Context, item: &FeedItem, order: &[String], prompt: Option<&Prompt>, names: &dyn ModelNames, cache: &mut MediaCache, audio: &mut AudioPlayer) -> Vec<LightboxAction> {
|
||||||
let mut actions = Vec::new();
|
let mut actions = Vec::new();
|
||||||
let screen = ctx.content_rect();
|
let screen = ctx.content_rect();
|
||||||
let size = vec2((screen.width() - 80.0).min(1200.0), (screen.height() - 80.0).min(760.0));
|
let size = vec2((screen.width() - 80.0).min(1200.0), (screen.height() - 80.0).min(760.0));
|
||||||
@@ -65,7 +70,7 @@ impl Lightbox {
|
|||||||
ui.horizontal_top(|ui| {
|
ui.horizontal_top(|ui| {
|
||||||
let media_rect = Rect::from_min_size(ui.cursor().min, vec2(ui.available_width() - SIDE_WIDTH - 16.0, size.y - 32.0));
|
let media_rect = Rect::from_min_size(ui.cursor().min, vec2(ui.available_width() - SIDE_WIDTH - 16.0, size.y - 32.0));
|
||||||
ui.allocate_rect(media_rect, Sense::hover());
|
ui.allocate_rect(media_rect, Sense::hover());
|
||||||
self.media(ui, media_rect, item, cache, &prev, &next, &mut actions);
|
self.media(ui, media_rect, item, cache, audio, &prev, &next, &mut actions);
|
||||||
ui.add_space(16.0);
|
ui.add_space(16.0);
|
||||||
ui.vertical(|ui| {
|
ui.vertical(|ui| {
|
||||||
ui.set_width(SIDE_WIDTH);
|
ui.set_width(SIDE_WIDTH);
|
||||||
@@ -83,7 +88,14 @@ impl Lightbox {
|
|||||||
}
|
}
|
||||||
|
|
||||||
#[allow(clippy::too_many_arguments)]
|
#[allow(clippy::too_many_arguments)]
|
||||||
fn media(&mut self, ui: &mut Ui, rect: Rect, item: &FeedItem, cache: &mut MediaCache, prev: &Option<String>, next: &Option<String>, actions: &mut Vec<LightboxAction>) {
|
fn media(&mut self, ui: &mut Ui, rect: Rect, item: &FeedItem, cache: &mut MediaCache, audio: &mut AudioPlayer, prev: &Option<String>, next: &Option<String>, actions: &mut Vec<LightboxAction>) {
|
||||||
|
if item.kind == MediaKind::Audio {
|
||||||
|
self.player = None;
|
||||||
|
ui.painter().rect_filled(rect, 0.0, Color32::BLACK);
|
||||||
|
audio_panel(ui, rect, item, audio);
|
||||||
|
self.nav_buttons(ui, rect, prev, next, actions);
|
||||||
|
return;
|
||||||
|
}
|
||||||
if item.kind == MediaKind::Video && video::ffmpeg_available() {
|
if item.kind == MediaKind::Video && video::ffmpeg_available() {
|
||||||
if self.player.as_ref().is_none_or(|p| p.url() != item.full_url) {
|
if self.player.as_ref().is_none_or(|p| p.url() != item.full_url) {
|
||||||
self.player = Some(VideoPlayer::new(&item.full_url));
|
self.player = Some(VideoPlayer::new(&item.full_url));
|
||||||
@@ -237,7 +249,8 @@ impl Lightbox {
|
|||||||
|
|
||||||
ui.add_space(16.0);
|
ui.add_space(16.0);
|
||||||
let w = (ui.available_width() - 8.0) / 2.0;
|
let w = (ui.available_width() - 8.0) / 2.0;
|
||||||
if item.prompt_token.is_some() && widgets::button(ui, Some(Icon::RotateCw), "Recreate", widgets::Kind::Primary, ui.available_width(), theme::CONTROL_H).clicked() {
|
// Audio can't be recreated yet (`Audio v1`).
|
||||||
|
if item.prompt_token.is_some() && item.kind != MediaKind::Audio && widgets::button(ui, Some(Icon::RotateCw), "Recreate", widgets::Kind::Primary, ui.available_width(), theme::CONTROL_H).clicked() {
|
||||||
actions.push(LightboxAction::Recreate);
|
actions.push(LightboxAction::Recreate);
|
||||||
}
|
}
|
||||||
ui.horizontal_wrapped(|ui| {
|
ui.horizontal_wrapped(|ui| {
|
||||||
@@ -282,6 +295,17 @@ impl Lightbox {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// Audio in the lightbox: a bordered panel with the music tile and the full-size player.
|
||||||
|
fn audio_panel(ui: &mut Ui, rect: Rect, item: &FeedItem, audio: &mut AudioPlayer) {
|
||||||
|
let width = (rect.width() - 48.0).min(AUDIO_PANEL_MAX);
|
||||||
|
let panel = Rect::from_center_size(rect.center(), vec2(width, AUDIO_PANEL_PAD * 2.0 + 64.0 + 20.0 + 44.0));
|
||||||
|
ui.painter().rect(panel, theme::RADIUS, theme::fade(theme::CONTROLS, 0.6), theme::hairline(), egui::StrokeKind::Inside);
|
||||||
|
waveform::music_tile(ui, Rect::from_center_size(pos2(panel.center().x, panel.top() + AUDIO_PANEL_PAD + 32.0), vec2(64.0, 64.0)), false);
|
||||||
|
let bar = Rect::from_min_max(pos2(panel.left() + AUDIO_PANEL_PAD, panel.bottom() - AUDIO_PANEL_PAD - 44.0), pos2(panel.right() - AUDIO_PANEL_PAD, panel.bottom() - AUDIO_PANEL_PAD));
|
||||||
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar));
|
||||||
|
waveform::player(&mut child, Id::new(("lightbox-player", &item.token)), audio, &item.full_url, item.duration_secs, false);
|
||||||
|
}
|
||||||
|
|
||||||
fn info_row(ui: &mut Ui, label: &str, value: impl FnOnce(&mut Ui)) {
|
fn info_row(ui: &mut Ui, label: &str, value: impl FnOnce(&mut Ui)) {
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
let (rect, _) = ui.allocate_exact_size(vec2(96.0, 20.0), Sense::hover());
|
let (rect, _) = ui.allocate_exact_size(vec2(96.0, 20.0), Sense::hover());
|
||||||
|
|||||||
@@ -7,6 +7,7 @@ use std::sync::Arc;
|
|||||||
use egui::{Align2, Color32, Id, Mesh, Order, Rect, Ui, pos2, vec2};
|
use egui::{Align2, Color32, Id, Mesh, Order, Rect, Ui, pos2, vec2};
|
||||||
use serde_json::{Map, Value};
|
use serde_json::{Map, Value};
|
||||||
|
|
||||||
|
use crate::backend::audio::AudioPlayer;
|
||||||
use crate::backend::media_cache::MediaCache;
|
use crate::backend::media_cache::MediaCache;
|
||||||
use crate::backend::{Backend, Modality, UploadedMedia};
|
use crate::backend::{Backend, Modality, UploadedMedia};
|
||||||
use crate::feed::grid::{self, FeedAction, FeedView, RatioCache, ViewMode};
|
use crate::feed::grid::{self, FeedAction, FeedView, RatioCache, ViewMode};
|
||||||
@@ -29,6 +30,8 @@ pub struct Env<'a> {
|
|||||||
pub ctx: egui::Context,
|
pub ctx: egui::Context,
|
||||||
pub backend: &'a Backend,
|
pub backend: &'a Backend,
|
||||||
pub cache: &'a mut MediaCache,
|
pub cache: &'a mut MediaCache,
|
||||||
|
/// Plays reference clips and feed tracks.
|
||||||
|
pub audio: &'a mut AudioPlayer,
|
||||||
pub toasts: &'a mut Toasts,
|
pub toasts: &'a mut Toasts,
|
||||||
pub catalog: &'a Catalog,
|
pub catalog: &'a Catalog,
|
||||||
pub signed_in: bool,
|
pub signed_in: bool,
|
||||||
@@ -84,6 +87,7 @@ pub enum AppRequest {
|
|||||||
pub enum FeedKey {
|
pub enum FeedKey {
|
||||||
Image,
|
Image,
|
||||||
Video,
|
Video,
|
||||||
|
Audio,
|
||||||
Library,
|
Library,
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -92,6 +96,7 @@ impl FeedKey {
|
|||||||
match kind {
|
match kind {
|
||||||
MediaKind::Image => FeedKey::Image,
|
MediaKind::Image => FeedKey::Image,
|
||||||
MediaKind::Video => FeedKey::Video,
|
MediaKind::Video => FeedKey::Video,
|
||||||
|
MediaKind::Audio => FeedKey::Audio,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -100,9 +105,33 @@ impl FeedKey {
|
|||||||
match self {
|
match self {
|
||||||
FeedKey::Image => "image",
|
FeedKey::Image => "image",
|
||||||
FeedKey::Video => "video",
|
FeedKey::Video => "video",
|
||||||
FeedKey::Library => "image,video",
|
FeedKey::Audio => "audio",
|
||||||
|
FeedKey::Library => "image,video,audio",
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// What the feed's items default to (the library mixes kinds; each file says its own).
|
||||||
|
pub fn kind(self) -> MediaKind {
|
||||||
|
match self {
|
||||||
|
FeedKey::Image | FeedKey::Library => MediaKind::Image,
|
||||||
|
FeedKey::Video => MediaKind::Video,
|
||||||
|
FeedKey::Audio => MediaKind::Audio,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The backend's name for this feed's library requests.
|
||||||
|
pub fn name(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
FeedKey::Image => "image",
|
||||||
|
FeedKey::Video => "video",
|
||||||
|
FeedKey::Audio => "audio",
|
||||||
|
FeedKey::Library => "library",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn from_name(name: &str) -> Option<Self> {
|
||||||
|
[FeedKey::Image, FeedKey::Video, FeedKey::Audio, FeedKey::Library].into_iter().find(|k| k.name() == name)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The page around the feed: the splash empty state (or the feed), a fade behind the prompt box,
|
/// The page around the feed: the splash empty state (or the feed), a fade behind the prompt box,
|
||||||
@@ -114,7 +143,7 @@ pub fn create_shell(ui: &mut Ui, env: &mut Env<'_>, kind: MediaKind, title: &str
|
|||||||
if feed.has_content() {
|
if feed.has_content() {
|
||||||
let view = FeedView { id: Id::new(("feed", kind.label())), mode: env.view_mode, pending: &feed.pending, failed: &feed.failed, items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: kind == MediaKind::Image, autoplay: env.autoplay, bottom_padding: bottom_offset + 24.0 };
|
let view = FeedView { id: Id::new(("feed", kind.label())), mode: env.view_mode, pending: &feed.pending, failed: &feed.failed, items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: kind == MediaKind::Image, autoplay: env.autoplay, bottom_padding: bottom_offset + 24.0 };
|
||||||
let mut feed_ui = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0))));
|
let mut feed_ui = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0))));
|
||||||
let actions = grid::show(&mut feed_ui, &view, env.cache, env.ratios, env.catalog);
|
let actions = grid::show(&mut feed_ui, &view, env.cache, env.ratios, env.audio, env.catalog);
|
||||||
handle_feed_actions(env, kind, feed, actions);
|
handle_feed_actions(env, kind, feed, actions);
|
||||||
bottom_fade(ui, area);
|
bottom_fade(ui, area);
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -0,0 +1,882 @@
|
|||||||
|
//! Create Audio (`CreateAudio` + `PromptBoxAudio`): a song, sound or sample from a prompt, with
|
||||||
|
//! the model's options (style, instrumental, lyrics, loop, beat & key, tuning), audio or image
|
||||||
|
//! references, the cost, and the audio feed.
|
||||||
|
|
||||||
|
use std::ops::RangeInclusive;
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
use egui::{Align2, Color32, FontId, Id, Rect, Response, Sense, Stroke, TextureHandle, Ui, pos2, vec2};
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use serde_json::{Map, Value, json};
|
||||||
|
|
||||||
|
use crate::backend::media_cache::Lookup;
|
||||||
|
use crate::backend::{EnqueueMeta, Modality};
|
||||||
|
use crate::feed::store::FeedStore;
|
||||||
|
use crate::feed::types::MediaKind;
|
||||||
|
use crate::models::{self, AudioCaps, Catalog, DEFAULT_AUDIO_MODEL, MUSICAL_KEYS, ModelInfo};
|
||||||
|
use crate::pages::common::{self, AppRequest, CostState, Env};
|
||||||
|
use crate::prompt_box::deck::cover_uv;
|
||||||
|
use crate::prompt_box::model_selector::model_selector;
|
||||||
|
use crate::prompt_box::pickers;
|
||||||
|
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, RefStatus, References};
|
||||||
|
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot};
|
||||||
|
use crate::theme;
|
||||||
|
use crate::ui::icons::{self, Icon};
|
||||||
|
use crate::ui::toast::Toasts;
|
||||||
|
use crate::ui::widgets::{self, Leading};
|
||||||
|
|
||||||
|
const PLACEHOLDER: &str = "Describe the music or sound you want...";
|
||||||
|
const STYLE_PLACEHOLDER: &str = "Style: e.g. dreamy synth-pop, female vocals";
|
||||||
|
/// The audio references' total length cap (`AUDIO_REF_MAX_DURATION_SECONDS`).
|
||||||
|
const MAX_REF_SECS: f32 = 600.0;
|
||||||
|
/// Beat & Key and Tuning (`SoundsSettingsPopover`, `AudioTuningPopover`).
|
||||||
|
const BPM_RANGE: RangeInclusive<u16> = 40..=240;
|
||||||
|
const DEFAULT_BPM: u16 = 120;
|
||||||
|
const SPEED_RANGE: RangeInclusive<f32> = 0.5..=2.0;
|
||||||
|
const VOLUME_RANGE: RangeInclusive<f32> = 0.5..=2.0;
|
||||||
|
const PITCH_RANGE: RangeInclusive<i8> = -12..=12;
|
||||||
|
const POPOVER_WIDTH: f32 = 240.0;
|
||||||
|
/// Shown when generate fails on the server's side (`OMNI_GENERATE_OUTAGE_MESSAGE`).
|
||||||
|
const OUTAGE_MESSAGE: &str = "Couldn't start generation - the service is temporarily unavailable. Try again shortly.";
|
||||||
|
|
||||||
|
/// What the page remembers between launches.
|
||||||
|
#[derive(Clone, Debug, Serialize, Deserialize)]
|
||||||
|
#[serde(default)]
|
||||||
|
pub struct AudioSettings {
|
||||||
|
pub prompt: String,
|
||||||
|
pub style_prompt: String,
|
||||||
|
pub model: Option<String>,
|
||||||
|
pub instrumental: bool,
|
||||||
|
pub keep_lyrics: bool,
|
||||||
|
pub loopable: bool,
|
||||||
|
/// `None` = Auto.
|
||||||
|
pub bpm: Option<u16>,
|
||||||
|
pub musical_key: String,
|
||||||
|
/// `None` = the model's default.
|
||||||
|
pub sample_rate_hz: Option<u32>,
|
||||||
|
pub speed: f32,
|
||||||
|
pub volume: f32,
|
||||||
|
/// Semitones.
|
||||||
|
pub pitch: i8,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Default for AudioSettings {
|
||||||
|
fn default() -> Self {
|
||||||
|
Self { prompt: String::new(), style_prompt: String::new(), model: None, instrumental: false, keep_lyrics: false, loopable: false, bpm: None, musical_key: "auto".to_owned(), sample_rate_hz: None, speed: 1.0, volume: 1.0, pitch: 0 }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Default)]
|
||||||
|
pub struct CreateAudioPage {
|
||||||
|
pub settings: AudioSettings,
|
||||||
|
pub refs: References,
|
||||||
|
pub feed: FeedStore,
|
||||||
|
pub cost: CostState,
|
||||||
|
pub generating: bool,
|
||||||
|
box_state: PromptBoxState,
|
||||||
|
box_height: f32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What the reference row asks for.
|
||||||
|
#[derive(Debug, PartialEq)]
|
||||||
|
enum RowAction {
|
||||||
|
AddAudio,
|
||||||
|
AudioFromLibrary,
|
||||||
|
AddImage,
|
||||||
|
Remove(u64),
|
||||||
|
Toggle(u64),
|
||||||
|
}
|
||||||
|
|
||||||
|
impl CreateAudioPage {
|
||||||
|
pub fn new(settings: AudioSettings) -> Self {
|
||||||
|
let mut page = Self { settings, box_height: 138.0, ..Default::default() };
|
||||||
|
page.box_state.editor.focus_requested = true;
|
||||||
|
page
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn ui(&mut self, ui: &mut Ui, env: &mut Env<'_>) {
|
||||||
|
let model = common::selected_model(&env.catalog.audio_page, self.settings.model.as_deref(), DEFAULT_AUDIO_MODEL).cloned();
|
||||||
|
if let Some(m) = &model {
|
||||||
|
let fields = self.fields(m, true);
|
||||||
|
self.cost.update(ui, env.backend, Modality::Audio, &fields);
|
||||||
|
}
|
||||||
|
let mut feed = std::mem::take(&mut self.feed);
|
||||||
|
let mut box_height = self.box_height;
|
||||||
|
common::create_shell(ui, env, MediaKind::Audio, "Create Audio", "Describe a song, a sound, or a sample", &mut feed, &mut box_height, |ui, env| self.prompt_box(ui, env, model.as_ref()));
|
||||||
|
self.feed = feed;
|
||||||
|
self.box_height = box_height;
|
||||||
|
}
|
||||||
|
|
||||||
|
fn prompt_box(&mut self, ui: &mut Ui, env: &mut Env<'_>, model: Option<&ModelInfo>) {
|
||||||
|
let missing_ref = model.is_some_and(|m| self.missing_required_ref(m));
|
||||||
|
let accepts: Vec<RefKind> = model.map_or_else(Vec::new, |m| [(RefKind::Audio, m.audio_refs_max), (RefKind::Image, m.image_refs_max)].into_iter().filter(|(_, n)| *n > 0).map(|(k, _)| k).collect());
|
||||||
|
let props = PromptBoxProps { id: Id::new("audio-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: None, deck: DeckMode::None, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !missing_ref, generating: self.generating, generate_tooltip: "Generate", warning: missing_ref.then_some("Audio track required"), banner: None, extra_input: !self.settings.style_prompt.is_empty() };
|
||||||
|
|
||||||
|
let playing = self.refs.audios.iter().map(|r| r.id).find(|id| env.audio.is_playing(&ref_key(*id)));
|
||||||
|
let image = self.refs.images.first().and_then(|r| r.preview.as_deref()).and_then(|key| match env.cache.get(&env.ctx, key) {
|
||||||
|
Lookup::Ready(t) => Some(t),
|
||||||
|
_ => None,
|
||||||
|
});
|
||||||
|
let models = &env.catalog.audio_page;
|
||||||
|
let mut prompt = std::mem::take(&mut self.settings.prompt);
|
||||||
|
let settings = &mut self.settings;
|
||||||
|
let refs = &self.refs;
|
||||||
|
let mut picked_model = None;
|
||||||
|
let mut row_actions = Vec::new();
|
||||||
|
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
|
||||||
|
(Slot::ToolbarLeft, _) => {
|
||||||
|
if let Some(id) = model_selector(ui, Id::new("audio-model"), models, model, false) {
|
||||||
|
picked_model = Some(id);
|
||||||
|
}
|
||||||
|
if let Some(m) = model {
|
||||||
|
audio_toolbar(ui, &m.audio, settings);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
(Slot::AbovePrompt, Some(m)) => reference_row(ui, m, refs, playing, image.as_ref(), &mut row_actions),
|
||||||
|
(Slot::BelowPrompt, Some(m)) if m.audio.style_prompt => style_row(ui, &mut settings.style_prompt),
|
||||||
|
_ => {},
|
||||||
|
};
|
||||||
|
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
|
||||||
|
self.settings.prompt = prompt;
|
||||||
|
if let Some(id) = picked_model {
|
||||||
|
self.settings.model = Some(id);
|
||||||
|
}
|
||||||
|
if let Some(m) = model {
|
||||||
|
for action in row_actions {
|
||||||
|
self.row_action(action, env, m);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for action in actions {
|
||||||
|
self.handle(action, env, model);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn handle(&mut self, action: PromptBoxAction, env: &mut Env<'_>, model: Option<&ModelInfo>) {
|
||||||
|
match action {
|
||||||
|
PromptBoxAction::Generate => self.generate(env, model),
|
||||||
|
// Toggles and tuning stay as they are.
|
||||||
|
PromptBoxAction::ClearAll => {
|
||||||
|
self.settings.prompt.clear();
|
||||||
|
self.settings.style_prompt.clear();
|
||||||
|
self.refs.clear();
|
||||||
|
},
|
||||||
|
PromptBoxAction::DroppedFiles(paths) => {
|
||||||
|
if let Some(m) = model {
|
||||||
|
self.add_dropped(env, m, &paths);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
PromptBoxAction::PastedImage(png) => match model {
|
||||||
|
Some(m) if m.image_refs_max > 0 => {
|
||||||
|
self.make_room_for_image(env);
|
||||||
|
common::upload_png(env, &mut self.refs, png, ImageSlot::Reference);
|
||||||
|
},
|
||||||
|
_ => env.toasts.error("Image references aren't available here"),
|
||||||
|
},
|
||||||
|
PromptBoxAction::Deck(_) | PromptBoxAction::MentionPicked(..) => {},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn row_action(&mut self, action: RowAction, env: &mut Env<'_>, m: &ModelInfo) {
|
||||||
|
match action {
|
||||||
|
RowAction::AddAudio => {
|
||||||
|
let paths = common::pick_files(&[RefKind::Audio], m.audio_refs_max > 1);
|
||||||
|
self.add_audio(env, m, &paths);
|
||||||
|
},
|
||||||
|
RowAction::AudioFromLibrary => {
|
||||||
|
let max = m.audio_refs_max.saturating_sub(self.refs.audios.len()).max(1);
|
||||||
|
env.requests.push(AppRequest::PickFromLibrary { kind: RefKind::Audio, slot: ImageSlot::Reference, max, page: MediaKind::Audio });
|
||||||
|
},
|
||||||
|
RowAction::AddImage => {
|
||||||
|
if let Some(path) = common::pick_files(&[RefKind::Image], false).first() {
|
||||||
|
self.make_room_for_image(env);
|
||||||
|
common::upload_path(env, &mut self.refs, path, RefKind::Image, ImageSlot::Reference);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
RowAction::Remove(id) => {
|
||||||
|
if env.audio.is_playing(&ref_key(id)) {
|
||||||
|
env.audio.stop();
|
||||||
|
}
|
||||||
|
self.refs.remove(id);
|
||||||
|
},
|
||||||
|
RowAction::Toggle(id) => {
|
||||||
|
if let Some(url) = self.refs.audios.iter().find(|r| r.id == id).and_then(|r| r.full_url.clone()) {
|
||||||
|
env.requests.push(AppRequest::ToggleAudio { ref_id: id, url });
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Dropped files (`usePromptBoxDrop`): audio wins over images; only the first image counts.
|
||||||
|
fn add_dropped(&mut self, env: &mut Env<'_>, m: &ModelInfo, paths: &[PathBuf]) {
|
||||||
|
let audios: Vec<PathBuf> = paths.iter().filter(|p| RefKind::from_path(p) == Some(RefKind::Audio)).cloned().collect();
|
||||||
|
let image = paths.iter().find(|p| RefKind::from_path(p) == Some(RefKind::Image));
|
||||||
|
if !audios.is_empty() {
|
||||||
|
self.add_audio(env, m, &audios);
|
||||||
|
} else if let Some(path) = image {
|
||||||
|
if m.image_refs_max == 0 {
|
||||||
|
env.toasts.error("Image references aren't available here");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.make_room_for_image(env);
|
||||||
|
common::upload_path(env, &mut self.refs, path, RefKind::Image, ImageSlot::Reference);
|
||||||
|
} else if !paths.is_empty() {
|
||||||
|
env.toasts.error(format!("Only {} files can be added here", accepted_kinds(m)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Uploads audio tracks into the free slots (dropping an image reference: they don't mix).
|
||||||
|
fn add_audio(&mut self, env: &mut Env<'_>, m: &ModelInfo, paths: &[PathBuf]) {
|
||||||
|
if paths.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if m.audio_refs_max == 0 {
|
||||||
|
env.toasts.error("Audio references aren't available here");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let free = m.audio_refs_max.saturating_sub(self.refs.audios.len());
|
||||||
|
if paths.len() > free {
|
||||||
|
env.toasts.error(format!("Max {} audio track{}", m.audio_refs_max, if m.audio_refs_max == 1 { "" } else { "s" }));
|
||||||
|
}
|
||||||
|
if free == 0 {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.make_room_for_audio(env.toasts);
|
||||||
|
for path in paths.iter().take(free) {
|
||||||
|
common::upload_path(env, &mut self.refs, path, RefKind::Audio, ImageSlot::Reference);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Library picks, ready to attach: audio within the slots and the total-length cap.
|
||||||
|
pub fn attach_from_library(&mut self, toasts: &mut Toasts, catalog: &Catalog, kind: RefKind, picks: Vec<RefMedia>) {
|
||||||
|
let Some(m) = common::selected_model(&catalog.audio_page, self.settings.model.as_deref(), DEFAULT_AUDIO_MODEL) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if kind != RefKind::Audio || picks.is_empty() {
|
||||||
|
common::attach_from_library(&mut self.refs, kind, ImageSlot::Reference, picks);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.make_room_for_audio(toasts);
|
||||||
|
let mut total = self.refs.total_secs(RefKind::Audio);
|
||||||
|
let mut accepted = Vec::new();
|
||||||
|
for pick in picks.into_iter().take(m.audio_refs_max.saturating_sub(self.refs.audios.len())) {
|
||||||
|
if total + pick.duration_secs > MAX_REF_SECS {
|
||||||
|
toasts.error(duration_message());
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
total += pick.duration_secs;
|
||||||
|
accepted.push(pick);
|
||||||
|
}
|
||||||
|
common::attach_from_library(&mut self.refs, kind, ImageSlot::Reference, accepted);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// After an upload lands: keep the audio references under the total-length cap.
|
||||||
|
pub fn check_durations(&mut self, toasts: &mut Toasts, ref_id: u64) {
|
||||||
|
if self.refs.audios.iter().any(|r| r.id == ref_id) && self.refs.total_secs(RefKind::Audio) > MAX_REF_SECS + 0.5 {
|
||||||
|
self.refs.remove(ref_id);
|
||||||
|
toasts.error(duration_message());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Audio and image references don't mix: adding audio drops the image.
|
||||||
|
fn make_room_for_audio(&mut self, toasts: &mut Toasts) {
|
||||||
|
if !self.refs.images.is_empty() {
|
||||||
|
self.refs.images.clear();
|
||||||
|
toasts.error("Removed image reference \u{2014} it can't be combined with audio");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One image at most, and never with audio.
|
||||||
|
fn make_room_for_image(&mut self, env: &mut Env<'_>) {
|
||||||
|
if !self.refs.audios.is_empty() {
|
||||||
|
if self.refs.audios.iter().any(|r| env.audio.is_playing(&ref_key(r.id))) {
|
||||||
|
env.audio.stop();
|
||||||
|
}
|
||||||
|
self.refs.audios.clear();
|
||||||
|
env.toasts.error("Removed audio reference \u{2014} it can't be combined with an image");
|
||||||
|
}
|
||||||
|
self.refs.images.clear();
|
||||||
|
}
|
||||||
|
|
||||||
|
fn missing_required_ref(&self, m: &ModelInfo) -> bool {
|
||||||
|
m.audio.requires_audio_ref && self.refs.audios.len() != 1
|
||||||
|
}
|
||||||
|
|
||||||
|
fn generate(&mut self, env: &mut Env<'_>, model: Option<&ModelInfo>) {
|
||||||
|
let Some(m) = model.filter(|_| !self.generating && !self.settings.prompt.trim().is_empty()) else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
if self.missing_required_ref(m) {
|
||||||
|
env.toasts.error(format!("{} needs an audio track to work from \u{2014} add one first", m.name));
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if !env.signed_in {
|
||||||
|
env.requests.push(AppRequest::SignIn);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if self.refs.any_uploading() {
|
||||||
|
env.toasts.error("Please wait for your references to finish uploading");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
self.generating = true;
|
||||||
|
let meta = EnqueueMeta { prompt: self.settings.prompt.trim().to_owned(), model_id: m.id.clone(), batch_count: 1, ref_image: None };
|
||||||
|
env.backend.generate(Modality::Audio, self.fields(m, false), meta);
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn on_enqueued(&mut self, env: &mut Env<'_>, job_tokens: &[String], meta: &EnqueueMeta) {
|
||||||
|
self.generating = false;
|
||||||
|
self.feed.add_enqueued(MediaKind::Audio, job_tokens, meta, env.catalog);
|
||||||
|
env.toasts.success("Audio generation enqueued!");
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn on_enqueue_failed(&mut self, env: &mut Env<'_>, message: &str, status: Option<u16>) {
|
||||||
|
self.generating = false;
|
||||||
|
env.toasts.error(failure_message(message, status));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The OmniGen request (`buildAudioGenerationBody`): only what the model takes, nothing null.
|
||||||
|
/// The estimate counts references with placeholders and needs no prompt.
|
||||||
|
fn fields(&self, m: &ModelInfo, estimate: bool) -> Map<String, Value> {
|
||||||
|
let (s, caps) = (&self.settings, &m.audio);
|
||||||
|
let mut f = Map::new();
|
||||||
|
f.insert("model".into(), json!(m.id));
|
||||||
|
let tokens = |list: &[RefMedia]| -> Vec<String> {
|
||||||
|
if estimate {
|
||||||
|
vec!["placeholder".to_owned(); list.len()]
|
||||||
|
} else {
|
||||||
|
list.iter().filter_map(|r| r.token.clone()).collect()
|
||||||
|
}
|
||||||
|
};
|
||||||
|
for (key, max, list) in [("audio_media_tokens", m.audio_refs_max, &self.refs.audios), ("image_media_tokens", m.image_refs_max, &self.refs.images)] {
|
||||||
|
let list = tokens(list.as_slice());
|
||||||
|
if max > 0 && !list.is_empty() {
|
||||||
|
f.insert(key.into(), json!(list));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(hz) = caps.sample_rate(s.sample_rate_hz).filter(|_| !caps.sample_rates.is_empty()) {
|
||||||
|
f.insert("sample_rate_hz".into(), json!(hz));
|
||||||
|
}
|
||||||
|
if estimate {
|
||||||
|
return f;
|
||||||
|
}
|
||||||
|
let (prompt, style) = (s.prompt.trim(), s.style_prompt.trim());
|
||||||
|
if !prompt.is_empty() {
|
||||||
|
f.insert("prompt".into(), json!(prompt));
|
||||||
|
}
|
||||||
|
if caps.style_prompt && !style.is_empty() {
|
||||||
|
f.insert("style_prompt".into(), json!(style));
|
||||||
|
}
|
||||||
|
for (supported, key, value) in [(caps.keep_lyrics, "keep_lyrics", s.keep_lyrics), (caps.instrumental, "is_instrumental", s.instrumental), (caps.loopable, "is_loopable", s.loopable)] {
|
||||||
|
if supported {
|
||||||
|
f.insert(key.into(), json!(value));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(bpm) = s.bpm.filter(|_| caps.bpm) {
|
||||||
|
f.insert("bpm".into(), json!(bpm));
|
||||||
|
}
|
||||||
|
if caps.musical_key {
|
||||||
|
f.insert("musical_key".into(), json!(musical_key(&s.musical_key).0));
|
||||||
|
}
|
||||||
|
for (supported, key, value) in [(caps.speed, "speed", s.speed), (caps.volume, "volume", s.volume)] {
|
||||||
|
if supported {
|
||||||
|
f.insert(key.into(), json!(hundredths(value)));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if caps.pitch {
|
||||||
|
f.insert("pitch".into(), json!(s.pitch));
|
||||||
|
}
|
||||||
|
f
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The audio pickers after the model (`PromptBoxAudio`'s toolbar): the toggles the model offers,
|
||||||
|
/// Beat & Key, and Tuning.
|
||||||
|
fn audio_toolbar(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
|
||||||
|
let toggles = [(caps.instrumental, &mut s.instrumental, "audio-instrumental", Icon::MicOff, "Instrumental"), (caps.keep_lyrics, &mut s.keep_lyrics, "audio-keep-lyrics", Icon::Mic, "Keep lyrics"), (caps.loopable, &mut s.loopable, "audio-loop", Icon::Repeat, "Loop")];
|
||||||
|
for (supported, value, id, icon, label) in toggles {
|
||||||
|
if supported && pickers::toggle(ui, Id::new(id), icon, label, *value, &format!("{label}: {}", if *value { "ON" } else { "OFF" })) {
|
||||||
|
*value = !*value;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if caps.bpm || caps.musical_key {
|
||||||
|
beat_and_key(ui, caps, s);
|
||||||
|
}
|
||||||
|
if caps.has_tuning() {
|
||||||
|
tuning(ui, caps, s);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Beat & Key popover: a BPM slider with Auto, and the musical key grid.
|
||||||
|
fn beat_and_key(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
|
||||||
|
let mut parts = Vec::new();
|
||||||
|
if caps.bpm {
|
||||||
|
parts.push(s.bpm.map_or_else(|| "Auto BPM".to_owned(), |b| format!("{b} BPM")));
|
||||||
|
}
|
||||||
|
if caps.musical_key {
|
||||||
|
parts.push(musical_key(&s.musical_key).2.to_owned());
|
||||||
|
}
|
||||||
|
let label = if parts.is_empty() { "Beat".to_owned() } else { parts.join(" \u{b7} ") };
|
||||||
|
pickers::popover_pill(ui, Id::new("audio-beat"), Leading::Icon(Icon::Drum), &label, "Beat & Key", |ui| {
|
||||||
|
ui.set_width(POPOVER_WIDTH);
|
||||||
|
widgets::menu_header(ui, "Beat & Key");
|
||||||
|
padded(ui, |ui| {
|
||||||
|
if caps.bpm {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
setting_name(ui, "BPM");
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
|
setting_value(ui, &s.bpm.map_or_else(|| "Auto".to_owned(), |b| b.to_string()));
|
||||||
|
if s.bpm.is_some() && ui.add(egui::Button::new(egui::RichText::new("Auto").size(11.5).color(theme::ACCENT_INK)).frame(false)).on_hover_cursor(egui::CursorIcon::PointingHand).clicked() {
|
||||||
|
s.bpm = None;
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
let mut bpm = s.bpm.unwrap_or(DEFAULT_BPM);
|
||||||
|
if full_slider(ui, &mut bpm, BPM_RANGE, 1.0).changed() {
|
||||||
|
s.bpm = Some(bpm);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if caps.musical_key {
|
||||||
|
ui.add_space(6.0);
|
||||||
|
setting_name(ui, "Musical key");
|
||||||
|
ui.spacing_mut().item_spacing = vec2(4.0, 4.0);
|
||||||
|
let width = ui.available_width();
|
||||||
|
let (auto, keys) = MUSICAL_KEYS.split_first().expect("keys");
|
||||||
|
if choice_chip(ui, auto.2, s.musical_key == auto.0, width, auto.1).clicked() {
|
||||||
|
s.musical_key = auto.0.to_owned();
|
||||||
|
}
|
||||||
|
let cell = (width - 12.0) / 4.0;
|
||||||
|
for row in keys.chunks(4) {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
for (value, label, short) in row {
|
||||||
|
if choice_chip(ui, short, s.musical_key == *value, cell, label).clicked() {
|
||||||
|
s.musical_key = (*value).to_owned();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The Tuning popover: sample rate chips and the speed, volume and pitch sliders.
|
||||||
|
fn tuning(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
|
||||||
|
pickers::popover_pill(ui, Id::new("audio-tuning"), Leading::Icon(Icon::SlidersHorizontal), "Tuning", "Tuning", |ui| {
|
||||||
|
ui.set_width(POPOVER_WIDTH);
|
||||||
|
widgets::menu_header(ui, "Tuning");
|
||||||
|
padded(ui, |ui| {
|
||||||
|
if !caps.sample_rates.is_empty() {
|
||||||
|
setting_name(ui, "Sample rate");
|
||||||
|
let current = caps.sample_rate(s.sample_rate_hz);
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.spacing_mut().item_spacing = vec2(4.0, 4.0);
|
||||||
|
for hz in &caps.sample_rates {
|
||||||
|
let label = models::sample_rate_label(*hz);
|
||||||
|
if choice_chip(ui, &label, current == Some(*hz), 72.0, &label).clicked() {
|
||||||
|
s.sample_rate_hz = Some(*hz);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.add_space(6.0);
|
||||||
|
}
|
||||||
|
if caps.speed {
|
||||||
|
labeled_slider(ui, "Speed", &format!("{:.2}\u{d7}", s.speed), &mut s.speed, SPEED_RANGE, 0.05);
|
||||||
|
}
|
||||||
|
if caps.volume {
|
||||||
|
labeled_slider(ui, "Volume", &format!("{:.2}\u{d7}", s.volume), &mut s.volume, VOLUME_RANGE, 0.05);
|
||||||
|
}
|
||||||
|
if caps.pitch {
|
||||||
|
labeled_slider(ui, "Pitch", &format!("{}{} st", if s.pitch > 0 { "+" } else { "" }, s.pitch), &mut s.pitch, PITCH_RANGE, 1.0);
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The references above the prompt (`AudioReferenceRow`): the audio tracks (play, remove), add or
|
||||||
|
/// pick more while there's room, and the image when the model takes one.
|
||||||
|
fn reference_row(ui: &mut Ui, m: &ModelInfo, refs: &References, playing: Option<u64>, image: Option<&TextureHandle>, actions: &mut Vec<RowAction>) {
|
||||||
|
if m.audio_refs_max == 0 && m.image_refs_max == 0 {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
ui.horizontal_wrapped(|ui| {
|
||||||
|
ui.spacing_mut().item_spacing = vec2(8.0, 6.0);
|
||||||
|
if m.audio_refs_max > 0 {
|
||||||
|
icons::icon(ui, Icon::Music, 14.0, theme::MUTED);
|
||||||
|
ui.label(egui::RichText::new(format!("Audio Track ({}/{})", refs.audios.len(), m.audio_refs_max)).size(12.5).color(theme::MUTED));
|
||||||
|
if m.audio.requires_audio_ref && refs.audios.is_empty() {
|
||||||
|
ui.label(egui::RichText::new("required").size(12.5).color(theme::BAD));
|
||||||
|
}
|
||||||
|
for (i, r) in refs.audios.iter().enumerate() {
|
||||||
|
audio_tile(ui, i, r, playing == Some(r.id), actions);
|
||||||
|
}
|
||||||
|
if refs.audios.len() < m.audio_refs_max {
|
||||||
|
let uploading = refs.audios.iter().any(|r| r.status == RefStatus::Uploading);
|
||||||
|
if ui.add_enabled_ui(!uploading, |ui| widgets::pill(ui, Leading::Icon(Icon::Upload), if uploading { "Uploading\u{2026}" } else { "Add audio" }, false)).inner.clicked() {
|
||||||
|
actions.push(RowAction::AddAudio);
|
||||||
|
}
|
||||||
|
if widgets::pill(ui, Leading::Icon(Icon::Folder), "From library", false).clicked() {
|
||||||
|
actions.push(RowAction::AudioFromLibrary);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if m.image_refs_max > 0 {
|
||||||
|
ui.add_space(8.0);
|
||||||
|
ui.label(egui::RichText::new("Image").size(12.5).color(theme::MUTED));
|
||||||
|
match refs.images.first() {
|
||||||
|
Some(r) => image_tile(ui, r, image, actions),
|
||||||
|
None => {
|
||||||
|
if widgets::pill(ui, Leading::Icon(Icon::Image), "Add image", false).clicked() {
|
||||||
|
actions.push(RowAction::AddImage);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
ui.add_space(8.0);
|
||||||
|
hairline(ui);
|
||||||
|
ui.add_space(8.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One audio reference (`AudioRefTile`): play / stop, "Audio 1 · 12s", remove.
|
||||||
|
fn audio_tile(ui: &mut Ui, index: usize, r: &RefMedia, playing: bool, actions: &mut Vec<RowAction>) {
|
||||||
|
let uploading = r.status == RefStatus::Uploading;
|
||||||
|
let text = if uploading { format!("Audio {}", index + 1) } else { format!("Audio {} \u{b7} {}s", index + 1, r.duration_secs.round() as i64) };
|
||||||
|
let galley = ui.painter().layout_no_wrap(text, FontId::new(12.5, theme::medium()), theme::INK);
|
||||||
|
let (rect, _) = ui.allocate_exact_size(vec2(6.0 + 24.0 + 8.0 + galley.size().x + 6.0 + 20.0 + 6.0, theme::CONTROL_H), Sense::hover());
|
||||||
|
ui.painter().rect(rect, theme::RADIUS, theme::CONTROLS, Stroke::new(1.0, theme::LINE), egui::StrokeKind::Inside);
|
||||||
|
let play = Rect::from_center_size(pos2(rect.left() + 18.0, rect.center().y), vec2(24.0, 24.0));
|
||||||
|
if uploading {
|
||||||
|
icons::paint_spinner(ui, play.shrink(4.0), theme::MUTED);
|
||||||
|
} else {
|
||||||
|
let resp = ui.interact(play, Id::new(("audio-ref-play", r.id)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
|
||||||
|
let label = if playing { "Stop" } else { "Play" };
|
||||||
|
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
|
||||||
|
let fill = match (playing, resp.hovered()) {
|
||||||
|
(true, _) => theme::ACCENT,
|
||||||
|
(false, true) => theme::WASH_HOVER,
|
||||||
|
(false, false) => theme::WASH,
|
||||||
|
};
|
||||||
|
ui.painter().circle_filled(play.center(), 12.0, fill);
|
||||||
|
if playing {
|
||||||
|
ui.painter().rect_filled(Rect::from_center_size(play.center(), vec2(8.0, 8.0)), 1.0, Color32::WHITE);
|
||||||
|
} else {
|
||||||
|
icons::paint(ui.painter(), play.shrink(7.0).translate(vec2(1.0, 0.0)), Icon::Play, theme::INK);
|
||||||
|
}
|
||||||
|
if resp.clicked() {
|
||||||
|
actions.push(RowAction::Toggle(r.id));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
ui.painter().galley(pos2(play.right() + 8.0, rect.center().y - galley.size().y / 2.0), galley, theme::INK);
|
||||||
|
let remove = Rect::from_center_size(pos2(rect.right() - 16.0, rect.center().y), vec2(20.0, 20.0));
|
||||||
|
if x_button(ui, remove, Id::new(("audio-ref-remove", r.id)), "Remove audio", true) {
|
||||||
|
actions.push(RowAction::Remove(r.id));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The image reference: a 36 px thumbnail with a remove × on hover.
|
||||||
|
fn image_tile(ui: &mut Ui, r: &RefMedia, texture: Option<&TextureHandle>, actions: &mut Vec<RowAction>) {
|
||||||
|
let (rect, _) = ui.allocate_exact_size(vec2(36.0, 36.0), Sense::hover());
|
||||||
|
ui.painter().rect(rect, theme::RADIUS, theme::CONTROLS, Stroke::new(1.0, theme::LINE), egui::StrokeKind::Inside);
|
||||||
|
if let Some(t) = texture {
|
||||||
|
egui::Image::new(t).uv(cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, rect.shrink(1.0));
|
||||||
|
}
|
||||||
|
if r.status == RefStatus::Uploading {
|
||||||
|
icons::paint_spinner(ui, rect.shrink(9.0), Color32::WHITE);
|
||||||
|
}
|
||||||
|
let remove = Rect::from_min_size(pos2(rect.right() - 14.0, rect.top() - 4.0), vec2(18.0, 18.0));
|
||||||
|
let shown = ui.rect_contains_pointer(rect.union(remove));
|
||||||
|
if x_button(ui, remove, Id::new(("image-ref-remove", r.id)), "Remove image", shown) {
|
||||||
|
actions.push(RowAction::Remove(r.id));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The style line under the prompt (`StylePromptRow`).
|
||||||
|
fn style_row(ui: &mut Ui, style: &mut String) {
|
||||||
|
ui.add_space(6.0);
|
||||||
|
hairline(ui);
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
icons::icon(ui, Icon::Tags, 14.0, theme::DIM);
|
||||||
|
let clear_w = if style.is_empty() { 0.0 } else { 28.0 };
|
||||||
|
ui.add(egui::TextEdit::singleline(style).id_salt("audio-style").hint_text(STYLE_PLACEHOLDER).frame(egui::Frame::NONE).font(FontId::new(14.0, egui::FontFamily::Proportional)).desired_width(ui.available_width() - clear_w));
|
||||||
|
if !style.is_empty() {
|
||||||
|
let (rect, _) = ui.allocate_exact_size(vec2(20.0, 20.0), Sense::hover());
|
||||||
|
if x_button(ui, rect, Id::new("audio-style-clear"), "Clear style", true) {
|
||||||
|
style.clear();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A small round × at `rect`, painted while `shown` (it stays clickable for keyboards and tests).
|
||||||
|
fn x_button(ui: &mut Ui, rect: Rect, id: Id, label: &str, shown: bool) -> bool {
|
||||||
|
let resp = ui.interact(rect, id, Sense::click());
|
||||||
|
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
|
||||||
|
if shown || resp.hovered() {
|
||||||
|
let fill = if resp.hovered() { theme::fade(theme::DANGER, 0.7) } else { Color32::from_black_alpha(128) };
|
||||||
|
ui.painter().circle_filled(rect.center(), rect.width() / 2.0, fill);
|
||||||
|
icons::paint(ui.painter(), rect.shrink(rect.width() * 0.27), Icon::X, Color32::WHITE);
|
||||||
|
}
|
||||||
|
resp.on_hover_text(label).on_hover_cursor(egui::CursorIcon::PointingHand).clicked()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A selectable chip in the popovers (keys, sample rates): outlined, filled when selected.
|
||||||
|
fn choice_chip(ui: &mut Ui, label: &str, selected: bool, width: f32, tooltip: &str) -> Response {
|
||||||
|
let (rect, resp) = ui.allocate_exact_size(vec2(width, 28.0), Sense::click());
|
||||||
|
let (fill, border) = match (selected, resp.hovered()) {
|
||||||
|
(true, _) => (theme::WASH_HOVER, theme::fade(Color32::WHITE, 0.8)),
|
||||||
|
(false, true) => (theme::WASH, theme::LINE_STRONG),
|
||||||
|
(false, false) => (Color32::TRANSPARENT, theme::LINE),
|
||||||
|
};
|
||||||
|
ui.painter().rect(rect, theme::RADIUS, fill, Stroke::new(1.0, border), egui::StrokeKind::Inside);
|
||||||
|
ui.painter().text(rect.center(), Align2::CENTER_CENTER, label, FontId::new(12.5, theme::medium()), if selected { theme::INK } else { theme::MUTED });
|
||||||
|
resp.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::Button, true, selected, label));
|
||||||
|
resp.on_hover_text(tooltip).on_hover_cursor(egui::CursorIcon::PointingHand)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A setting's name, its value at the right, and a slider across the popover.
|
||||||
|
fn labeled_slider<N: egui::emath::Numeric>(ui: &mut Ui, name: &str, value: &str, number: &mut N, range: RangeInclusive<N>, step: f64) {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
setting_name(ui, name);
|
||||||
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| setting_value(ui, value));
|
||||||
|
});
|
||||||
|
full_slider(ui, number, range, step);
|
||||||
|
ui.add_space(4.0);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn full_slider<N: egui::emath::Numeric>(ui: &mut Ui, number: &mut N, range: RangeInclusive<N>, step: f64) -> Response {
|
||||||
|
ui.style_mut().spacing.slider_width = ui.available_width();
|
||||||
|
ui.add(egui::Slider::new(number, range).step_by(step).show_value(false))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn setting_name(ui: &mut Ui, name: &str) {
|
||||||
|
ui.label(egui::RichText::new(name).size(12.0).color(theme::MUTED));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn setting_value(ui: &mut Ui, value: &str) {
|
||||||
|
ui.label(egui::RichText::new(value).size(12.0).family(theme::semibold()).color(theme::INK));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn padded(ui: &mut Ui, content: impl FnOnce(&mut Ui)) {
|
||||||
|
egui::Frame::new().inner_margin(egui::Margin::symmetric(8, 6)).show(ui, content);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn hairline(ui: &mut Ui) {
|
||||||
|
let (line, _) = ui.allocate_exact_size(vec2(ui.available_width(), 1.0), Sense::hover());
|
||||||
|
ui.painter().rect_filled(line, 0.0, theme::fade(Color32::WHITE, 0.1));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The key a stored value names (Auto for anything unknown): value, label, short label.
|
||||||
|
fn musical_key(value: &str) -> &'static (&'static str, &'static str, &'static str) {
|
||||||
|
MUSICAL_KEYS.iter().find(|k| k.0 == value).unwrap_or(&MUSICAL_KEYS[0])
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The audio player's key for a reference clip.
|
||||||
|
fn ref_key(id: u64) -> String {
|
||||||
|
format!("ref:{id}")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A slider's value to two decimals (0.05 steps drift in `f32`).
|
||||||
|
fn hundredths(value: f32) -> f64 {
|
||||||
|
(f64::from(value) * 100.0).round() / 100.0
|
||||||
|
}
|
||||||
|
|
||||||
|
/// "audio", "audio or image" (for the drop toasts).
|
||||||
|
fn accepted_kinds(m: &ModelInfo) -> String {
|
||||||
|
let kinds: Vec<&str> = [("audio", m.audio_refs_max), ("image", m.image_refs_max)].into_iter().filter(|(_, n)| *n > 0).map(|(k, _)| k).collect();
|
||||||
|
if kinds.is_empty() {
|
||||||
|
"reference".to_owned()
|
||||||
|
} else {
|
||||||
|
kinds.join(" or ")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn duration_message() -> String {
|
||||||
|
format!("Total audio duration cannot exceed {}s", MAX_REF_SECS as u32)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// What a failed generate tells the user: out of credits, an outage, or the server's message.
|
||||||
|
fn failure_message(message: &str, status: Option<u16>) -> String {
|
||||||
|
match status {
|
||||||
|
Some(402) => "Not enough credits for this generation".to_owned(),
|
||||||
|
Some(code) if code >= 500 => OUTAGE_MESSAGE.to_owned(),
|
||||||
|
_ if message.is_empty() => "Failed to start audio generation".to_owned(),
|
||||||
|
_ => message.to_owned(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn model(json: Value) -> ModelInfo {
|
||||||
|
models::audio_model(&serde_json::from_value(json).expect("audio model"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The server's audio models (`configs/omni_gen/audio_models.rs`).
|
||||||
|
fn suno_music() -> ModelInfo {
|
||||||
|
model(json!({ "model": "suno_music", "style_prompt_supported": true, "instrumental_toggle_supported": true }))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn suno_sounds() -> ModelInfo {
|
||||||
|
model(json!({ "model": "suno_sounds", "loopable_toggle_supported": true, "bpm_supported": true, "musical_key_supported": true }))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn suno_remix() -> ModelInfo {
|
||||||
|
model(json!({ "model": "suno_remix", "style_prompt_supported": true, "keep_lyrics_supported": true, "audio_references_supported": true, "audio_references_max": 1 }))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn seed_audio() -> ModelInfo {
|
||||||
|
model(json!({ "model": "seed_audio_1p0", "audio_references_supported": true, "audio_references_max": 3, "image_references_supported": true, "image_references_max": 1, "sample_rate_hz_options": [8000, 16000, 24000, 32000, 44100, 48000], "sample_rate_hz_default": 24000, "speed_supported": true, "volume_supported": true, "pitch_supported": true }))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn page(settings: AudioSettings) -> CreateAudioPage {
|
||||||
|
CreateAudioPage { settings, ..Default::default() }
|
||||||
|
}
|
||||||
|
|
||||||
|
fn audio_ref(token: &str, secs: f32) -> RefMedia {
|
||||||
|
RefMedia::from_library(RefKind::Audio, token.into(), None, Some(format!("https://cdn/{token}.mp3")), secs)
|
||||||
|
}
|
||||||
|
|
||||||
|
mod request_bodies {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn suno_music_sends_only_what_it_takes() {
|
||||||
|
let p = page(AudioSettings { prompt: " lofi beat ".into(), style_prompt: " chill ".into(), instrumental: true, keep_lyrics: true, bpm: Some(90), ..Default::default() });
|
||||||
|
assert_eq!(Value::Object(p.fields(&suno_music(), false)), json!({ "model": "suno_music", "prompt": "lofi beat", "style_prompt": "chill", "is_instrumental": true }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn suno_sounds_sends_its_toggles_beat_and_key() {
|
||||||
|
let mut p = page(AudioSettings { prompt: "door creak".into(), ..Default::default() });
|
||||||
|
assert_eq!(Value::Object(p.fields(&suno_sounds(), false)), json!({ "model": "suno_sounds", "prompt": "door creak", "is_loopable": false, "musical_key": "auto" }), "Auto BPM is left to the model");
|
||||||
|
p.settings.bpm = Some(128);
|
||||||
|
p.settings.musical_key = "c_minor".into();
|
||||||
|
let f = p.fields(&suno_sounds(), false);
|
||||||
|
assert_eq!((f["bpm"].clone(), f["musical_key"].clone()), (json!(128), json!("c_minor")));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn seed_audio_sends_references_and_tuning() {
|
||||||
|
let mut p = page(AudioSettings { prompt: "rain".into(), speed: 1.05, volume: 0.5, pitch: -3, sample_rate_hz: Some(11025), ..Default::default() });
|
||||||
|
p.refs.audios = vec![audio_ref("mf_a", 10.0), audio_ref("mf_b", 20.0)];
|
||||||
|
assert_eq!(Value::Object(p.fields(&seed_audio(), false)), json!({ "model": "seed_audio_1p0", "prompt": "rain", "audio_media_tokens": ["mf_a", "mf_b"], "sample_rate_hz": 24000, "speed": 1.05, "volume": 0.5, "pitch": -3 }));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn estimates_count_references_without_the_prompt() {
|
||||||
|
let mut p = page(AudioSettings { prompt: "rain".into(), sample_rate_hz: Some(44100), ..Default::default() });
|
||||||
|
p.refs.audios = vec![RefMedia::uploading(RefKind::Audio, None), audio_ref("mf_b", 5.0)];
|
||||||
|
assert_eq!(Value::Object(p.fields(&seed_audio(), true)), json!({ "model": "seed_audio_1p0", "audio_media_tokens": ["placeholder", "placeholder"], "sample_rate_hz": 44100 }));
|
||||||
|
assert_eq!(Value::Object(p.fields(&suno_music(), true)), json!({ "model": "suno_music" }), "references the model can't take don't count");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
mod rules {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn remixes_need_exactly_one_track() {
|
||||||
|
let mut p = page(AudioSettings::default());
|
||||||
|
assert!(p.missing_required_ref(&suno_remix()));
|
||||||
|
assert!(!p.missing_required_ref(&suno_music()));
|
||||||
|
p.refs.audios.push(audio_ref("mf_a", 30.0));
|
||||||
|
assert!(!p.missing_required_ref(&suno_remix()));
|
||||||
|
p.refs.audios.push(audio_ref("mf_b", 30.0));
|
||||||
|
assert!(p.missing_required_ref(&suno_remix()), "two tracks are too many");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn library_picks_respect_slots_the_cap_and_drop_the_image() {
|
||||||
|
let mut catalog = Catalog::default();
|
||||||
|
catalog.set_audio(vec![seed_audio()]);
|
||||||
|
let mut p = page(AudioSettings { model: Some("seed_audio_1p0".into()), ..Default::default() });
|
||||||
|
p.refs.images.push(RefMedia::from_library(RefKind::Image, "mf_img".into(), None, None, 0.0));
|
||||||
|
let mut toasts = Toasts::default();
|
||||||
|
p.attach_from_library(&mut toasts, &catalog, RefKind::Audio, vec![audio_ref("mf_a", 400.0), audio_ref("mf_b", 300.0), audio_ref("mf_c", 10.0)]);
|
||||||
|
let tokens: Vec<_> = p.refs.audios.iter().filter_map(|r| r.token.as_deref()).collect();
|
||||||
|
assert_eq!(tokens, ["mf_a"], "the second would pass 600s, so picking stops there");
|
||||||
|
assert!(p.refs.images.is_empty(), "audio and an image don't mix");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn uploads_past_the_cap_are_dropped() {
|
||||||
|
let mut p = page(AudioSettings::default());
|
||||||
|
p.refs.audios = vec![audio_ref("mf_a", 590.0), audio_ref("mf_b", 30.0)];
|
||||||
|
let late = p.refs.audios[1].id;
|
||||||
|
p.check_durations(&mut Toasts::default(), late);
|
||||||
|
assert_eq!(p.refs.audios.len(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn failures_explain_credits_and_outages() {
|
||||||
|
assert_eq!(failure_message("Payment required", Some(402)), "Not enough credits for this generation");
|
||||||
|
assert_eq!(failure_message("boom", Some(503)), OUTAGE_MESSAGE);
|
||||||
|
assert_eq!(failure_message("Prompt too long", Some(400)), "Prompt too long");
|
||||||
|
assert_eq!(failure_message("", None), "Failed to start audio generation");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The toolbar headlessly (egui_kittest): toggles, Beat & Key and Tuning.
|
||||||
|
mod toolbar {
|
||||||
|
use egui_kittest::Harness;
|
||||||
|
use egui_kittest::kittest::{NodeT, Queryable};
|
||||||
|
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
struct State {
|
||||||
|
caps: AudioCaps,
|
||||||
|
settings: AudioSettings,
|
||||||
|
fonts_ready: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn toggles_show_per_model_and_flip() {
|
||||||
|
let mut harness = build(suno_music().audio);
|
||||||
|
assert!(harness.query_by_label("Loop").is_none(), "Suno Music has no loop toggle");
|
||||||
|
harness.get_by_label("Instrumental").click();
|
||||||
|
harness.run();
|
||||||
|
assert!(harness.state().settings.instrumental);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn beat_and_key_sets_the_key_and_bpm() {
|
||||||
|
let mut harness = build(suno_sounds().audio);
|
||||||
|
harness.get_by_label("Auto BPM \u{b7} Auto").click();
|
||||||
|
harness.run();
|
||||||
|
harness.get_by_label("Cm").click();
|
||||||
|
harness.run();
|
||||||
|
assert_eq!(harness.state().settings.musical_key, "c_minor");
|
||||||
|
assert!(harness.query_by_label("Auto BPM \u{b7} Cm").is_some(), "the pill shows the key");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tuning_picks_a_sample_rate() {
|
||||||
|
let mut harness = build(seed_audio().audio);
|
||||||
|
harness.get_by_label("Tuning").click();
|
||||||
|
harness.run();
|
||||||
|
// egui reports a selected button as toggled.
|
||||||
|
assert!(harness.query_by_label("24 kHz").is_some_and(|n| n.accesskit_node().toggled() == Some(egui::accesskit::Toggled::True)), "the default is selected");
|
||||||
|
harness.get_by_label("44.1 kHz").click();
|
||||||
|
harness.run();
|
||||||
|
assert_eq!(harness.state().settings.sample_rate_hz, Some(44100));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn build(caps: AudioCaps) -> Harness<'static, State> {
|
||||||
|
let mut harness = Harness::builder().with_size(egui::vec2(1000.0, 600.0)).build_ui_state(
|
||||||
|
|ui, s: &mut State| {
|
||||||
|
if !s.fonts_ready {
|
||||||
|
crate::theme::apply(ui.ctx());
|
||||||
|
s.fonts_ready = true;
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
ui.add_space(400.0);
|
||||||
|
ui.horizontal(|ui| audio_toolbar(ui, &s.caps, &mut s.settings));
|
||||||
|
},
|
||||||
|
State { caps, settings: AudioSettings::default(), fonts_ready: false },
|
||||||
|
);
|
||||||
|
harness.run();
|
||||||
|
harness
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -15,7 +15,7 @@ use crate::pages::toolbar;
|
|||||||
use crate::prompt_box::deck::{DeckAction, DeckLimits};
|
use crate::prompt_box::deck::{DeckAction, DeckLimits};
|
||||||
use crate::prompt_box::model_selector::model_selector;
|
use crate::prompt_box::model_selector::model_selector;
|
||||||
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
|
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
|
||||||
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, editor};
|
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot, editor};
|
||||||
|
|
||||||
const PLACEHOLDER: &str = "Describe what you want in the image...";
|
const PLACEHOLDER: &str = "Describe what you want in the image...";
|
||||||
|
|
||||||
@@ -69,38 +69,41 @@ impl CreateImagePage {
|
|||||||
let deck = if max_images > 0 { DeckMode::References(DeckLimits { max_images, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: env.signed_in }) } else { DeckMode::None };
|
let deck = if max_images > 0 { DeckMode::References(DeckLimits { max_images, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: env.signed_in }) } else { DeckMode::None };
|
||||||
let accepts: &[RefKind] = if max_images > 0 { &[RefKind::Image] } else { &[] };
|
let accepts: &[RefKind] = if max_images > 0 { &[RefKind::Image] } else { &[] };
|
||||||
let prompt_max = model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max);
|
let prompt_max = model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max);
|
||||||
let props = PromptBoxProps { id: Id::new("image-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: Some(prompt_max), deck, accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some(), generating: self.generating, generate_tooltip: "Generate", warning: None, banner: None };
|
let props = PromptBoxProps { id: Id::new("image-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: Some(prompt_max), deck, accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some(), generating: self.generating, generate_tooltip: "Generate", warning: None, banner: None, extra_input: false };
|
||||||
let models = &env.catalog.image_page;
|
let models = &env.catalog.image_page;
|
||||||
let settings = &mut self.settings;
|
let settings = &mut self.settings;
|
||||||
let mut picked_model = None;
|
|
||||||
let mut left = |ui: &mut Ui| {
|
|
||||||
if let Some(id) = model_selector(ui, Id::new("image-model"), models, model, false) {
|
|
||||||
picked_model = Some(id);
|
|
||||||
}
|
|
||||||
let Some(m) = model else {
|
|
||||||
return;
|
|
||||||
};
|
|
||||||
let aspect = ModelInfo::resolve(settings.aspect_ratio.as_deref(), &m.aspect_ratios, m.aspect_default.as_ref());
|
|
||||||
if let Some(v) = toolbar::aspect_ratio(ui, Id::new("image-aspect"), m, aspect.as_deref().unwrap_or("")) {
|
|
||||||
settings.aspect_ratio = Some(v);
|
|
||||||
}
|
|
||||||
let resolution = ModelInfo::resolve(settings.resolution.as_deref(), &m.resolutions, m.resolution_default.as_ref());
|
|
||||||
if let Some(v) = toolbar::resolution(ui, Id::new("image-resolution"), m, resolution.as_deref().unwrap_or("")) {
|
|
||||||
settings.resolution = Some(v);
|
|
||||||
}
|
|
||||||
let quality = ModelInfo::resolve(settings.quality.as_deref(), &m.qualities, m.quality_default.as_ref());
|
|
||||||
if let Some(v) = toolbar::quality(ui, Id::new("image-quality"), m, quality.as_deref().unwrap_or("")) {
|
|
||||||
settings.quality = Some(v);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
let count = settings.count;
|
let count = settings.count;
|
||||||
|
let mut picked_model = None;
|
||||||
let mut picked_count = None;
|
let mut picked_count = None;
|
||||||
let mut right = |ui: &mut Ui| {
|
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
|
||||||
if let Some(m) = model {
|
(Slot::ToolbarLeft, _) => {
|
||||||
picked_count = toolbar::count(ui, Id::new("image-count"), m, count, "images");
|
if let Some(id) = model_selector(ui, Id::new("image-model"), models, model, false) {
|
||||||
}
|
picked_model = Some(id);
|
||||||
|
}
|
||||||
|
let Some(m) = model else {
|
||||||
|
return;
|
||||||
|
};
|
||||||
|
let aspect = ModelInfo::resolve(settings.aspect_ratio.as_deref(), &m.aspect_ratios, m.aspect_default.as_ref());
|
||||||
|
if let Some(v) = toolbar::aspect_ratio(ui, Id::new("image-aspect"), m, aspect.as_deref().unwrap_or("")) {
|
||||||
|
settings.aspect_ratio = Some(v);
|
||||||
|
}
|
||||||
|
let resolution = ModelInfo::resolve(settings.resolution.as_deref(), &m.resolutions, m.resolution_default.as_ref());
|
||||||
|
if let Some(v) = toolbar::resolution(ui, Id::new("image-resolution"), m, resolution.as_deref().unwrap_or("")) {
|
||||||
|
settings.resolution = Some(v);
|
||||||
|
}
|
||||||
|
let quality = ModelInfo::resolve(settings.quality.as_deref(), &m.qualities, m.quality_default.as_ref());
|
||||||
|
if let Some(v) = toolbar::quality(ui, Id::new("image-quality"), m, quality.as_deref().unwrap_or("")) {
|
||||||
|
settings.quality = Some(v);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
(Slot::ToolbarRight, Some(m)) => {
|
||||||
|
if let Some(n) = toolbar::count(ui, Id::new("image-count"), m, count, "images") {
|
||||||
|
picked_count = Some(n);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {},
|
||||||
};
|
};
|
||||||
let actions = prompt_box::show(ui, &mut settings.prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut left, &mut right);
|
let actions = prompt_box::show(ui, &mut settings.prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
|
||||||
if let Some(id) = picked_model {
|
if let Some(id) = picked_model {
|
||||||
self.settings.model = Some(id);
|
self.settings.model = Some(id);
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ use crate::prompt_box::editor::{self, MentionItem, MentionKind};
|
|||||||
use crate::prompt_box::model_selector::model_selector;
|
use crate::prompt_box::model_selector::model_selector;
|
||||||
use crate::prompt_box::pickers::{self, PickOption, Trigger};
|
use crate::prompt_box::pickers::{self, PickOption, Trigger};
|
||||||
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
|
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
|
||||||
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState};
|
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot};
|
||||||
use crate::ui::icons::Icon;
|
use crate::ui::icons::Icon;
|
||||||
use crate::ui::widgets::Leading;
|
use crate::ui::widgets::Leading;
|
||||||
|
|
||||||
@@ -113,7 +113,7 @@ impl CreateVideoPage {
|
|||||||
}
|
}
|
||||||
let needs_start = model.is_some_and(|m| self.needs_start_frame(m));
|
let needs_start = model.is_some_and(|m| self.needs_start_frame(m));
|
||||||
let banner = model.filter(|m| !m.text_to_video && !self.has_image_input()).map(|_| TEXT_ONLY_BANNER);
|
let banner = model.filter(|m| !m.text_to_video && !self.has_image_input()).map(|_| TEXT_ONLY_BANNER);
|
||||||
let props = PromptBoxProps { id: Id::new("video-prompt-box"), placeholder: if reference { PLACEHOLDER_REFERENCE } else { PLACEHOLDER }, enter_to_generate: env.enter_to_generate, mentions: &mentions, max_length: Some(model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max)), deck, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !needs_start, generating: self.generating, generate_tooltip: if needs_start { "Add a starting image before generating" } else { "Generate" }, warning: needs_start.then_some("Starting frame required"), banner };
|
let props = PromptBoxProps { id: Id::new("video-prompt-box"), placeholder: if reference { PLACEHOLDER_REFERENCE } else { PLACEHOLDER }, enter_to_generate: env.enter_to_generate, mentions: &mentions, max_length: Some(model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max)), deck, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !needs_start, generating: self.generating, generate_tooltip: if needs_start { "Add a starting image before generating" } else { "Generate" }, warning: needs_start.then_some("Starting frame required"), banner, extra_input: false };
|
||||||
|
|
||||||
let models = &env.catalog.video_page;
|
let models = &env.catalog.video_page;
|
||||||
let mut prompt = std::mem::take(&mut self.settings.prompt);
|
let mut prompt = std::mem::take(&mut self.settings.prompt);
|
||||||
@@ -122,25 +122,28 @@ impl CreateVideoPage {
|
|||||||
let with_refs = reference && !self.refs.images.is_empty();
|
let with_refs = reference && !self.refs.images.is_empty();
|
||||||
let mut picked_model = None;
|
let mut picked_model = None;
|
||||||
let mut picked_mode = None;
|
let mut picked_mode = None;
|
||||||
let mut open_characters = false;
|
|
||||||
let mut left = |ui: &mut Ui| {
|
|
||||||
if let Some(id) = model_selector(ui, Id::new("video-model"), models, model, true) {
|
|
||||||
picked_model = Some(id);
|
|
||||||
}
|
|
||||||
if let Some(m) = model {
|
|
||||||
picked_mode = video_toolbar(ui, m, settings, with_refs).or(picked_mode);
|
|
||||||
if characters_on && crate::prompt_box::pickers::toggle(ui, Id::new("video-characters"), Icon::User, "@Characters", false, "Characters") {
|
|
||||||
open_characters = true;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
};
|
|
||||||
let mut picked_count = None;
|
let mut picked_count = None;
|
||||||
let mut right = |ui: &mut Ui| {
|
let mut open_characters = false;
|
||||||
if let Some(m) = model {
|
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
|
||||||
picked_count = toolbar::count(ui, Id::new("video-count"), m, count, "videos");
|
(Slot::ToolbarLeft, _) => {
|
||||||
}
|
if let Some(id) = model_selector(ui, Id::new("video-model"), models, model, true) {
|
||||||
|
picked_model = Some(id);
|
||||||
|
}
|
||||||
|
if let Some(m) = model {
|
||||||
|
picked_mode = video_toolbar(ui, m, settings, with_refs).or(picked_mode);
|
||||||
|
if characters_on && pickers::toggle(ui, Id::new("video-characters"), Icon::User, "@Characters", false, "Characters") {
|
||||||
|
open_characters = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
(Slot::ToolbarRight, Some(m)) => {
|
||||||
|
if let Some(n) = toolbar::count(ui, Id::new("video-count"), m, count, "videos") {
|
||||||
|
picked_count = Some(n);
|
||||||
|
}
|
||||||
|
},
|
||||||
|
_ => {},
|
||||||
};
|
};
|
||||||
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut left, &mut right);
|
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
|
||||||
self.settings.prompt = prompt;
|
self.settings.prompt = prompt;
|
||||||
if let Some(id) = picked_model {
|
if let Some(id) = picked_model {
|
||||||
self.settings.model = Some(id);
|
self.settings.model = Some(id);
|
||||||
@@ -479,7 +482,7 @@ impl CreateVideoPage {
|
|||||||
"audioref" => RefKind::Audio,
|
"audioref" => RefKind::Audio,
|
||||||
_ => RefKind::Image,
|
_ => RefKind::Image,
|
||||||
};
|
};
|
||||||
let item = RefMedia::from_library(kind, c.media_token.clone(), c.media_links.thumbnail(256), Some(c.media_links.cdn_url.clone()), 0.0);
|
let item = RefMedia::from_library(kind, c.media_token.clone(), c.media_links.thumbnail(256).filter(|_| kind != RefKind::Audio), Some(c.media_links.cdn_url.clone()), 0.0);
|
||||||
match (c.semantic.as_str(), kind) {
|
match (c.semantic.as_str(), kind) {
|
||||||
("vid_end_frame", _) => self.refs.last_frame = Some(item),
|
("vid_end_frame", _) => self.refs.last_frame = Some(item),
|
||||||
(_, RefKind::Image) if s.input_mode == InputMode::Keyframe && self.refs.first_frame.is_none() => self.refs.first_frame = Some(item),
|
(_, RefKind::Image) if s.input_mode == InputMode::Keyframe && self.refs.first_frame.is_none() => self.refs.first_frame = Some(item),
|
||||||
|
|||||||
@@ -1,4 +1,5 @@
|
|||||||
pub mod common;
|
pub mod common;
|
||||||
|
pub mod create_audio;
|
||||||
pub mod create_image;
|
pub mod create_image;
|
||||||
pub mod create_video;
|
pub mod create_video;
|
||||||
pub mod other;
|
pub mod other;
|
||||||
|
|||||||
@@ -23,12 +23,13 @@ pub fn home(ui: &mut Ui, signed_in: bool, requests: &mut Vec<AppRequest>) -> Opt
|
|||||||
ui.add_space(10.0);
|
ui.add_space(10.0);
|
||||||
ui.label(widgets::heading("Capable tools for artists.", 52.0));
|
ui.label(widgets::heading("Capable tools for artists.", 52.0));
|
||||||
ui.add_space(10.0);
|
ui.add_space(10.0);
|
||||||
ui.label(egui::RichText::new("Generate images and videos with the best models, steer them with references and keyframes, and keep everything in your library.").size(17.0).color(theme::MUTED));
|
ui.label(egui::RichText::new("Generate images, videos and audio with the best models, steer them with references and keyframes, and keep everything in your library.").size(17.0).color(theme::MUTED));
|
||||||
ui.add_space(28.0);
|
ui.add_space(28.0);
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
ui.spacing_mut().item_spacing.x = 16.0;
|
ui.spacing_mut().item_spacing.x = 16.0;
|
||||||
let card_w = (width - 16.0) / 2.0;
|
let cards = [(Page::CreateImage, Icon::Image, "Create Image", "Describe anything. See it in seconds."), (Page::CreateVideo, Icon::Video, "Create Video", "Describe a scene. See it in motion."), (Page::CreateAudio, Icon::Music, "Create Audio", "Describe a song, a sound, or a sample.")];
|
||||||
for (page, icon, title, text) in [(Page::CreateImage, Icon::Image, "Create Image", "Describe anything. See it in seconds."), (Page::CreateVideo, Icon::Video, "Create Video", "Describe a scene. See it in motion.")] {
|
let card_w = (width - 16.0 * (cards.len() - 1) as f32) / cards.len() as f32;
|
||||||
|
for (page, icon, title, text) in cards {
|
||||||
if card(ui, card_w, icon, title, text) {
|
if card(ui, card_w, icon, title, text) {
|
||||||
open = Some(page);
|
open = Some(page);
|
||||||
}
|
}
|
||||||
@@ -75,7 +76,7 @@ pub fn library(ui: &mut Ui, env: &mut Env<'_>, feed: &mut FeedStore) {
|
|||||||
}
|
}
|
||||||
let view = FeedView { id: Id::new("library-feed"), mode: env.view_mode, pending: &[], failed: &[], items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: true, autoplay: env.autoplay, bottom_padding: 24.0 };
|
let view = FeedView { id: Id::new("library-feed"), mode: env.view_mode, pending: &[], failed: &[], items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: true, autoplay: env.autoplay, bottom_padding: 24.0 };
|
||||||
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0)).with_min_y(area.top() + 2.0)));
|
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0)).with_min_y(area.top() + 2.0)));
|
||||||
for action in grid::show(&mut child, &view, env.cache, env.ratios, env.catalog) {
|
for action in grid::show(&mut child, &view, env.cache, env.ratios, env.audio, env.catalog) {
|
||||||
match action {
|
match action {
|
||||||
FeedAction::Open(token) => {
|
FeedAction::Open(token) => {
|
||||||
let kind = feed.find(&token).map_or(MediaKind::Image, |i| i.kind);
|
let kind = feed.find(&token).map_or(MediaKind::Image, |i| i.kind);
|
||||||
|
|||||||
@@ -58,6 +58,21 @@ pub struct PromptBoxProps<'a> {
|
|||||||
pub warning: Option<&'a str>,
|
pub warning: Option<&'a str>,
|
||||||
/// An info strip above the prompt.
|
/// An info strip above the prompt.
|
||||||
pub banner: Option<&'a str>,
|
pub banner: Option<&'a str>,
|
||||||
|
/// Something clear-all also clears besides the prompt and references (the audio style).
|
||||||
|
pub extra_input: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where a page draws into the box.
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
|
||||||
|
pub enum Slot {
|
||||||
|
/// Rows above the prompt (the audio box's references).
|
||||||
|
AbovePrompt,
|
||||||
|
/// Rows below the prompt (the audio box's style).
|
||||||
|
BelowPrompt,
|
||||||
|
/// The toolbar's left: the model selector and pickers.
|
||||||
|
ToolbarLeft,
|
||||||
|
/// The toolbar's right, before clear-all: the count picker.
|
||||||
|
ToolbarRight,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Something the page should do.
|
/// Something the page should do.
|
||||||
@@ -81,11 +96,10 @@ pub struct PromptBoxState {
|
|||||||
focused: bool,
|
focused: bool,
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Draws the box in `ui` (which should be as wide as the box). `toolbar_left` holds the model
|
/// Draws the box in `ui` (which should be as wide as the box). `slots` draws the page's parts
|
||||||
/// selector and pickers; `toolbar_right` the count picker. Both may be drawn twice in a frame
|
/// into each [`Slot`]; the toolbar's left may be drawn twice in a frame (box and focus mode), each
|
||||||
/// (box and focus mode), each under its own id.
|
/// under its own id.
|
||||||
#[allow(clippy::too_many_arguments)]
|
pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, slots: &mut dyn FnMut(&mut Ui, Slot)) -> Vec<PromptBoxAction> {
|
||||||
pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, toolbar_left: &mut dyn FnMut(&mut Ui), toolbar_right: &mut dyn FnMut(&mut Ui)) -> Vec<PromptBoxAction> {
|
|
||||||
let mut actions = Vec::new();
|
let mut actions = Vec::new();
|
||||||
let frame = egui::Frame::new().fill(theme::fade(theme::CONTROLS, 0.95)).stroke(Stroke::new(1.0, if state.focused { theme::ACCENT } else { theme::LINE })).corner_radius(theme::RADIUS).inner_margin(16);
|
let frame = egui::Frame::new().fill(theme::fade(theme::CONTROLS, 0.95)).stroke(Stroke::new(1.0, if state.focused { theme::ACCENT } else { theme::LINE })).corner_radius(theme::RADIUS).inner_margin(16);
|
||||||
let response = frame.show(ui, |ui| {
|
let response = frame.show(ui, |ui| {
|
||||||
@@ -96,6 +110,7 @@ pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut Pro
|
|||||||
ui.spacing_mut().item_spacing.x = 12.0;
|
ui.spacing_mut().item_spacing.x = 12.0;
|
||||||
deck_widget(ui, props, refs, cache, false, &mut actions);
|
deck_widget(ui, props, refs, cache, false, &mut actions);
|
||||||
ui.vertical(|ui| {
|
ui.vertical(|ui| {
|
||||||
|
slots(ui, Slot::AbovePrompt);
|
||||||
let out = editor_ui(ui, prompt, state, props, cache, None);
|
let out = editor_ui(ui, prompt, state, props, cache, None);
|
||||||
state.focused = out.focused;
|
state.focused = out.focused;
|
||||||
if out.submit {
|
if out.submit {
|
||||||
@@ -107,17 +122,18 @@ pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut Pro
|
|||||||
if out.open_fullscreen {
|
if out.open_fullscreen {
|
||||||
state.fullscreen = true;
|
state.fullscreen = true;
|
||||||
}
|
}
|
||||||
|
slots(ui, Slot::BelowPrompt);
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
ui.add_space(14.0);
|
ui.add_space(14.0);
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
ui.spacing_mut().item_spacing.x = 8.0;
|
ui.spacing_mut().item_spacing.x = 8.0;
|
||||||
toolbar_left(ui);
|
slots(ui, Slot::ToolbarLeft);
|
||||||
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
ui.spacing_mut().item_spacing.x = 6.0;
|
ui.spacing_mut().item_spacing.x = 6.0;
|
||||||
generate_cluster(ui, props, &mut actions);
|
generate_cluster(ui, props, &mut actions);
|
||||||
clear_all_button(ui, prompt, refs, state, &mut actions);
|
clear_all_button(ui, prompt, refs, state, props.extra_input, &mut actions);
|
||||||
toolbar_right(ui);
|
slots(ui, Slot::ToolbarRight);
|
||||||
if let Some(warning) = props.warning {
|
if let Some(warning) = props.warning {
|
||||||
warning_label(ui, warning);
|
warning_label(ui, warning);
|
||||||
}
|
}
|
||||||
@@ -129,7 +145,7 @@ pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut Pro
|
|||||||
drop_target(ui, props, rect, &mut actions);
|
drop_target(ui, props, rect, &mut actions);
|
||||||
paste_images(ui, props, &mut actions);
|
paste_images(ui, props, &mut actions);
|
||||||
if state.fullscreen {
|
if state.fullscreen {
|
||||||
focus_mode(ui.ctx(), prompt, refs, state, props, cache, toolbar_left, &mut actions);
|
focus_mode(ui.ctx(), prompt, refs, state, props, cache, slots, &mut actions);
|
||||||
}
|
}
|
||||||
if state.confirm_clear {
|
if state.confirm_clear {
|
||||||
confirm_clear(ui.ctx(), props.id, state, &mut actions);
|
confirm_clear(ui.ctx(), props.id, state, &mut actions);
|
||||||
@@ -172,10 +188,10 @@ fn generate_cluster(ui: &mut Ui, props: &PromptBoxProps<'_>, actions: &mut Vec<P
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// The eraser button and the hairline after it (`PromptClearAllButton`).
|
/// The eraser button and the hairline after it (`PromptClearAllButton`).
|
||||||
fn clear_all_button(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, actions: &mut Vec<PromptBoxAction>) {
|
fn clear_all_button(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, extra_input: bool, actions: &mut Vec<PromptBoxAction>) {
|
||||||
let (rule, _) = ui.allocate_exact_size(vec2(1.0, 20.0), Sense::hover());
|
let (rule, _) = ui.allocate_exact_size(vec2(1.0, 20.0), Sense::hover());
|
||||||
ui.painter().rect_filled(rule, 0.0, theme::fade(theme::INK, 0.15));
|
ui.painter().rect_filled(rule, 0.0, theme::fade(theme::INK, 0.15));
|
||||||
let clearable = !prompt.is_empty() || !refs.is_empty();
|
let clearable = !prompt.is_empty() || !refs.is_empty() || extra_input;
|
||||||
let resp = ui.add_enabled_ui(clearable, |ui| widgets::ghost(ui, Icon::Eraser, None, "Clear all", false)).inner;
|
let resp = ui.add_enabled_ui(clearable, |ui| widgets::ghost(ui, Icon::Eraser, None, "Clear all", false)).inner;
|
||||||
if resp.clicked() {
|
if resp.clicked() {
|
||||||
if refs.is_empty() {
|
if refs.is_empty() {
|
||||||
@@ -312,7 +328,7 @@ fn paste_images(ui: &mut Ui, props: &PromptBoxProps<'_>, actions: &mut Vec<Promp
|
|||||||
|
|
||||||
/// Focus mode: a large editor in a modal, with the deck and toolbar at hand.
|
/// Focus mode: a large editor in a modal, with the deck and toolbar at hand.
|
||||||
#[allow(clippy::too_many_arguments)]
|
#[allow(clippy::too_many_arguments)]
|
||||||
fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, toolbar_left: &mut dyn FnMut(&mut Ui), actions: &mut Vec<PromptBoxAction>) {
|
fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, slots: &mut dyn FnMut(&mut Ui, Slot), actions: &mut Vec<PromptBoxAction>) {
|
||||||
let height = ctx.content_rect().height() * 0.7;
|
let height = ctx.content_rect().height() * 0.7;
|
||||||
let mut done = false;
|
let mut done = false;
|
||||||
let closed = widgets::modal(ctx, props.id.with("focus-mode"), 896.0, |ui| {
|
let closed = widgets::modal(ctx, props.id.with("focus-mode"), 896.0, |ui| {
|
||||||
@@ -335,12 +351,12 @@ fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state
|
|||||||
ui.push_id("focus-deck", |ui| deck_widget(ui, props, refs, cache, true, actions));
|
ui.push_id("focus-deck", |ui| deck_widget(ui, props, refs, cache, true, actions));
|
||||||
ui.add_space(12.0);
|
ui.add_space(12.0);
|
||||||
ui.horizontal(|ui| {
|
ui.horizontal(|ui| {
|
||||||
ui.push_id("focus-toolbar", |ui| ui.horizontal(|ui| toolbar_left(ui)));
|
ui.push_id("focus-toolbar", |ui| ui.horizontal(|ui| slots(ui, Slot::ToolbarLeft)));
|
||||||
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
|
||||||
if widgets::button(ui, None, "Done", widgets::Kind::Primary, 0.0, theme::CONTROL_H).clicked() {
|
if widgets::button(ui, None, "Done", widgets::Kind::Primary, 0.0, theme::CONTROL_H).clicked() {
|
||||||
done = true;
|
done = true;
|
||||||
}
|
}
|
||||||
ui.push_id("focus-clear", |ui| clear_all_button(ui, prompt, refs, state, actions));
|
ui.push_id("focus-clear", |ui| clear_all_button(ui, prompt, refs, state, props.extra_input, actions));
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -90,14 +90,19 @@ pub fn popover(anchor: &Response, popup_id: Id, content: impl FnOnce(&mut Ui)) {
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// A toolbar pill that opens a popover of custom `content` above it.
|
||||||
|
pub fn popover_pill(ui: &mut Ui, id: Id, lead: Leading<'_>, label: &str, tooltip: &str, content: impl FnOnce(&mut Ui)) {
|
||||||
|
let popup_id = ui.make_persistent_id(id).with("popup");
|
||||||
|
let open = Popup::is_id_open(ui.ctx(), popup_id);
|
||||||
|
let resp = ui.push_id(id, |ui| widgets::pill(ui, lead, label, open)).inner.on_hover_text(tooltip);
|
||||||
|
popover(&resp, popup_id, content);
|
||||||
|
}
|
||||||
|
|
||||||
/// The duration popover: a clock pill reading `5s`, opening a 1-second-step slider between the
|
/// The duration popover: a clock pill reading `5s`, opening a 1-second-step slider between the
|
||||||
/// model's minimum and maximum. Returns the new value.
|
/// model's minimum and maximum. Returns the new value.
|
||||||
pub fn duration_picker(ui: &mut Ui, id: Id, current: u32, min: u32, max: u32) -> Option<u32> {
|
pub fn duration_picker(ui: &mut Ui, id: Id, current: u32, min: u32, max: u32) -> Option<u32> {
|
||||||
let popup_id = ui.make_persistent_id(id).with("popup");
|
|
||||||
let open = Popup::is_id_open(ui.ctx(), popup_id);
|
|
||||||
let resp = ui.push_id(id, |ui| widgets::pill(ui, Leading::Icon(Icon::Clock), &format!("{current}s"), open)).inner.on_hover_text("Duration");
|
|
||||||
let mut value = current;
|
let mut value = current;
|
||||||
popover(&resp, popup_id, |ui| {
|
popover_pill(ui, id, Leading::Icon(Icon::Clock), &format!("{current}s"), "Duration", |ui| {
|
||||||
ui.set_width(240.0);
|
ui.set_width(240.0);
|
||||||
widgets::menu_header(ui, "Duration");
|
widgets::menu_header(ui, "Duration");
|
||||||
ui.add_space(4.0);
|
ui.add_space(4.0);
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ use crate::pages::toolbar;
|
|||||||
use crate::prompt_box::deck::DeckLimits;
|
use crate::prompt_box::deck::DeckLimits;
|
||||||
use crate::prompt_box::editor::{MentionKind, mention_items};
|
use crate::prompt_box::editor::{MentionKind, mention_items};
|
||||||
use crate::prompt_box::refs::{RefKind, References};
|
use crate::prompt_box::refs::{RefKind, References};
|
||||||
use crate::prompt_box::{DeckMode, PromptBoxProps, PromptBoxState, show};
|
use crate::prompt_box::{DeckMode, PromptBoxProps, PromptBoxState, Slot, show};
|
||||||
|
|
||||||
struct State {
|
struct State {
|
||||||
// Keeps the media cache's runtime alive.
|
// Keeps the media cache's runtime alive.
|
||||||
@@ -45,15 +45,16 @@ fn build(state: State) -> Harness<'static, State> {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
let mentions = if s.with_mentions { mention_items(MentionKind::Image, [None, None].into_iter()) } else { Vec::new() };
|
let mentions = if s.with_mentions { mention_items(MentionKind::Image, [None, None].into_iter()) } else { Vec::new() };
|
||||||
let props = PromptBoxProps { id: egui::Id::new("test-box"), placeholder: "Describe what you want in the image...", enter_to_generate: s.enter_to_generate, mentions: &mentions, max_length: Some(Some(1000)), deck: DeckMode::References(DeckLimits { max_images: 4, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: false }), accepts: &[RefKind::Image], credits: Some(15), generate_enabled: !s.prompt.trim().is_empty(), generating: false, generate_tooltip: "Generate", warning: None, banner: None };
|
let props = PromptBoxProps { id: egui::Id::new("test-box"), placeholder: "Describe what you want in the image...", enter_to_generate: s.enter_to_generate, mentions: &mentions, max_length: Some(Some(1000)), deck: DeckMode::References(DeckLimits { max_images: 4, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: false }), accepts: &[RefKind::Image], credits: Some(15), generate_enabled: !s.prompt.trim().is_empty(), generating: false, generate_tooltip: "Generate", warning: None, banner: None, extra_input: false };
|
||||||
let (model, aspect) = (&s.model, &mut s.aspect);
|
let (model, aspect) = (&s.model, &mut s.aspect);
|
||||||
let mut left = |ui: &mut egui::Ui| {
|
let mut slots = |ui: &mut egui::Ui, slot: Slot| {
|
||||||
if let Some(v) = toolbar::aspect_ratio(ui, egui::Id::new("test-aspect"), model, aspect) {
|
if slot == Slot::ToolbarLeft {
|
||||||
*aspect = v;
|
if let Some(v) = toolbar::aspect_ratio(ui, egui::Id::new("test-aspect"), model, aspect) {
|
||||||
|
*aspect = v;
|
||||||
|
}
|
||||||
}
|
}
|
||||||
};
|
};
|
||||||
let mut right = |_: &mut egui::Ui| {};
|
let actions = show(ui, &mut s.prompt, &s.refs, &mut s.box_state, &props, &mut s.cache, &mut slots);
|
||||||
let actions = show(ui, &mut s.prompt, &s.refs, &mut s.box_state, &props, &mut s.cache, &mut left, &mut right);
|
|
||||||
s.actions.extend(actions.iter().map(|a| format!("{a:?}")));
|
s.actions.extend(actions.iter().map(|a| format!("{a:?}")));
|
||||||
},
|
},
|
||||||
state,
|
state,
|
||||||
|
|||||||
@@ -87,6 +87,7 @@ impl Page {
|
|||||||
"home" => Page::Home,
|
"home" => Page::Home,
|
||||||
"create-image" | "image" => Page::CreateImage,
|
"create-image" | "image" => Page::CreateImage,
|
||||||
"create-video" | "video" => Page::CreateVideo,
|
"create-video" | "video" => Page::CreateVideo,
|
||||||
|
"create-audio" | "audio" => Page::CreateAudio,
|
||||||
"library" => Page::Library,
|
"library" => Page::Library,
|
||||||
_ => return None,
|
_ => return None,
|
||||||
})
|
})
|
||||||
@@ -94,7 +95,7 @@ impl Page {
|
|||||||
|
|
||||||
/// Pages the native app has; the rest open a "coming soon" page.
|
/// Pages the native app has; the rest open a "coming soon" page.
|
||||||
pub fn is_ready(self) -> bool {
|
pub fn is_ready(self) -> bool {
|
||||||
matches!(self, Page::Home | Page::CreateImage | Page::CreateVideo | Page::Library)
|
matches!(self, Page::Home | Page::CreateImage | Page::CreateVideo | Page::CreateAudio | Page::Library)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// The same tool in the web app.
|
/// The same tool in the web app.
|
||||||
|
|||||||
@@ -18,6 +18,7 @@ pub enum Icon {
|
|||||||
Coins,
|
Coins,
|
||||||
Copy,
|
Copy,
|
||||||
Download,
|
Download,
|
||||||
|
Drum,
|
||||||
Eraser,
|
Eraser,
|
||||||
Eye,
|
Eye,
|
||||||
ExternalLink,
|
ExternalLink,
|
||||||
@@ -35,15 +36,21 @@ pub enum Icon {
|
|||||||
LogIn,
|
LogIn,
|
||||||
LogOut,
|
LogOut,
|
||||||
Maximize,
|
Maximize,
|
||||||
|
Mic,
|
||||||
|
MicOff,
|
||||||
Music,
|
Music,
|
||||||
PanelLeft,
|
PanelLeft,
|
||||||
|
Pause,
|
||||||
Pencil,
|
Pencil,
|
||||||
Play,
|
Play,
|
||||||
Plus,
|
Plus,
|
||||||
|
Repeat,
|
||||||
RotateCw,
|
RotateCw,
|
||||||
Settings,
|
Settings,
|
||||||
|
SlidersHorizontal,
|
||||||
Sparkles,
|
Sparkles,
|
||||||
SquareCheck,
|
SquareCheck,
|
||||||
|
Tags,
|
||||||
Trash,
|
Trash,
|
||||||
Upload,
|
Upload,
|
||||||
User,
|
User,
|
||||||
@@ -158,6 +165,14 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
g.line(p, &[(7.0, 10.0), (12.0, 15.0), (17.0, 10.0)]);
|
g.line(p, &[(7.0, 10.0), (12.0, 15.0), (17.0, 10.0)]);
|
||||||
g.line(p, &[(4.0, 21.0), (20.0, 21.0)]);
|
g.line(p, &[(4.0, 21.0), (20.0, 21.0)]);
|
||||||
},
|
},
|
||||||
|
Icon::Drum => {
|
||||||
|
g.line(p, &[(2.0, 2.0), (10.0, 8.0)]);
|
||||||
|
g.line(p, &[(22.0, 2.0), (14.0, 8.0)]);
|
||||||
|
g.closed(p, &[(2.0, 9.0), (6.0, 6.5), (12.0, 5.5), (18.0, 6.5), (22.0, 9.0), (18.0, 11.5), (12.0, 12.5), (6.0, 11.5)]);
|
||||||
|
g.line(p, &[(2.0, 9.0), (2.0, 17.0), (6.0, 19.5), (12.0, 20.5), (18.0, 19.5), (22.0, 17.0), (22.0, 9.0)]);
|
||||||
|
g.line(p, &[(7.0, 11.8), (7.0, 19.8)]);
|
||||||
|
g.line(p, &[(17.0, 11.8), (17.0, 19.8)]);
|
||||||
|
},
|
||||||
Icon::Eraser => {
|
Icon::Eraser => {
|
||||||
g.closed(p, &[(7.0, 21.0), (3.5, 17.5), (14.0, 7.0), (21.0, 14.0), (14.0, 21.0)]);
|
g.closed(p, &[(7.0, 21.0), (3.5, 17.5), (14.0, 7.0), (21.0, 14.0), (14.0, 21.0)]);
|
||||||
g.line(p, &[(22.0, 21.0), (7.0, 21.0)]);
|
g.line(p, &[(22.0, 21.0), (7.0, 21.0)]);
|
||||||
@@ -249,6 +264,14 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
g.line(p, &[(21.0, 3.0), (14.0, 10.0)]);
|
g.line(p, &[(21.0, 3.0), (14.0, 10.0)]);
|
||||||
g.line(p, &[(3.0, 21.0), (10.0, 14.0)]);
|
g.line(p, &[(3.0, 21.0), (10.0, 14.0)]);
|
||||||
},
|
},
|
||||||
|
Icon::Mic | Icon::MicOff => {
|
||||||
|
g.rect(p, 9.0, 2.0, 6.0, 13.0, 3.0);
|
||||||
|
g.line(p, &[(19.0, 10.0), (19.0, 12.0), (17.0, 16.0), (12.0, 19.0), (7.0, 16.0), (5.0, 12.0), (5.0, 10.0)]);
|
||||||
|
g.line(p, &[(12.0, 19.0), (12.0, 22.0)]);
|
||||||
|
if icon == Icon::MicOff {
|
||||||
|
g.line(p, &[(2.0, 2.0), (22.0, 22.0)]);
|
||||||
|
}
|
||||||
|
},
|
||||||
Icon::Music => {
|
Icon::Music => {
|
||||||
g.line(p, &[(9.0, 18.0), (9.0, 5.0), (21.0, 3.0), (21.0, 16.0)]);
|
g.line(p, &[(9.0, 18.0), (9.0, 5.0), (21.0, 3.0), (21.0, 16.0)]);
|
||||||
g.circle(p, 6.0, 18.0, 3.0);
|
g.circle(p, 6.0, 18.0, 3.0);
|
||||||
@@ -258,6 +281,10 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
g.rect(p, 3.0, 3.0, 18.0, 18.0, 2.0);
|
g.rect(p, 3.0, 3.0, 18.0, 18.0, 2.0);
|
||||||
g.line(p, &[(9.0, 3.0), (9.0, 21.0)]);
|
g.line(p, &[(9.0, 3.0), (9.0, 21.0)]);
|
||||||
},
|
},
|
||||||
|
Icon::Pause => {
|
||||||
|
p.rect_filled(Rect::from_min_size(g.p(6.0, 4.0), vec2(4.0, 16.0) * g.scale), 1.0, color);
|
||||||
|
p.rect_filled(Rect::from_min_size(g.p(14.0, 4.0), vec2(4.0, 16.0) * g.scale), 1.0, color);
|
||||||
|
},
|
||||||
Icon::Pencil => {
|
Icon::Pencil => {
|
||||||
g.closed(p, &[(17.0, 3.0), (21.0, 7.0), (7.5, 20.5), (2.0, 22.0), (3.5, 16.5)]);
|
g.closed(p, &[(17.0, 3.0), (21.0, 7.0), (7.5, 20.5), (2.0, 22.0), (3.5, 16.5)]);
|
||||||
g.line(p, &[(15.0, 5.0), (19.0, 9.0)]);
|
g.line(p, &[(15.0, 5.0), (19.0, 9.0)]);
|
||||||
@@ -270,6 +297,12 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
g.line(p, &[(5.0, 12.0), (19.0, 12.0)]);
|
g.line(p, &[(5.0, 12.0), (19.0, 12.0)]);
|
||||||
g.line(p, &[(12.0, 5.0), (12.0, 19.0)]);
|
g.line(p, &[(12.0, 5.0), (12.0, 19.0)]);
|
||||||
},
|
},
|
||||||
|
Icon::Repeat => {
|
||||||
|
g.line(p, &[(17.0, 2.0), (21.0, 6.0), (17.0, 10.0)]);
|
||||||
|
g.line(p, &[(3.0, 11.0), (3.0, 10.0), (5.0, 7.0), (7.0, 6.0), (21.0, 6.0)]);
|
||||||
|
g.line(p, &[(7.0, 22.0), (3.0, 18.0), (7.0, 14.0)]);
|
||||||
|
g.line(p, &[(21.0, 13.0), (21.0, 14.0), (19.0, 17.0), (17.0, 18.0), (3.0, 18.0)]);
|
||||||
|
},
|
||||||
Icon::RotateCw => {
|
Icon::RotateCw => {
|
||||||
g.arc(p, 12.0, 12.0, 9.0, -60.0, 250.0);
|
g.arc(p, 12.0, 12.0, 9.0, -60.0, 250.0);
|
||||||
g.line(p, &[(21.0, 3.0), (21.0, 8.5), (15.5, 8.5)]);
|
g.line(p, &[(21.0, 3.0), (21.0, 8.5), (15.5, 8.5)]);
|
||||||
@@ -282,6 +315,17 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
g.line(p, &[(12.0 + 7.5 * a.cos(), 12.0 + 7.5 * a.sin()), (12.0 + 10.0 * a.cos(), 12.0 + 10.0 * a.sin())]);
|
g.line(p, &[(12.0 + 7.5 * a.cos(), 12.0 + 7.5 * a.sin()), (12.0 + 10.0 * a.cos(), 12.0 + 10.0 * a.sin())]);
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
Icon::SlidersHorizontal => {
|
||||||
|
g.line(p, &[(21.0, 4.0), (14.0, 4.0)]);
|
||||||
|
g.line(p, &[(10.0, 4.0), (3.0, 4.0)]);
|
||||||
|
g.line(p, &[(21.0, 12.0), (12.0, 12.0)]);
|
||||||
|
g.line(p, &[(8.0, 12.0), (3.0, 12.0)]);
|
||||||
|
g.line(p, &[(21.0, 20.0), (16.0, 20.0)]);
|
||||||
|
g.line(p, &[(12.0, 20.0), (3.0, 20.0)]);
|
||||||
|
g.line(p, &[(14.0, 2.0), (14.0, 6.0)]);
|
||||||
|
g.line(p, &[(8.0, 10.0), (8.0, 14.0)]);
|
||||||
|
g.line(p, &[(16.0, 18.0), (16.0, 22.0)]);
|
||||||
|
},
|
||||||
Icon::Sparkles => {
|
Icon::Sparkles => {
|
||||||
let star = |cx: f32, cy: f32, r: f32| {
|
let star = |cx: f32, cy: f32, r: f32| {
|
||||||
let k = r * 0.28;
|
let k = r * 0.28;
|
||||||
@@ -290,6 +334,11 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
|
|||||||
star(10.0, 12.0, 8.0);
|
star(10.0, 12.0, 8.0);
|
||||||
star(19.0, 5.0, 3.0);
|
star(19.0, 5.0, 3.0);
|
||||||
},
|
},
|
||||||
|
Icon::Tags => {
|
||||||
|
g.closed(p, &[(3.0, 5.0), (5.0, 3.0), (10.6, 3.0), (18.0, 10.4), (18.0, 12.6), (12.6, 18.0), (10.4, 18.0), (3.0, 10.6)]);
|
||||||
|
g.line(p, &[(15.0, 3.0), (22.0, 10.0), (22.0, 12.0), (17.0, 17.0)]);
|
||||||
|
g.dot(p, 7.5, 7.5, 1.1);
|
||||||
|
},
|
||||||
Icon::Trash => {
|
Icon::Trash => {
|
||||||
g.line(p, &[(3.0, 6.0), (21.0, 6.0)]);
|
g.line(p, &[(3.0, 6.0), (21.0, 6.0)]);
|
||||||
g.line(p, &[(19.0, 6.0), (19.0, 20.0), (17.0, 22.0), (7.0, 22.0), (5.0, 20.0), (5.0, 6.0)]);
|
g.line(p, &[(19.0, 6.0), (19.0, 20.0), (17.0, 22.0), (7.0, 22.0), (5.0, 20.0), (5.0, 6.0)]);
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
pub mod creator_icons;
|
pub mod creator_icons;
|
||||||
pub mod icons;
|
pub mod icons;
|
||||||
pub mod toast;
|
pub mod toast;
|
||||||
|
pub mod waveform;
|
||||||
pub mod widgets;
|
pub mod widgets;
|
||||||
pub mod window;
|
pub mod window;
|
||||||
|
|||||||
@@ -0,0 +1,194 @@
|
|||||||
|
//! The track player (`WaveformAudioPlayer`): a round white play button, the waveform (click to
|
||||||
|
//! seek, played part in blue), the time, and a volume flyout. Placeholder bars stand in while the
|
||||||
|
//! waveform decodes; a plain progress line if it can't.
|
||||||
|
|
||||||
|
use egui::{Color32, FontId, Id, Popup, Rect, Sense, Stroke, Ui, pos2, vec2};
|
||||||
|
|
||||||
|
use crate::backend::audio::{self, AudioPlayer, WaveformLookup};
|
||||||
|
use crate::prompt_box::pickers;
|
||||||
|
use crate::theme;
|
||||||
|
use crate::ui::icons::{self, Icon};
|
||||||
|
|
||||||
|
/// Bars drawn while the waveform loads (`PLACEHOLDER_BAR_COUNT`).
|
||||||
|
const PLACEHOLDER_BARS: usize = 48;
|
||||||
|
/// Waveform bars: width and the step between them.
|
||||||
|
const BAR_WIDTH: f32 = 2.0;
|
||||||
|
const BAR_STEP: f32 = 3.5;
|
||||||
|
const WAVE_INK: Color32 = Color32::from_rgba_premultiplied(89, 89, 89, 89);
|
||||||
|
const PROGRESS_INK: Color32 = Color32::from_rgb(0x2d, 0x81, 0xff);
|
||||||
|
/// How often a playing track redraws its progress.
|
||||||
|
const FRAME: std::time::Duration = std::time::Duration::from_millis(33);
|
||||||
|
|
||||||
|
/// Draws the player across `ui`'s width; `duration_hint` (seconds) is used until the waveform
|
||||||
|
/// knows the real length. `compact` is the list rows' smaller size. Returns the player's rect.
|
||||||
|
pub fn player(ui: &mut Ui, id: Id, audio: &mut AudioPlayer, url: &str, duration_hint: Option<f32>, compact: bool) -> Rect {
|
||||||
|
let (button_d, wave_h): (f32, f32) = if compact { (32.0, 32.0) } else { (40.0, 44.0) };
|
||||||
|
let (rect, _) = ui.allocate_exact_size(vec2(ui.available_width(), wave_h.max(button_d)), Sense::hover());
|
||||||
|
// Clicks between the controls stay with the player (they don't open the card underneath).
|
||||||
|
ui.interact(rect, id.with("bg"), Sense::click());
|
||||||
|
let ctx = ui.ctx().clone();
|
||||||
|
let waveform = audio.waveform(&ctx, url);
|
||||||
|
let playback = audio.track(url);
|
||||||
|
let duration = match &waveform {
|
||||||
|
WaveformLookup::Ready(w) => Some(w.duration),
|
||||||
|
_ => duration_hint,
|
||||||
|
}
|
||||||
|
.filter(|d| *d > 0.0);
|
||||||
|
let position = playback.map_or(0.0, |p| p.position);
|
||||||
|
let playing = playback.is_some_and(|p| !p.paused && !p.loading);
|
||||||
|
if playing {
|
||||||
|
ctx.request_repaint_after(FRAME);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Play / pause.
|
||||||
|
let button = Rect::from_center_size(pos2(rect.left() + button_d / 2.0, rect.center().y), vec2(button_d, button_d));
|
||||||
|
let resp = ui.interact(button, id.with("play"), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
|
||||||
|
let label = if playing { "Pause" } else { "Play" };
|
||||||
|
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
|
||||||
|
ui.painter().circle_filled(button.center(), button_d / 2.0, if resp.hovered() { theme::fade(Color32::WHITE, 0.88) } else { Color32::WHITE });
|
||||||
|
let glyph = Rect::from_center_size(button.center(), vec2(button_d * 0.4, button_d * 0.4));
|
||||||
|
match playback {
|
||||||
|
Some(p) if p.loading => icons::paint_spinner(ui, glyph, Color32::BLACK),
|
||||||
|
_ if playing => icons::paint(ui.painter(), glyph, Icon::Pause, Color32::BLACK),
|
||||||
|
_ => icons::paint(ui.painter(), glyph.translate(vec2(button_d * 0.04, 0.0)), Icon::Play, Color32::BLACK),
|
||||||
|
}
|
||||||
|
if resp.clicked() {
|
||||||
|
audio.toggle_track(&ctx, url);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Volume, then the time, from the right.
|
||||||
|
let volume_rect = Rect::from_center_size(pos2(rect.right() - 14.0, rect.center().y), vec2(28.0, 28.0));
|
||||||
|
volume(ui, id, volume_rect, audio);
|
||||||
|
let time = format!("{} / {}", format_time(position), duration.map_or_else(|| "-:--".to_owned(), format_time));
|
||||||
|
let galley = ui.painter().layout_no_wrap(time, FontId::new(11.0, theme::mono()), theme::MUTED);
|
||||||
|
let time_x = volume_rect.left() - 8.0 - galley.size().x;
|
||||||
|
ui.painter().galley(pos2(time_x, rect.center().y - galley.size().y / 2.0), galley, theme::MUTED);
|
||||||
|
|
||||||
|
// The waveform between them.
|
||||||
|
let wave = Rect::from_min_max(pos2(button.right() + 10.0, rect.center().y - wave_h / 2.0), pos2(time_x - 10.0, rect.center().y + wave_h / 2.0));
|
||||||
|
if wave.width() > 8.0 {
|
||||||
|
let progress = duration.map_or(0.0, |d| (position / d).clamp(0.0, 1.0));
|
||||||
|
let wave_resp = ui.interact(wave, id.with("wave"), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
|
||||||
|
match &waveform {
|
||||||
|
WaveformLookup::Ready(w) => paint_bars(ui, wave, &audio::resample(&w.peaks, bar_count(wave.width())), progress),
|
||||||
|
WaveformLookup::Loading => paint_bars(ui, wave, &placeholder_bars(url), progress),
|
||||||
|
WaveformLookup::Failed => paint_line(ui, wave, progress),
|
||||||
|
}
|
||||||
|
if let Some(pointer) = wave_resp.interact_pointer_pos().filter(|_| wave_resp.clicked()) {
|
||||||
|
let frac = ((pointer.x - wave.left()) / wave.width()).clamp(0.0, 1.0);
|
||||||
|
audio.seek_track(&ctx, url, duration.map_or(0.0, |d| frac * d));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
rect
|
||||||
|
}
|
||||||
|
|
||||||
|
/// "m:ss".
|
||||||
|
pub fn format_time(secs: f32) -> String {
|
||||||
|
let s = secs.max(0.0) as u64;
|
||||||
|
format!("{}:{:02}", s / 60, s % 60)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The volume button and its 0–100 slider flyout.
|
||||||
|
fn volume(ui: &mut Ui, id: Id, rect: Rect, audio: &mut AudioPlayer) {
|
||||||
|
let popup_id = id.with("volume-popup");
|
||||||
|
let open = Popup::is_id_open(ui.ctx(), popup_id);
|
||||||
|
let resp = ui.interact(rect, id.with("volume"), Sense::click()).on_hover_text("Volume").on_hover_cursor(egui::CursorIcon::PointingHand);
|
||||||
|
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, "Volume"));
|
||||||
|
if resp.hovered() || open {
|
||||||
|
ui.painter().rect_filled(rect, theme::RADIUS, theme::WASH);
|
||||||
|
}
|
||||||
|
let level = audio.track_volume();
|
||||||
|
icons::paint(ui.painter(), rect.shrink(7.0), if level <= 0.0 { Icon::VolumeOff } else { Icon::Volume }, if resp.hovered() || open { theme::INK } else { theme::DIM });
|
||||||
|
let mut percent = (level * 100.0).round() as u32;
|
||||||
|
pickers::popover(&resp, popup_id, |ui| {
|
||||||
|
ui.horizontal(|ui| {
|
||||||
|
ui.add_space(6.0);
|
||||||
|
ui.style_mut().spacing.slider_width = 120.0;
|
||||||
|
ui.add(egui::Slider::new(&mut percent, 0..=100).show_value(false));
|
||||||
|
ui.label(egui::RichText::new(percent.to_string()).size(11.0).family(theme::mono()).color(theme::MUTED));
|
||||||
|
ui.add_space(4.0);
|
||||||
|
});
|
||||||
|
});
|
||||||
|
if percent != (level * 100.0).round() as u32 {
|
||||||
|
audio.set_track_volume(percent as f32 / 100.0);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn bar_count(width: f32) -> usize {
|
||||||
|
((width + BAR_STEP - BAR_WIDTH) / BAR_STEP).floor().max(1.0) as usize
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Bars centred on the waveform's midline, blue up to `progress` (0–1).
|
||||||
|
fn paint_bars(ui: &Ui, rect: Rect, heights: &[f32], progress: f32) {
|
||||||
|
if heights.is_empty() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
let step = rect.width() / heights.len() as f32;
|
||||||
|
let width = BAR_WIDTH.min(step * 0.7);
|
||||||
|
let split = rect.left() + rect.width() * progress;
|
||||||
|
let p = ui.painter();
|
||||||
|
for (i, h) in heights.iter().enumerate() {
|
||||||
|
let x = rect.left() + step * (i as f32 + 0.5);
|
||||||
|
let half = (h.clamp(0.0, 1.0) * rect.height() / 2.0).max(1.0);
|
||||||
|
let bar = Rect::from_center_size(pos2(x, rect.center().y), vec2(width, half * 2.0));
|
||||||
|
p.rect_filled(bar, 1.0, if x <= split { PROGRESS_INK } else { WAVE_INK });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The fallback when the waveform can't be decoded: a thin track with the played part in blue.
|
||||||
|
fn paint_line(ui: &Ui, rect: Rect, progress: f32) {
|
||||||
|
let track = Rect::from_center_size(rect.center(), vec2(rect.width(), 4.0));
|
||||||
|
ui.painter().rect_filled(track, 2.0, WAVE_INK);
|
||||||
|
ui.painter().rect_filled(Rect::from_min_size(track.min, vec2(track.width() * progress, track.height())), 2.0, PROGRESS_INK);
|
||||||
|
ui.painter().line_segment([pos2(track.left() + track.width() * progress, rect.top() + 6.0), pos2(track.left() + track.width() * progress, rect.bottom() - 6.0)], Stroke::new(1.0, PROGRESS_INK));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Deterministic stand-in bars for a track (seeded by its URL), 25–89 % tall.
|
||||||
|
pub fn placeholder_bars(url: &str) -> Vec<f32> {
|
||||||
|
let mut seed = url.chars().fold(0_u32, |s, c| s.wrapping_mul(31).wrapping_add(c as u32));
|
||||||
|
(0..PLACEHOLDER_BARS)
|
||||||
|
.map(|_| {
|
||||||
|
let percent = 25 + seed % 65;
|
||||||
|
seed = seed.wrapping_mul(1_103_515_245).wrapping_add(12_345);
|
||||||
|
percent as f32 / 100.0
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// A tile with a music note (`MusicIcon` in its box), for audio without artwork.
|
||||||
|
pub fn music_tile(ui: &Ui, rect: Rect, round: bool) {
|
||||||
|
let p = ui.painter();
|
||||||
|
if round {
|
||||||
|
p.circle_filled(rect.center(), rect.width() / 2.0, theme::WASH_HOVER);
|
||||||
|
} else {
|
||||||
|
p.rect(rect, theme::RADIUS, theme::WASH, theme::hairline(), egui::StrokeKind::Inside);
|
||||||
|
}
|
||||||
|
icons::paint(p, Rect::from_center_size(rect.center(), rect.size() * 0.42), Icon::Music, theme::fade(Color32::WHITE, 0.75));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn times_read_minutes_and_seconds() {
|
||||||
|
assert_eq!(format_time(0.0), "0:00");
|
||||||
|
assert_eq!(format_time(65.9), "1:05");
|
||||||
|
assert_eq!(format_time(600.0), "10:00");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn placeholder_bars_are_stable_per_track() {
|
||||||
|
let a = placeholder_bars("https://cdn/a.mp3");
|
||||||
|
assert_eq!(a.len(), PLACEHOLDER_BARS);
|
||||||
|
assert_eq!(a, placeholder_bars("https://cdn/a.mp3"));
|
||||||
|
assert_ne!(a, placeholder_bars("https://cdn/b.mp3"));
|
||||||
|
assert!(a.iter().all(|h| (0.25..0.9).contains(h)));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn bars_fill_the_width() {
|
||||||
|
assert_eq!(bar_count(2.0), 1);
|
||||||
|
assert_eq!(bar_count(352.0), 101);
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user