Compare commits

...
Author SHA1 Message Date
Dylan M. Taylor 07471d404d ci: verify the actions lockfile in CI
Fails the build when .github/workflows/actions.lock stops matching the
workflows, or when a locked pin no longer verifies upstream, so a stale or
tampered lockfile cannot go unnoticed between dependency updates.
2026-11-12 07:22:42 -05:00
Dylan M. Taylor 78401059b6 ci: lock workflow actions through an actions.lock lockfile
Mutable action tags are a supply-chain risk: a moved tag runs different code
without a diff. Every workflow action is now pinned through
.github/workflows/actions.lock, and CI verifies the lockfile so an out-of-sync
or tampered one fails instead of running.

arch-package was pinned to 9d67905, a commit no longer on any branch or tag of
craft-common-actions, so it is back on main like the other craft actions here.

S7637 wants an inline SHA per action and does not read actions.lock, so the
rule is scoped off for workflow files until Sonar reads lockfiles.
2026-11-12 07:22:16 -05:00
Dylan M. Taylor 84a6a7dcf5 Report 0.42.1 in the macOS build config (#2836)
The 0.42.1 bump in #2835 updated `tauri.conf.json` and `version.rs` but
left `tauri-mac.conf.json` at 0.42.0, so the macOS release build would
report the wrong version.

- Sets `tauri-mac.conf.json` to 0.42.1.
- Extends
`package_identity_and_display_version_match_the_release_configuration`
to also check the macOS config. It fails before this change and passes
after; all 19 updater tests pass.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
  * Updated the macOS app version to 0.42.1.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 21:57:00 -05:00
Dylan M. Taylor 4ecd4b7eea Add manual verified updates and bump ArtCraft to 0.42.1 (#2835)
ArtCraft’s Help menu has news but no application update check, and its
Flatpak cannot invoke the host updater. Add **Help → Check for
Updates…** with background checks, current/available versions, explicit
Update and restart, progress and failure states, and Releases fallback.
Browser users are offered a reload and the desktop releases.

Use the same verified `craft-update` core as the native Craft apps,
pinned to the immutable reviewed commit in
https://github.com/storytold/craft-libs/pull/970. Keep ArtCraft’s
existing `ai.artcraft.app` identity and release naming. Honor launcher
pins, reject active generation tasks before download and again before
replacement, preserve the native window during installation, defer the
helper until accepted exit, and report helper outcomes on startup.
Registered NSIS copies use Releases; they are not overwritten as
portable folders.

Grant the host update interface and read-only launcher choices in the
existing Flatpak manifest. No frontend network-policy expansion. Add a
separate version commit changing the canonical Tauri configuration and
display version to **0.42.1**, with a regression keeping them
synchronized. Once this and the shared-library prerequisite are green
and merged, merge main into the release branch, finish all platform
drafts, verify signed checksums/assets, and publish.

Validation: native all-target cargo check inside GNOME SDK 51; all six
focused Rust updater tests; all ten Help-menu/UI tests; TypeScript
project build; production frontend build; Flatpak permission regression
red before and green after. All six native SDK tests also pass at the
final 0.42.1 version. The existing lockfile needs compatible chrono and
crc32fast updates for the shared updater; other dependency revisions are
preserved.
2026-11-11 21:36:49 -05:00
Dylan M. Taylor a787c371f6 ci: use StepSecurity's maintained actions (#2832)
Each third-party action that StepSecurity publishes a secure drop-in
replacement for now uses that replacement, pinned to a commit with its
version as a comment. Versions follow what we used: the newest
StepSecurity release in the same major, or the closest major they
publish. dtolnay/rust-toolchain refs that named a toolchain (stable,
nightly, 1.76) become the replacement's `toolchain` input.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated the Linux, macOS, and Windows release workflows to use pinned
build and publishing actions. Stable Rust toolchain selection and
existing target settings are preserved.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 14:57:24 -05:00
Dylan M. Taylor e546fff742 TabState: setActiveTab returns nothing instead of always true (Sonar S3516) (#2831)
`setActiveTab` returned `true` on every path, the last open Sonar issue
in this project (blocker, S3516). No caller reads the result, so it now
returns `Promise<void>`.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Refactor**
* Tab selection works as before: selecting the active tab keeps it
selected, and selecting another tab switches to it. This update changes
how the selection operation reports its result internally, without
changing what users see or how tabs behave. No other user-facing changes
are included.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-11 14:43:57 -05:00
Dylan M. Taylor 1d79cdf52d ci: run Harden-Runner on every job (#2829)
Adds step-security/harden-runner (audit, pinned to v2.22.4 like
craft-common-actions) as the first step of each job that did not have
it,
so every workflow in the org's public repos reports its egress to
StepSecurity. Audit, not block: block-mode allow lists come from each
job's observed traffic.
2026-11-11 12:08:54 -05:00
Dylan M. Taylor 55e6e5cabb frontend: Sonar follow-ups (S8786, S6852, S7059) (#2824)
Two issues SonarCloud raised on main after #2821/#2822 merged:

- **typescript:S8786** `SceneState.ts` `base64ToFile`: #2821's
`/:([^;]*);/` is still flagged. The mime type is now read with
`indexOf`: the text between the first `:` and the `;` after it, else
`image/png`. Checked equal to the regex on `data:image/png;base64`,
`data:;base64`, a header with no `:` or `;`, and several `:`.
- **typescript:S6852** `MentionTextarea`: the `role="textbox"` editor
gets `tabIndex={disabled ? -1 : 0}`. A `contentEditable` element was
focusable already, so nothing changes for users.

Typechecked locally against main (no new errors).

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved image import handling so MIME types are identified more
reliably, with PNG used as a fallback when the type cannot be
determined.
* Disabled text editors are no longer reachable through sequential
keyboard navigation; enabled editors remain tabbable.
* Physics support is initialized during editor setup after the required
canvas and container elements are available.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- **typescript:S7059** `editor.ts`: `ensureAmmoLoaded()` (an async load)
moves out of the `Editor` constructor to `EngineProvider`, the only
place an `Editor` is built, right before `new Editor(...)`. The loader
is a singleton, so the script is still appended once.
2026-11-10 22:51:32 -05:00
Dylan M. Taylor 7c8419fabd frontend: Number.parseFloat/parseInt/isFinite/isNaN (Sonar S7773) (#2819)
Clears the 82 open typescript:S7773 issues (35 files), only on the lines
Sonar flagged.

- `parseFloat` / `parseInt` → `Number.parseFloat` / `Number.parseInt`:
the same functions (`Number.parseFloat === parseFloat`).
- `isFinite` / `isNaN` → `Number.isFinite` / `Number.isNaN`: they differ
only for arguments that aren't numbers. TypeScript's lib types the
globals as taking `number`, and each call already passes one (`maxLen`,
`Number(value)`, a parsed count), so nothing changes. Typecheck is left
to CI.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Improvements**
* Video frame extraction and watermark-removal pages now provide
consistent playback controls, including seeking, playback, mute, and
volume adjustment.
* Prompt-length indicators and validation consistently reflect
configured limits, including unlimited limits.
* **New Features**
* Prompt fields display a character counter that highlights text
exceeding the configured limit.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Also: rebased on main, and the length counter and over-limit check that
the five prompt boxes repeated are now `PromptLengthCounter` /
`isOverPromptLimit` (`promptbox/src/lib/PromptLengthCounter.tsx`). That
removes the duplication the new-code gate counted against this PR.
2026-11-10 22:40:36 -05:00
FredAppsandDylan M. Taylor 130bfd71d6 libs/components: mark fire-and-forget promises with void (Sonar S9383) (#2807)
## Summary

Resolves all 66 SonarCloud **typescript:S9383** findings ("Promises must
be awaited, end with a call to .catch, end with a call to .then with a
rejection handler or be explicitly marked as ignored with the `void`
operator") in `frontend/libs/components`, across 41 files:

| Library                | Findings |
|------------------------|----------|
| `upload-modal`         | 14       |
| `promptbox`            | 11       |
| `settings-modal`       | 11       |
| `pagescene`            | 10       |
| `video-editor`         | 7        |
| `lightbox-modal`       | 3        |
| `pagedraw`             | 3        |
| `generation-list`      | 2        |
| `help-menu`            | 2        |
| `gallery-modal`        | 1        |
| `pricing-modal`        | 1        |
| `provider-setup-modal` | 1        |

The flagged expressions include settings and session loaders
(`fetchData()`, `fetchSession()`) called from effects, upload starts and
retries in the upload modals, enqueue handlers in the prompt boxes,
`audio.play()` and `el.play()` previews, scene render and load calls in
`pagescene`, bookmark actions in `video-editor`, and async IIFEs.

This is part 3 of 3 for artcraft's 157 S9383 findings. Part 1 (Tauri
libraries) is #2805, and part 2 (`apps/artcraft/app`) is #2806.

Some of these files are also changed by open PRs for unrelated reasons,
for example #2615 (settings panes), #2519, #2520 and #2083. Each change
here inserts one word at the start of a statement, so a rebase either
way is trivial.

## Approach

This ports the approach of storytold/artcraft-services#179, which fixes
the same rule in artcraft-services. **#179 is still open and unmerged.**
This PR follows its approach and does not depend on it.

- Each flagged expression is a fire-and-forget promise: a listener
`setup()`, an `unlisten.then(f => f())` cleanup, a loader called from an
effect, an upload kicked off from a handler, or an async IIFE. Prefixing
it with `void` marks the ignored promise explicitly, which is the fix
the rule documents.
- It deliberately does not add `.catch()` everywhere. These call sites
already handle failures inside the async function or surface them
through their own state, and empty handlers would hide errors.
- Async IIFEs become `void (async () => …)()`. `if (cond) call()`
becomes `if (cond) void call()`.
- There is no behaviour change. `void` only discards the promise value,
which was already being discarded.

Each insertion is placed at the exact `textRange` start offset of the
SonarCloud finding, with no other edits.

## Testing

- Each changed file parses with `esbuild <file>`, using the same check
as #179.
- I parsed each changed file with the TypeScript 5.9 compiler API
(`createSourceFile`). That found no parse errors, and every inserted
`void` is either a whole expression statement or an arrow-function body
applied to a call, never part of a larger expression.
- `frontend/.eslintrc.json` has no `no-void` rule.
- I did not run the full Nx type-check, lint or test targets, because
`frontend/node_modules` is not installed in my environment. Please rely
on CI.
- No tests are added. Discarding an already-discarded promise value is
not observable, so no test can fail before this change and pass after
it.

**Added by @dylanmtaylor, to clear the new-code duplication gate.** The
`void` lines sat inside copy-pasted code:
- `ProviderAccountBlock` replaces the identical `GrokAccountBlock` and
`MidjourneyAccountBlock`.
- `UploadFilesPanels` (file list, resize handle, previous/next; each
modal passes its preview) replaces the layout the 3D, Image and Splat
upload modals repeated.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **New Features**
* Multi-file uploads across image, 3D, and splat workflows now share a
two-panel layout with file statuses, preview selection, removal, retry,
and previous/next navigation.
* Grok and Midjourney account controls now use a shared interface that
displays login status and provides available connect or disconnect
actions. Failed credential checks clear the displayed session, and
account controls show a loading state while checking.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Dylan M. Taylor <dylan@dylanmtaylor.com>
2026-11-10 22:39:17 -05:00
Dylan M. Taylor d6772fdd94 frontend: uuidv4 for local ids instead of Math.random (Sonar S2245) (#2820)
Clears the 38 open typescript:S2245 issues (18 files).

Each `Math.random().toString(36)…` made a local id: an upload row, a
reference title, a batch or subscriber id, an idempotency token, an
image cache-buster. They now use `uuidv4()` from `uuid`, which is
already a dependency and imported the same way elsewhere (`import { v4
as uuidv4 } from "uuid"`). It reads `crypto.getRandomValues`, so it also
works where `crypto.randomUUID` doesn't (outside a secure context). The
`crypto.randomUUID ? … : Math.random()…` fallbacks therefore collapse to
`uuidv4()`, and `subscriberId ? subscriberId : …` becomes `subscriberId
|| uuidv4()`.

Ids are now 36 characters instead of 5–13. Nothing parses or compares
their length. Typecheck is left to CI.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Improvements**
* Creative tools now handle identifiers more consistently for generated
content, uploads, reference media, and generation batches.
* Completed generation results are displayed more reliably when there
isn’t a matching pending result.
* These improvements span image, video, audio, 3D, drawing, and
visual-effects workflows, helping keep related media and results
organized across the app.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Also: `ImageTo3DStore` and `ImageTo3DWorldStore` shared the same 30-line
`completeGeneration` (find the subscriber's result, else the first
pending one, and mark it completed, or add a completed one). It's now
one `completeGenerationResult` in `pages/Stores/`, which removes the
duplication the new-code gate counted against this PR.
2026-11-10 21:49:49 -05:00
Dylan M. Taylor 0c11395467 Sonar: clear the shell, workflow, Rust and HTML findings (#2818)
Clears the open SonarCloud issues outside the TypeScript frontend,
except the Rust ones, which moved to #2823 so each PR stays small.
Compile and tests are left to CI.

**Security**
- `frontend/api_docs.html` (Web:S5725): `integrity` (sha384 of the
pinned `swagger-ui-dist@5.32.0` files) and `crossorigin="anonymous"` on
the three unpkg assets, over `https:`.
- S6505: `--ignore-scripts` on `npm install` / `npm ci` in the dev and
build scripts and in the macOS and Windows publish workflows.
`artcraft-linux-publish.yml` already builds with `npm ci
--ignore-scripts`, and the frontend has no install scripts of its own.
- S7637: `dtolnay/rust-toolchain@stable` and
`tauri-apps/tauri-action@v0` pinned by commit SHA. For tauri-action
that's the commit `v0` points to (`84b9d35…`). The Linux workflow had
pinned the tag object instead, and is fixed the same way.
- S6505 / S8543: the macOS build runs `./node_modules/.bin/nx` instead
of `npx nx`, as the Linux workflow does.

**Reliability**
- shell:S7688: `[[ ]]` in place of `[` / `test` in
`frontend_preflight.sh` and the Linux packaging scripts.

**Analysis scope** (`sonar-project.properties`), each with its reason in
the file:
- Not analysed: `crates/vendor/**` (third-party), the upstream Python
reference implementations beside the Grok and Sora clients (nothing runs
them), and the hand-run developer tools `frontend/scripts/unix-dev.mjs`
and `frontend/tools/performance/**`. These held the S8564, S1244, S2245,
S4036, S5443 and S8707 findings.
- rust:S4347 not raised on `DeterministicRng`, the fixed-seed generator
for tests.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Improvements**
* File downloads and database maintenance now use asynchronous
operations, helping the app remain responsive during these tasks.
  * API documentation assets load over HTTPS with integrity checks.
* **Build and Reliability**
* Build and publishing workflows use pinned tool versions and skip
package lifecycle scripts during dependency installation.
  * Linux packaging checks use consistent shell conditionals.
* Frontend builds use the locally installed Nx command where applicable.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
- typescript:S2187 not raised on the ApiManager tests, which build their
cases with shared helpers that call `it` (`BetaKeysApi.test.ts`).
typescript:S2819 not raised on `*.worker.ts`: a dedicated worker's
messages come only from the page that started it.
2026-11-10 21:39:12 -05:00
Dylan M. Taylor 773e5925b9 Rust: tokio::fs in async functions, TaskId Display, refresh token out of source (Sonar) (#2823)
The Rust part of #2818, split out so each PR stays small. Clears the
open rust:S7493, rust:S7458 and secrets:S6418 issues. Compile and tests
are left to CI.

- rust:S7493: `tokio::fs` in place of blocking `std::fs` inside async
functions (`auto_download`, `simple_http_download`, the
fal/Sora/WorldLabs downloads, the `sqlite_tasks` reset), as in
#2803/#2804. The downloaded bytes are handed to `tokio::fs::write` owned
(no copy), per CodeRabbit on #2818. `sqlite_tasks` now depends on
workspace `tokio` with `fs` (one line in `Cargo.lock`).
- rust:S7458: `TaskId`'s inherent `to_string` is removed. `Display`
gives the same string.
- secrets:S6418: the `#[ignore]` WorldLabs refresh-token test reads
`WORLDLABS_REFRESH_TOKEN` instead of holding a token in source. ⚠️ The
token is still in git history, so its owner should revoke it.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Refactor**
* Downloaded files and task checkpoints are now handled using
asynchronous file operations across several generation workflows.
* Database-file cleanup during connection and migration is also
asynchronous; removal errors continue to be logged, and migration
proceeds.
* These changes preserve the existing download, checkpoint, and
error-handling behavior while reducing blocking file operations.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 21:38:23 -05:00
Dylan M. Taylor 8d6fc363b1 frontend: roles, keyboard access and label association (Sonar S6848, S6853) (#2822)
Clears the 59 open typescript:S6848 issues ("non-native interactive
element without a role").

- **Clickable things that act like buttons (16)**: upload and preview
tiles, storyboard frames, help items, reference thumbnails, remove
buttons, deck cards, asset items, popover menu items. They get
`role="button"`, `tabIndex={0}` and `onKeyDown={clickOnEnterOrSpace}`,
the pattern `GalleryRow` already uses. `clickOnEnterOrSpace` (new,
`@storyteller/common`) calls the element's own `click()`, so each
`onClick` runs unchanged. Menu items also get `aria-disabled` when
disabled (their `onClick` already ignores the click). The bulk-select
box is `role="checkbox"` with `aria-checked`. In `ImagePromptRow` the
attributes come before dnd-kit's `attributes`/`listeners` spread, so
when reordering is on, dnd-kit's own role and keyboard sensor still win.
- **Not controls (42)**: modal and menu backdrops, wrappers that only
stop event propagation, mouse-only drag/resize handles and slider
tracks, trigger wrappers around a child button, the tooltip body. They
get `role="presentation"`, which says what they already are to assistive
tech. Nothing changes for mouse users.
- **The prompt editor** (`contentEditable` in `MentionTextarea`) is
`role="textbox"` with `aria-multiline="true"`.

Mouse behaviour is unchanged. Keyboard users can now Tab to and activate
the 16 button-like elements. Typecheck is left to CI.

**typescript:S6853 (13)**: labels tied to their controls. In the
storyboard metadata editor, the signup form and the cost modal, each
label gets `htmlFor` and its control a matching `id` (`Input` and
`Select` pass `id` through, and `Select`'s goes on its `ListboxButton`).
These render once per page. The two colour-picker labels, which already
wrap their `<input type="color">`, get `sr-only` text in place of their
`title`-only name.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Accessibility**
* Added keyboard access to image tiles, asset cards, list items, and
other controls, including Enter and Space activation.
* Improved screen-reader support with clearer labels for controls and
thumbnails, and associations between form labels and fields.
* Updated accessibility semantics for decorative content and interactive
elements.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 20:22:41 -05:00
Dylan M. Taylor f9f4c99e4f frontend: Sonar reliability fixes (promises in conditions, async executors, identical branches, reduce, regex) (#2821)
Clears 51 open TypeScript issues. Typecheck and build are left to CI.

**typescript:S6544 (17)**
- A cached promise checked by truthiness (`if (existing)`, `if
(cached)`, `if (ammoLoadPromise)`…) is checked as `!== undefined` / `!==
null` against its map or field type. Same branches, since a Promise is
always truthy.
- `scene.ts` `load_glb_wrapped_no_cors` / `load_splat_wrapped_no_cors`:
the fetch moves out of the `async` Promise executor. The methods were
already `async`, so a failed fetch now `throw`s (rejects the same way),
and only the callback APIs (`GLTFLoader.parse`, `SplatMesh` `onLoad`)
stay inside `new Promise`.
- `SceneManagerAPI` declares the promises its implementation returns
(`create`/`retrieve` → `Promise<THREE.Object3D | undefined>`,
`update`/`delete` → `Promise<void>`). Nothing calls through the
interface.
- `convertFbxToGlb.worker.ts`: `onmessage` calls `void convert(event)`.

**typescript:S3923 / S1764 / S3516 / S905 (12)**: identical branches
collapsed (`variant === "world" ? "max-w-[1600px]" : "max-w-[1600px]"`,
`ScissorIcon : ScissorIcon`, `index % 2 === 0 ? point + offset : point +
offset`, PromptBox3D's two `setIsEnqueueing(false); return;` arms,
`{DEFAULT_GRID_CONFIG ? null : null}` and its now-unused import), `x &&
x`, the `this.gridHelper;` statement, and `CurrentUserBadge`, which
always rendered `null` (its markup was commented out).

**typescript:S6959 (5)**: each `reduce` starts from the first element.
Every one is behind an emptiness check, and comparing the first element
with itself keeps it.

**typescript:S2871 (4)**: `sort((a, b) => a.localeCompare(b))` for the
order-independent cache keys and the board and font name lists.

**typescript:S8786 (8)**
- Markdown link text is `[^\][]+`. `@image(url, size)` drops the `\s*`
before the size (it's `.trim()`ed already). ATX headings `trimStart()`
the text instead of `\s*`. Slugs trim one `-` (runs were already
collapsed to one).
- `color.ts` `stripCssNoise`: `!important` is replaced by a space, then
trailing `;` are trimmed in a loop.
- Data-URL mime: `/:([^;]*);/`.
- `ControlPanelSceneObject` number check: `/^-?\d*(\.\d*)?$/`. The old
`.` was unescaped and matched any character, so `12a3` passed as a
number.

**Others**: `window.open(…, "noopener,noreferrer")` (S5148).
`AlertTitle` renders `children` explicitly (S6850). The folder name hash
wraps with `Math.imul(…, 1)` instead of `| 0`, the same int32 result
(S7767). `autoFocus` is removed from the two rename inputs, which their
refs already focus and select (S9379).

**Not here**: S7059 in `editor.ts` (`ensureAmmoLoaded()` in the
constructor) is on a line #2807 changes.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Bug Fixes**
* Improved reliability when loading 3D scene assets and completing
drag-and-drop operations.
* Corrected numeric input validation, scene import handling, and video
duration tie handling.
* Improved media, audio, and video cache handling to avoid unnecessary
retries or missed waits.
* Refined markdown parsing for links, headings, image directives, and
heading anchors.
  * Discord invites now open with additional browser protections.
  * Improved CSS color cleanup and handling of pasted scene elements.
* **Improvements**
* Made ordering of tags, fonts, and prompt-related items more
consistent.
  * Improved video editor timeline selections.
* Removed the non-functional account badge from the video editor header.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

**typescript:S1848 (2)**: `new SplatMesh({ …, onLoad })` created an
instance only for its callback. `scene.ts` now awaits `mesh.initialized`
(Spark's promise, which resolves after load), so a failed load rejects
instead of leaving the promise pending. `viewer-3d.tsx` runs the same
body in `initialized.then`, with a `.catch` that logs.

Kept as is: `TabState.setActiveTab` still resolves `true` (S3516 stays
open), because `TabState.spec.ts` and the tab-change benchmark read its
result.
2026-11-10 20:22:36 -05:00
FredApps 71adfd4701 artcraft app: mark fire-and-forget promises with void (Sonar S9383) (#2806)
## Summary

Resolves all 38 SonarCloud **typescript:S9383** findings ("Promises must
be awaited, end with a call to .catch, end with a call to .then with a
rejection handler or be explicitly marked as ignored with the `void`
operator") in `frontend/apps/artcraft/app`, across 19 files. They are:

- startup calls in `index.tsx`, such as `SyncStorytellerApiConfig()` and
`persistLogin()`
- polling in `useActiveJobs`, `useBackgroundLoadingMedia`,
`signals/activeJobs.ts` and `signals/authentication/fetchers.ts`
- listener cleanups (`unlisten*.then(f => f())`) in `TaskQueue` and
`useDesktopGenerationFeed`
- UI handlers that start uploads or renames, such as
`ImageTo3DExperience`, `UploadFilesMedia`, `SceneTitleInput`,
`ProfileDropdown` and `DesktopGenerationGallery`
- async IIFEs in `useDesktopUsername`, `TaskQueue` and
`BaseImageSelector`

This is part 2 of 3 for artcraft's 157 S9383 findings. Part 1 (Tauri
libraries) is #2805. Part 3 (`libs/components`) is #2807.

Several of these files are also changed by open PRs for unrelated
reasons, for example #2519, #2520, #2615 and #2064. Each change here
inserts one word at the start of a statement, so a rebase either way is
trivial.

## Approach

This ports the approach of storytold/artcraft-services#179, which fixes
the same rule in artcraft-services. **#179 is still open and unmerged.**
This PR follows its approach and does not depend on it.

- Each flagged expression is a fire-and-forget promise: a listener
`setup()`, an `unlisten.then(f => f())` cleanup, a loader called from an
effect, an upload kicked off from a handler, or an async IIFE. Prefixing
it with `void` marks the ignored promise explicitly, which is the fix
the rule documents.
- It deliberately does not add `.catch()` everywhere. These call sites
already handle failures inside the async function or surface them
through their own state, and empty handlers would hide errors.
- Async IIFEs become `void (async () => …)()`. `if (cond) call()`
becomes `if (cond) void call()`.
- There is no behaviour change. `void` only discards the promise value,
which was already being discarded.

Each insertion is placed at the exact `textRange` start offset of the
SonarCloud finding, with no other edits.

## Testing

- Each changed file parses with `esbuild <file>`, using the same check
as #179.
- I parsed each changed file with the TypeScript 5.9 compiler API
(`createSourceFile`). That found no parse errors, and every inserted
`void` is either a whole expression statement or an arrow-function body
applied to a call, never part of a larger expression.
- `frontend/.eslintrc.json` has no `no-void` rule.
- I did not run the full Nx type-check, lint or test targets, because
`frontend/node_modules` is not installed in my environment. Please rely
on CI.
- No tests are added. Discarding an already-discarded promise value is
not observable, so no test can fail before this change and pass after
it.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated background tasks across uploads, sign-in, polling, scene
editing, and other app workflows to handle asynchronous operations
consistently.
* Existing workflows and their success and error handling remain
unchanged; no user-facing behavior changes are expected.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 21:35:11 +01:00
FredApps c9da5cac18 tauri-events/tauri-api/tauri-utils: mark fire-and-forget promises with void (Sonar S9383) (#2805)
## Summary

Resolves all 53 SonarCloud **typescript:S9383** findings ("Promises must
be awaited, end with a call to .catch, end with a call to .then with a
rejection handler or be explicitly marked as ignored with the `void`
operator") in the Tauri frontend libraries:

| Library | Findings | What is marked |

|----------------------------|----------|------------------------------------------------------------------------|
| `frontend/libs/tauri-events` | 48 | `setup()` and the `unlisten.then(f
=> f())` cleanups in each `use*Event` hook |
| `frontend/libs/tauri-api` | 4 | Same pattern in
`CanvasBgRemovedEvent`, plus `setupListener()` in `useSoraLoginListener`
|
| `frontend/libs/tauri-utils` | 1 | The async IIFE in
`useTauriWindowControls` |

This is part 1 of 3 for artcraft's 157 S9383 findings. Part 2
(`apps/artcraft/app`) is #2806, and part 3 (`libs/components`) is #2807.

## Approach

This ports the approach of storytold/artcraft-services#179, which fixes
the same rule in artcraft-services. **#179 is still open and unmerged.**
This PR follows its approach and does not depend on it.

- Each flagged expression is a fire-and-forget promise: a listener
`setup()`, an `unlisten.then(f => f())` cleanup, a loader called from an
effect, an upload kicked off from a handler, or an async IIFE. Prefixing
it with `void` marks the ignored promise explicitly, which is the fix
the rule documents.
- It deliberately does not add `.catch()` everywhere. These call sites
already handle failures inside the async function or surface them
through their own state, and empty handlers would hide errors.
- Async IIFEs become `void (async () => …)()`. `if (cond) call()`
becomes `if (cond) void call()`.
- There is no behaviour change. `void` only discards the promise value,
which was already being discarded.

Each insertion is placed at the exact `textRange` start offset of the
SonarCloud finding, with no other edits.

## Testing

- Each changed file parses with `esbuild <file>`, using the same check
as #179.
- I parsed each changed file with the TypeScript 5.9 compiler API
(`createSourceFile`). That found no parse errors, and every inserted
`void` is either a whole expression statement or an arrow-function body
applied to a call, never part of a larger expression.
- `frontend/.eslintrc.json` has no `no-void` rule.
- I did not run the full Nx type-check, lint or test targets, because
`frontend/node_modules` is not installed in my environment. Please rely
on CI.
- No tests are added. Discarding an already-discarded promise value is
not observable, so no test can fail before this change and pass after
it.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Refactor**
* Clarified that asynchronous event listener setup, cleanup, and
window-control initialization run without blocking. Existing
subscription behavior and cleanup conditions remain unchanged; no
user-facing functionality changes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 21:21:36 +01:00
FredApps 83aad5eb54 Exclude TypeScript and JavaScript from Sonar coverage (#2813)
## Summary

Adds `sonar.coverage.exclusions=**/*.ts,**/*.tsx,**/*.js,**/*.jsx` to
`sonar-project.properties`. There were no existing exclusion entries.

The shared SonarCloud workflow only publishes the Rust lcov report
(`sonar.rust.lcov.reportPaths=lcov.info`). Artcraft's workflow wrapper
does not pass a TypeScript or JavaScript coverage report, so those files
can never appear covered.

That is why [#2805](https://github.com/storytold/artcraft/pull/2805) and
[#2806](https://github.com/storytold/artcraft/pull/2806) fail the
quality gate on **new_coverage** (0.0% against the 80% target) after the
organisation was recreated. They only add `void` in front of existing
TypeScript calls:

- #2805: 53 new lines to cover, all uncovered
- #2806: 38 new lines to cover, all uncovered

[#2807](https://github.com/storytold/artcraft/pull/2807) also touches
TypeScript (`void` in `libs/components`) and is listed here for the same
exclusion, but its current gate failure is duplicated new code (10.6%),
which this PR does not address.

Same approach as
[anicraft#106](https://github.com/storytold/anicraft/pull/106), which
excluded JS, PowerShell and Python for the same reason. filmcraft and
lightcraft already exclude `**/tools/**,**/docs/**,**/examples/**`. This
PR uses extensions because the uncovered TypeScript and JavaScript live
under `frontend/`.

No product code changes. No open PR already edits
`sonar-project.properties`.

## Testing

Visual review of `sonar-project.properties` against anicraft, filmcraft
and lightcraft, and of the shared `sonarcloud.yml` coverage inputs. No
Rust or frontend tests were run (properties file only). The quality-gate
effect on #2805 and #2806 will be visible after this merges and those
branches are updated from main.

- [x] Clean room: no proprietary code, scripts or assets copied
(AGENTS.md)

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated project configuration for coverage reporting. No user-facing
changes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 21:07:25 +01:00
FredApps 50afb36f9c api_clients: use tokio::fs inside async functions (Sonar S7493) (#2803)
## Summary

Replaces blocking `std::fs` calls inside `async fn`s with `tokio::fs` in
four API client crates. SonarCloud reports these as **rust:S7493** ("use
an asynchronous file API instead of this blocking file operation"). This
PR resolves 13 of the 40 artcraft findings.

| Crate | File | Change |

|---------------------|-------------------------------------------------------------|------------------------------------------------------------|
| `artcraft_client` | `endpoints/media_files/*_from_file.rs` (4 files) |
`std::fs::read` → `tokio::fs::read(..).await` |
| `artcraft_client` | `recipes/download_media_file.rs` | `File::create`
+ `write_all` + `flush` → `tokio::fs::write` |
| `artcraft_router` | `generate_video/providers/kinovi/upload.rs` |
`std::fs::read` → `tokio::fs::read(..).await` |
| `midjourney_client` | `credentials/credential_manager.rs`
(`clear_credentials`) | `std::fs::remove_file` →
`tokio::fs::remove_file(..).await` |
| `openai_sora_client`|
`requests/deprecated/job_status/sora_job_status.rs` | `create_dir_all` /
`File::create` + `write_all` → `tokio::fs` |

The workspace `tokio` dependency enables only `macros`.
`artcraft_client` (which had `tokio` only as a dev-dependency) and
`artcraft_router` now declare `tokio = { workspace = true, features =
["fs"] }`. `midjourney_client` and `openai_sora_client` already use
`tokio` with `full`. `Cargo.lock` does not change. In
`save_generations_to_dir`, the blocking `dir.exists()` check before
`create_dir_all` is dropped, because `create_dir_all` already succeeds
when the directory exists.

The behaviour stays the same: same paths, same error mapping, and the
same `NotFound` handling in `clear_credentials`.

## Not included

- `crates/schema/database/sqlite_tasks/src/connection.rs:25` (S7493):
#2360 removes this `remove_file` entirely, so it's left to that PR.
- `crates/desktop/artcraft` (26 findings): handled in a separate PR.

#2365, #2373 and #2785 edit nearby lines in the `*_from_file.rs` upload
files (client builder, idempotency token). None of them touch the
`std::fs::read` lines. Expect at most a trivial rebase.

## Testing

Run locally on Linux x86_64 with stable Rust:

- `cargo check -p artcraft_client -p artcraft_router -p
midjourney_client -p openai_sora_client` passes.
- `cargo test` on the same four crates passes: 2555 + 39 + 16 + 6 unit
tests, with ignored tests left ignored. That includes
`midjourney_client`'s existing
`loads_a_saved_login_and_logout_removes_every_format`, which exercises
the changed `clear_credentials` path.
- `cargo clippy` on these crates reports nothing on the changed lines.
Clippy cannot finish on `main` itself: `enums` fails on
`clippy::deprecated_semver` and `openai_sora_client` fails on
`should_implement_trait` in `task_id.rs`, both pre-existing.

No new test is added. This is a non-blocking I/O refactor with identical
observable behaviour, so there is no test that fails before the change
and passes after it. The other changed paths do network I/O and have no
offline tests. Please rely on CI and the SonarCloud re-analysis to
confirm the S7493 findings close.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Refactor**
* File uploads and downloads, generated image saving, and
credential-file cleanup now use asynchronous file operations.
* When a local file can’t be read during a video upload, the existing
fallback to downloading from the source URL remains in place.
* Existing handling of file-read, write, and removal errors is
preserved, including ignoring missing credential files.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 18:53:57 +01:00
FredApps c3fa0453a4 desktop: use tokio::fs for Sora and Midjourney file I/O (Sonar S7493) (#2804)
## Summary

Replaces blocking `std::fs` calls inside `async fn`s in the desktop
crate with `tokio::fs`. SonarCloud reports these as **rust:S7493** ("use
an asynchronous file API instead of this blocking file operation"). This
PR resolves 17 of the 26 desktop findings.

| File (under `crates/desktop/artcraft/src/services/`) | Findings |
Change |

|----------------------------------------------------------------|----------|----------------------------------------------------------------------------|
| `sora/state/read_sora_credentials_from_disk_with_upgrade.rs` | 10 |
`read_to_string` and `OpenOptions`+`write_all`+`flush` →
`tokio::fs::{read_to_string, write}` |
| `sora/state/initialize_sora_jwt_bearer_token.rs` | 2 |
`std::fs::{read_to_string, write}` → `tokio::fs` |
| `sora/state/sora_credential_manager.rs` (`call_sentinel_refresh`) | 2
| `OpenOptions`+`write_all`+`flush` → `tokio::fs::write` |
| `midjourney/utils/download_midjourney_image.rs` | 2 |
`File::create`+`write_all` → `tokio::fs::write` |
| `midjourney/threads/upload_midjourney_batch.rs` | 1 |
`std::fs::remove_file` → `tokio::fs::remove_file(..).await` |

`tokio::fs::write` creates or truncates the file and writes everything.
That is the same as the
`OpenOptions::new().create(true).write(true).truncate(true)` +
`write_all` + `flush` sequence it replaces. The synchronous helpers in
`sora_credential_manager.rs` (`persist_*_to_disk`) are not `async`, are
not flagged, and are left unchanged. The `.exists()` guards are also
left as they are. The desktop crate already depends on `tokio` with the
`fs` feature, so no manifest or lockfile changes.

## Not included (overlap with open PRs)

The remaining 9 desktop S7493 findings are in files that open PRs are
rewriting. They are left alone to avoid conflicting changes:

- `core/utils/simple_http_download.rs`,
`sora/.../handle_successful_generations.rs`,
`worldlabs/threads/worldlabs_marble_task_polling.rs`: #2777 (fail
downloads on non-2xx responses).
- `core/threads/.../fal/handle_fal_complete.rs`: #2550, #2373.
- `core/utils/auto_download.rs`: #2785, #2686, #2393.

The api_clients findings are in #2803.

## Testing

- Each changed file parses with `rustfmt --check`. The formatting diffs
it reports already exist on `main`, and no files were reformatted.
- **I could not build the desktop crate locally.** `cargo check -p
artcraft` stops at `libdbus-sys` because the build machine lacks the
GTK, WebKit and D-Bus system libraries Tauri needs. Please rely on CI
for compilation before marking this ready.
- No new tests are added. This is a behaviour-preserving switch to
non-blocking I/O, so no test fails before the change and passes after
it, and these paths have no existing offline tests.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated background handling for saved images, checkpoints, and account
credentials. Existing cleanup, credential processing, and error
reporting behavior remain unchanged.
* These maintenance updates do not change the app’s available features
or how people interact with it.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 18:36:03 +01:00
FredAppsandFredApps 1b99c184ef ci: one macOS and one Windows release build per branch at a time, with job time limits (#2809)
This changes release workflows (house rule 9). It only adds limits, but
it needs a maintainer's look before merging.

## Problem

`artcraft-macos-publish.yml` and `artcraft-windows-publish.yml` have no
`concurrency` and no `timeout-minutes`. `artcraft-linux-publish.yml`
already has both.

- **Overlapping builds:** two pushes to `release` or `artcraft-release`
in quick succession start two builds of the same platform side by side.
Both create or update the same `artcraft-v<version>` draft release
through `tauri-action`. Nothing stops this today. It happens whenever a
second push lands while a 10-36 minute build is still running. (The two
runs on 2026-11-07 were 35 minutes apart and did not overlap.)
- **Hung builds:** a build that hangs runs until GitHub's 6-hour
default. The Windows run on 2026-06-12 did exactly that and was
cancelled at 360 minutes. Successful builds take far less:

| workflow | successful builds, last 15 runs (2026-04 to 2026-11) |
|---|---|
| macOS publish | 10-22 min |
| Windows publish | 17-36 min |

## Change

Two files, limits only. Triggers, steps, secrets, signing, release
settings and action versions are unchanged.

- **Concurrency:** `concurrency: { group: macos-release-${{ github.ref
}}, cancel-in-progress: false }`, and likewise `windows-release-…`. This
is the same pattern as `artcraft-linux-publish.yml` (`linux-release-${{
github.ref }}`).
- A release build that is already running is **never cancelled**, since
it may already have uploaded to the draft release.
- One GitHub behaviour to be aware of: a concurrency group holds one
running and one pending run. If a third push lands while one build runs
and another waits, GitHub replaces the waiting run with the newest. So
the next build is always of the latest commit on that branch, and an
intermediate commit may not get its own build. For a release branch I
think that is what we want. If every commit must get a build, this PR
shouldn't be merged as it is; tell me and I'll change it.
- The groups are per platform and per branch, so macOS, Windows and
Linux still build in parallel with each other.
- **Timeouts:** `timeout-minutes: 60` for macOS and `90` for Windows.
That is about 2.5-3× the slowest successful build, and Windows matches
the Linux build job's 90. A hang now stops after an hour or an hour and
a half instead of six.

Not changed: `dtolnay/rust-toolchain@stable` and
`tauri-apps/tauri-action@v0` are still unpinned tags. Pinning them would
change which action code builds a release, so it should be a separate
decision.

## Verification

- `actionlint` is clean on the three publish workflows.
- These workflows only run on pushes to `release`, `artcraft-release`,
`artcraft-macos` and `artcraft-windows`, so this PR's checks don't
exercise them. The new settings take effect on the next release push.
- I didn't trigger a release build.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Release builds for macOS and Windows now finish in progress without
being cancelled by newer runs. When multiple builds are queued for the
same branch or ref, only the newest pending build is kept.
* Publishing jobs now have maximum run times of 60 minutes for macOS and
90 minutes for Windows.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 16:45:35 +01:00
AxylumandCédric Legrand 1510abf563 url_utils: satisfy Clippy pedantic lints (#2681)
## Update — 2026-11-01

The latest revision removes the systematic `#[must_use]` additions and
any product-name backticks introduced solely for optional
`clippy::pedantic` compliance. The useful behavior-preserving cleanups
remain. Earlier summary and verification text below describes the
original revision; the latest PR comment records the current checks.

---

## Summary
- resolve all 54 distinct diagnostics in `url_utils`: 17 pedantic, 36
deprecation warnings, and one Eq/Hash contract issue
- preserve the public compatibility module and nested module paths with
narrow documented expectations
- implement `Hash` against the same bare value as `PartialEq`, with a
regression test
- apply the repository two-space rustfmt rules within `url_utils` only

No public API or dependency changes.

## Verification
- `SQLX_OFFLINE=true cargo clippy -p url_utils --all-targets -- -W
clippy::pedantic -D warnings`
- `SQLX_OFFLINE=true cargo test -p url_utils` — 32 unit tests and 2
doctests passed
- `cargo fmt -p url_utils -- --check`
- Independent read-only review: no Rust findings; the untracked planning
file is excluded from the PR

---------

Co-authored-by: Cédric Legrand <cedric@legrand-tech.fr>
2026-11-10 11:32:48 +01:00
Dylan M. Taylor 1218293fe2 docs: add CodeRabbit badge to README (#2790)
Add CodeRabbit review badge to README alongside SonarCloud.
2026-11-09 10:09:12 -05:00
Dylan M. Taylor 56b3090924 ci: dual-push Flatpak OCI bundles to Docker Hub (#2784)
Mirror Flatpak OCI bundles to Docker Hub
(`docker.io/storytold/artcraft`) alongside GHCR when Docker Hub
credentials (`DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN`) are present.
2026-11-08 20:22:46 -05:00
Dylan M. Taylor 79d87b2b54 fix(flatpak): persist data directory, keep accelerated rendering, and enrich metainfo (#2761)
## Summary

Transfers verified improvements and reviewer feedback from the FlatPark
package review to ArtCraft's first-party Flatpak build:

- **Data Persistence:** Adds `--persist=Artcraft` to `finish-args` in
`flatpak.yml` so user settings, credentials, provider tokens, and local
task SQLite queue stored in `~/Artcraft` persist in
`~/.var/app/ai.artcraft.app/Artcraft` across launches instead of being
discarded on exit.
- **Accelerated Rendering:** Adds `artcraft-wrapper` setting
`WEBKIT_DISABLE_DMABUF_RENDERER="${WEBKIT_DISABLE_DMABUF_RENDERER:-0}"`
and
`WEBKIT_DISABLE_COMPOSITING_MODE="${WEBKIT_DISABLE_COMPOSITING_MODE:-0}"`.
Keeping WebKitGTK's accelerated rendering path enabled prevents
rendering glitches on `org.gnome.Platform//51` while preserving user
override capability.
- **Dependency Checking:** Updates the Flatpak build runtime
verification to inspect the real binary (`/app/bin/artcraft.real`).
- **Enriched AppStream Metainfo:** Updates
`ai.artcraft.app.metainfo.xml` with CC0-1.0 metadata license,
bugtracker/vcs-browser/contact URLs, bulleted feature description,
search keywords, branding colors, hardware display requirements, input
controls, and OARS-1.1 content rating.

## Verification

- `actionlint .github/workflows/*.yml` passed.
- `shellcheck packaging/linux/*.sh packaging/linux/artcraft-wrapper`
passed.
- `xmllint --noout packaging/linux/ai.artcraft.app.metainfo.xml` passed.
- `appstreamcli validate packaging/linux/ai.artcraft.app.metainfo.xml`
passed with 0 errors and 0 info notices.
2026-11-06 22:48:03 -05:00
dependabot[bot] 1ae54cabc0 ci(deps): bump the actions group across 1 directory with 4 updates (#2760)
Bumps the actions group with 4 updates in the / directory:
[actions/checkout](https://github.com/actions/checkout),
[actions/setup-node](https://github.com/actions/setup-node),
[actions/upload-artifact](https://github.com/actions/upload-artifact)
and
[actions/download-artifact](https://github.com/actions/download-artifact).

Updates `actions/checkout` from 4 to 7
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/releases">actions/checkout's
releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<h2>What's Changed</h2>
<ul>
<li>block checking out fork pr for pull_request_target and workflow_run
by <a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
<li>Bump actions/publish-immutable-action from 0.0.3 to 0.0.4 in the
minor-actions-dependencies group across 1 directory by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2458">actions/checkout#2458</a></li>
<li>Bump flatted from 3.3.1 to 3.4.2 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2460">actions/checkout#2460</a></li>
<li>Bump js-yaml from 4.1.0 to 4.2.0 by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2461">actions/checkout#2461</a></li>
<li>Bump <code>@​actions/core</code> and
<code>@​actions/tool-cache</code> and Remove uuid by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2459">actions/checkout#2459</a></li>
<li>upgrade module to esm and update dependencies by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2463">actions/checkout#2463</a></li>
<li>Bump the minor-npm-dependencies group across 1 directory with 3
updates by <a
href="https://github.com/dependabot"><code>@​dependabot</code></a>[bot]
in <a
href="https://redirect.github.com/actions/checkout/pull/2462">actions/checkout#2462</a></li>
<li>getting ready for checkout v7 release by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2464">actions/checkout#2464</a></li>
<li>update error wording by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2467">actions/checkout#2467</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6.0.3...v7.0.0">https://github.com/actions/checkout/compare/v6.0.3...v7.0.0</a></p>
<h2>v6.1.0</h2>
<h2>What's Changed</h2>
<ul>
<li><strong>[BREAKING]</strong> backport
<code>allow-unsafe-pr-checkout</code> to v6 by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2500">actions/checkout#2500</a></li>
<li>backport fixes to releases-v6 by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2527">actions/checkout#2527</a></li>
</ul>
<p><a
href="https://github.blog/changelog/2026-06-18-safer-pull_request_target-defaults-for-github-actions-checkout/">https://github.blog/changelog/2026-06-18-safer-pull_request_target-defaults-for-github-actions-checkout/</a>
for more details about this breaking change</p>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6.0.3...v6.1.0">https://github.com/actions/checkout/compare/v6.0.3...v6.1.0</a></p>
<h2>v6.0.3</h2>
<h2>What's Changed</h2>
<ul>
<li>Update changelog by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2357">actions/checkout#2357</a></li>
<li>fix: expand merge commit SHA regex and add SHA-256 test cases by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
<li>Fix checkout init for SHA-256 repositories by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2439">actions/checkout#2439</a></li>
<li>Update changelog for v6.0.3 by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2446">actions/checkout#2446</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/yaananth"><code>@​yaananth</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6...v6.0.3">https://github.com/actions/checkout/compare/v6...v6.0.3</a></p>
<h2>v6.0.2</h2>
<h2>What's Changed</h2>
<ul>
<li>Add orchestration_id to git user-agent when ACTIONS_ORCHESTRATION_ID
is set by <a
href="https://github.com/TingluoHuang"><code>@​TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2355">actions/checkout#2355</a></li>
<li>Fix tag handling: preserve annotations and explicit fetch-tags by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/checkout/compare/v6.0.1...v6.0.2">https://github.com/actions/checkout/compare/v6.0.1...v6.0.2</a></p>
<h2>v6.0.1</h2>
<h2>What's Changed</h2>
<ul>
<li>Update all references from v5 and v4 to v6 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2314">actions/checkout#2314</a></li>
<li>Add worktree support for persist-credentials includeIf by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li>
<li>Clarify v6 README by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2328">actions/checkout#2328</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Changelog</summary>
<p><em>Sourced from <a
href="https://github.com/actions/checkout/blob/main/CHANGELOG.md">actions/checkout's
changelog</a>.</em></p>
<blockquote>
<h1>Changelog</h1>
<h2>v7.0.1</h2>
<ul>
<li>Skip running unsafe pr check if input is default by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2518">actions/checkout#2518</a></li>
<li>Trim only ascii whitespace for branch by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2521">actions/checkout#2521</a></li>
<li>Escape values passed to --unset by <a
href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2530">actions/checkout#2530</a></li>
<li>Various dependency updates</li>
</ul>
<h2>v7.0.0</h2>
<ul>
<li>Block checking out fork PR for pull_request_target and workflow_run
by <a href="https://github.com/aiqiaoy"><code>@​aiqiaoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2454">actions/checkout#2454</a></li>
<li>Various dependency updates</li>
</ul>
<h2>v6.0.3</h2>
<ul>
<li>Fix checkout init for SHA-256 repositories by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2439">actions/checkout#2439</a></li>
<li>fix: expand merge commit SHA regex and add SHA-256 test cases by <a
href="https://github.com/yaananth"><code>@​yaananth</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2414">actions/checkout#2414</a></li>
</ul>
<h2>v6.0.2</h2>
<ul>
<li>Fix tag handling: preserve annotations and explicit fetch-tags by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2356">actions/checkout#2356</a></li>
</ul>
<h2>v6.0.1</h2>
<ul>
<li>Add worktree support for persist-credentials includeIf by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2327">actions/checkout#2327</a></li>
</ul>
<h2>v6.0.0</h2>
<ul>
<li>Persist creds to a separate file by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2286">actions/checkout#2286</a></li>
<li>Update README to include Node.js 24 support details and requirements
by <a href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2248">actions/checkout#2248</a></li>
</ul>
<h2>v5.0.1</h2>
<ul>
<li>Port v6 cleanup to v5 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2301">actions/checkout#2301</a></li>
</ul>
<h2>v5.0.0</h2>
<ul>
<li>Update actions checkout to use node 24 by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2226">actions/checkout#2226</a></li>
</ul>
<h2>v4.3.1</h2>
<ul>
<li>Port v6 cleanup to v4 by <a
href="https://github.com/ericsciple"><code>@​ericsciple</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2305">actions/checkout#2305</a></li>
</ul>
<h2>v4.3.0</h2>
<ul>
<li>docs: update README.md by <a
href="https://github.com/motss"><code>@​motss</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1971">actions/checkout#1971</a></li>
<li>Add internal repos for checking out multiple repositories by <a
href="https://github.com/mouismail"><code>@​mouismail</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1977">actions/checkout#1977</a></li>
<li>Documentation update - add recommended permissions to Readme by <a
href="https://github.com/benwells"><code>@​benwells</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2043">actions/checkout#2043</a></li>
<li>Adjust positioning of user email note and permissions heading by <a
href="https://github.com/joshmgross"><code>@​joshmgross</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2044">actions/checkout#2044</a></li>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@​nebuk89</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2194">actions/checkout#2194</a></li>
<li>Update CODEOWNERS for actions by <a
href="https://github.com/TingluoHuang"><code>@​TingluoHuang</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/2224">actions/checkout#2224</a></li>
<li>Update package dependencies by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/2236">actions/checkout#2236</a></li>
</ul>
<h2>v4.2.2</h2>
<ul>
<li><code>url-helper.ts</code> now leverages well-known environment
variables by <a href="https://github.com/jww3"><code>@​jww3</code></a>
in <a
href="https://redirect.github.com/actions/checkout/pull/1941">actions/checkout#1941</a></li>
<li>Expand unit test coverage for <code>isGhes</code> by <a
href="https://github.com/jww3"><code>@​jww3</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1946">actions/checkout#1946</a></li>
</ul>
<h2>v4.2.1</h2>
<ul>
<li>Check out other refs/* by commit if provided, fall back to ref by <a
href="https://github.com/orhantoy"><code>@​orhantoy</code></a> in <a
href="https://redirect.github.com/actions/checkout/pull/1924">actions/checkout#1924</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/checkout/commit/3d3c42e5aac5ba805825da76410c181273ba90b1"><code>3d3c42e</code></a>
prep v7.0.1 release (<a
href="https://redirect.github.com/actions/checkout/issues/2531">#2531</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/28802689a136bfcdb721715abd713740beecbe07"><code>2880268</code></a>
escape values passed to --unset (<a
href="https://redirect.github.com/actions/checkout/issues/2530">#2530</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/12cd2235efa0937479335606d7c3ac9f6c0973b1"><code>12cd223</code></a>
trim only ascii whitespace for branch (<a
href="https://redirect.github.com/actions/checkout/issues/2521">#2521</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/62661c4e71a304b2823ed026347b8d34c3eac541"><code>62661c4</code></a>
skip running unsafe pr check if input is default (<a
href="https://redirect.github.com/actions/checkout/issues/2518">#2518</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/e8d4307400f9427dba7cb98e488d6ab85f1cec5f"><code>e8d4307</code></a>
Bump the minor-actions-dependencies group with 2 updates (<a
href="https://redirect.github.com/actions/checkout/issues/2499">#2499</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/631c942040754b6e095e929c1677c07e10ed4f87"><code>631c942</code></a>
eslint 9 (<a
href="https://redirect.github.com/actions/checkout/issues/2474">#2474</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/4f1f4aec02e41874fa0262ea8ff5172d7978ad1e"><code>4f1f4ae</code></a>
Bump actions/upload-artifact from 4 to 7 (<a
href="https://redirect.github.com/actions/checkout/issues/2476">#2476</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/ba097532fb203f7e88c9c3c0b899b49469908a92"><code>ba09753</code></a>
Bump actions/checkout from 6 to 7 (<a
href="https://redirect.github.com/actions/checkout/issues/2488">#2488</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/b9e0990d219a03df7633c93f6f005a8fecbcab22"><code>b9e0990</code></a>
Bump docker/login-action from 3.3.0 to 4.2.0 (<a
href="https://redirect.github.com/actions/checkout/issues/2479">#2479</a>)</li>
<li><a
href="https://github.com/actions/checkout/commit/e8cb398be4a550817e382abf69e4c12c76fce1f2"><code>e8cb398</code></a>
Bump docker/build-push-action from 6.5.0 to 7.2.0 (<a
href="https://redirect.github.com/actions/checkout/issues/2478">#2478</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/checkout/compare/v4...v7">compare
view</a></li>
</ul>
</details>
<br />

Updates `actions/setup-node` from 4 to 7
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/setup-node/releases">actions/setup-node's
releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<h2>What's Changed</h2>
<h3>Enhancements:</h3>
<ul>
<li>Add cache-primary-key and cache-matched-key as outputs by <a
href="https://github.com/gowridurgad"><code>@​gowridurgad</code></a> in
<a
href="https://redirect.github.com/actions/setup-node/pull/1577">actions/setup-node#1577</a></li>
<li>Migrate to ESM and upgrade dependencies by <a
href="https://github.com/gowridurgad"><code>@​gowridurgad</code></a> in
<a
href="https://redirect.github.com/actions/setup-node/pull/1574">actions/setup-node#1574</a></li>
</ul>
<h3>Bug fixes:</h3>
<ul>
<li>Remove dummy NODE_AUTH_TOKEN export by <a
href="https://github.com/gowridurgad"><code>@​gowridurgad</code></a> in
<a
href="https://redirect.github.com/actions/setup-node/pull/1558">actions/setup-node#1558</a></li>
<li>Only use <code>mirrorToken</code> in <code>getManifest</code> if
it's provided by <a
href="https://github.com/deiga"><code>@​deiga</code></a> in <a
href="https://redirect.github.com/actions/setup-node/pull/1548">actions/setup-node#1548</a></li>
</ul>
<h3>Documentation updates:</h3>
<ul>
<li>Add documentation for publishing to npm with Trusted Publisher
(OIDC) by <a
href="https://github.com/chiranjib-swain"><code>@​chiranjib-swain</code></a>
in <a
href="https://redirect.github.com/actions/setup-node/pull/1536">actions/setup-node#1536</a></li>
<li>docs: Update restore-only cache documentation by <a
href="https://github.com/priya-kinthali"><code>@​priya-kinthali</code></a>
in <a
href="https://redirect.github.com/actions/setup-node/pull/1550">actions/setup-node#1550</a></li>
<li>docs: Update caching recommendations to mitigate cache poisoning
risks by <a
href="https://github.com/chiranjib-swain"><code>@​chiranjib-swain</code></a>
in <a
href="https://redirect.github.com/actions/setup-node/pull/1567">actions/setup-node#1567</a></li>
</ul>
<h3>Dependency update:</h3>
<ul>
<li>Upgrade <code>@​actions/cache</code> to 5.1.0, log cache write
denied by <a
href="https://github.com/jasongin"><code>@​jasongin</code></a> in <a
href="https://redirect.github.com/actions/setup-node/pull/1569">actions/setup-node#1569</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/chiranjib-swain"><code>@​chiranjib-swain</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/setup-node/pull/1536">actions/setup-node#1536</a></li>
<li><a href="https://github.com/deiga"><code>@​deiga</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-node/pull/1548">actions/setup-node#1548</a></li>
<li><a href="https://github.com/jasongin"><code>@​jasongin</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/setup-node/pull/1569">actions/setup-node#1569</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-node/compare/v6...v7.0.0">https://github.com/actions/setup-node/compare/v6...v7.0.0</a></p>
<h2>v6.5.0</h2>
<h2>What's Changed</h2>
<ul>
<li>Update <code>@​actions/cache</code> to 5.1.0 and add security
overrides for undici and fast-xml-parser by <a
href="https://github.com/HarithaVattikuti"><code>@​HarithaVattikuti</code></a>
in <a
href="https://redirect.github.com/actions/setup-node/pull/1579">actions/setup-node#1579</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-node/compare/v6.4.0...v6.5.0">https://github.com/actions/setup-node/compare/v6.4.0...v6.5.0</a></p>
<h2>v6.4.0</h2>
<h2>What's Changed</h2>
<h3>Dependency updates:</h3>
<ul>
<li>Upgrade <a
href="https://github.com/actions"><code>@​actions</code></a>
dependencies by <a
href="https://github.com/Copilot"><code>@​Copilot</code></a> in <a
href="https://redirect.github.com/actions/setup-node/pull/1525">actions/setup-node#1525</a></li>
<li>Update Node.js versions in versions.yml and bump package to v6.4.0
by <a
href="https://github.com/priya-kinthali"><code>@​priya-kinthali</code></a>
in <a
href="https://redirect.github.com/actions/setup-node/pull/1533">actions/setup-node#1533</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Copilot"><code>@​Copilot</code></a> made
their first contribution in <a
href="https://redirect.github.com/actions/setup-node/pull/1525">actions/setup-node#1525</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/setup-node/compare/v6...v6.4.0">https://github.com/actions/setup-node/compare/v6...v6.4.0</a></p>
<h2>v6.3.0</h2>
<h2>What's Changed</h2>
<h3>Enhancements:</h3>
<ul>
<li>Support parsing <code>devEngines</code> field by <a
href="https://github.com/susnux"><code>@​susnux</code></a> in <a
href="https://redirect.github.com/actions/setup-node/pull/1283">actions/setup-node#1283</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/setup-node/commit/949feb2413d6458794dcd2491c4babbbce0c15c1"><code>949feb2</code></a>
Bump eslint from 10.5.0 to 10.11.0 and package.json from 7.0.0 to 7.1.0
(<a
href="https://redirect.github.com/actions/setup-node/issues/1635">#1635</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/7207c4f1189c7ee83ddf20c7c68bd1af6fdd751c"><code>7207c4f</code></a>
chore: update dependencies and add license files (<a
href="https://redirect.github.com/actions/setup-node/issues/1642">#1642</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/54337e0d02de9202d0948aaff36ad556375497b9"><code>54337e0</code></a>
Bump eslint-plugin-n from 18.1.0 to 18.3.0 (<a
href="https://redirect.github.com/actions/setup-node/issues/1638">#1638</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/fb54fa8c14c63410cc752f0ae3b7b3ca6af161fc"><code>fb54fa8</code></a>
Bump jest-each from 30.4.1 to 30.5.2 (<a
href="https://redirect.github.com/actions/setup-node/issues/1636">#1636</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/7f84f1d52d0c56afb45c62fee31a93b843cea4d6"><code>7f84f1d</code></a>
Bump pnpm/action-setup from 6.0.10 to 6.1.0 (<a
href="https://redirect.github.com/actions/setup-node/issues/1631">#1631</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/0c54c7b07420dcba69a7d832587f55ebdf771c9e"><code>0c54c7b</code></a>
Bump <code>@​typescript-eslint/parser</code> from 8.62.0 to 8.70.0 (<a
href="https://redirect.github.com/actions/setup-node/issues/1615">#1615</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/a8294190049a9c296f01b9a432aff41dd83d8284"><code>a829419</code></a>
Bump <code>@​types/semver</code> from 7.7.1 to 7.8.0 (<a
href="https://redirect.github.com/actions/setup-node/issues/1616">#1616</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/680d1e489b82f1243de15bc7665c0007e3c9a860"><code>680d1e4</code></a>
Support absolute paths in node-version-file (<a
href="https://redirect.github.com/actions/setup-node/issues/1633">#1633</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/db0fb3a98ed314b69026b6e8c9be26a7e2fe65a8"><code>db0fb3a</code></a>
Bump baseline-browser-mapping from 2.10.38 to 2.11.22 (<a
href="https://redirect.github.com/actions/setup-node/issues/1629">#1629</a>)</li>
<li><a
href="https://github.com/actions/setup-node/commit/94196ee1d15439c1b6651cd87ef14e88ec435966"><code>94196ee</code></a>
Bump pnpm/action-setup from 6.0.9 to 6.0.10 (<a
href="https://redirect.github.com/actions/setup-node/issues/1602">#1602</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/setup-node/compare/v4...v7">compare
view</a></li>
</ul>
</details>
<br />

Updates `actions/upload-artifact` from 4 to 7
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/upload-artifact/releases">actions/upload-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v7.0.0</h2>
<h2>v7 What's new</h2>
<h3>Direct Uploads</h3>
<p>Adds support for uploading single files directly (unzipped). Callers
can set the new <code>archive</code> parameter to <code>false</code> to
skip zipping the file during upload. Right now, we only support single
files. The action will fail if the glob passed resolves to multiple
files. The <code>name</code> parameter is also ignored with this
setting. Instead, the name of the artifact will be the name of the
uploaded file.</p>
<h3>ESM</h3>
<p>To support new versions of the <code>@actions/*</code> packages,
we've upgraded the package to ESM.</p>
<h2>What's Changed</h2>
<ul>
<li>Add proxy integration test by <a
href="https://github.com/Link"><code>@​Link</code></a>- in <a
href="https://redirect.github.com/actions/upload-artifact/pull/754">actions/upload-artifact#754</a></li>
<li>Upgrade the module to ESM and bump dependencies by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/upload-artifact/pull/762">actions/upload-artifact#762</a></li>
<li>Support direct file uploads by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/upload-artifact/pull/764">actions/upload-artifact#764</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a href="https://github.com/Link"><code>@​Link</code></a>- made
their first contribution in <a
href="https://redirect.github.com/actions/upload-artifact/pull/754">actions/upload-artifact#754</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-artifact/compare/v6...v7.0.0">https://github.com/actions/upload-artifact/compare/v6...v7.0.0</a></p>
<h2>v6.0.0</h2>
<h2>v6 - What's new</h2>
<blockquote>
<p>[!IMPORTANT]
actions/upload-artifact@v6 now runs on Node.js 24 (<code>runs.using:
node24</code>) and requires a minimum Actions Runner version of 2.327.1.
If you are using self-hosted runners, ensure they are updated before
upgrading.</p>
</blockquote>
<h3>Node.js 24</h3>
<p>This release updates the runtime to Node.js 24. v5 had preliminary
support for Node.js 24, however this action was by default still running
on Node.js 20. Now this action by default will run on Node.js 24.</p>
<h2>What's Changed</h2>
<ul>
<li>Upload Artifact Node 24 support by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/719">actions/upload-artifact#719</a></li>
<li>fix: update <code>@​actions/artifact</code> for Node.js 24 punycode
deprecation by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/744">actions/upload-artifact#744</a></li>
<li>prepare release v6.0.0 for Node.js 24 support by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/745">actions/upload-artifact#745</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/upload-artifact/compare/v5.0.0...v6.0.0">https://github.com/actions/upload-artifact/compare/v5.0.0...v6.0.0</a></p>
<h2>v5.0.0</h2>
<h2>What's Changed</h2>
<p><strong>BREAKING CHANGE:</strong> this update supports Node
<code>v24.x</code>. This is not a breaking change per-se but we're
treating it as such.</p>
<ul>
<li>Update README.md by <a
href="https://github.com/GhadimiR"><code>@​GhadimiR</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/681">actions/upload-artifact#681</a></li>
<li>Update README.md by <a
href="https://github.com/nebuk89"><code>@​nebuk89</code></a> in <a
href="https://redirect.github.com/actions/upload-artifact/pull/712">actions/upload-artifact#712</a></li>
<li>Readme: spell out the first use of GHES by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/upload-artifact/pull/727">actions/upload-artifact#727</a></li>
<li>Update GHES guidance to include reference to Node 20 version by <a
href="https://github.com/patrikpolyak"><code>@​patrikpolyak</code></a>
in <a
href="https://redirect.github.com/actions/upload-artifact/pull/725">actions/upload-artifact#725</a></li>
<li>Bump <code>@actions/artifact</code> to <code>v4.0.0</code></li>
<li>Prepare <code>v5.0.0</code> by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/upload-artifact/pull/734">actions/upload-artifact#734</a></li>
</ul>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/upload-artifact/commit/cf430e030ddbb5b0abf93d22962f4752f3646cd9"><code>cf430e0</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/822">#822</a>
from actions/tunc-d-prepare-upload-v7-0-2</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/c2acdedd35a8824a1552c7137f8c18288c3ea469"><code>c2acded</code></a>
chore: prepare v7.0.2 release</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/4687c1bbaa0a58abb517d90b4bbade6754c80c1a"><code>4687c1b</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/821">#821</a>
from actions/tunc-d-upload-artifact-package-bump</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/8cbe1bebdb858105d8fdd88f81404d4de4595719"><code>8cbe1be</code></a>
Bump <code>@​actions/artifact</code> to 6.3.1</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/043fb46d1a93c77aae656e7c1c64a875d1fc6a0a"><code>043fb46</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/upload-artifact/issues/797">#797</a>
from actions/yacaovsnc/update-dependency</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/634250c1388765ea7ed0f053e636f1f399000b94"><code>634250c</code></a>
Include changes in typespec/ts-http-runtime 0.3.5</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/e454baaac2be505c9450e11b8f3215c6fc023ce8"><code>e454baa</code></a>
Readme: bump all the example versions to v7 (<a
href="https://redirect.github.com/actions/upload-artifact/issues/796">#796</a>)</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/74fad66b98a6d799dc004d3353ccd0e6f6b2530e"><code>74fad66</code></a>
Update the readme with direct upload details (<a
href="https://redirect.github.com/actions/upload-artifact/issues/795">#795</a>)</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/bbbca2ddaa5d8feaa63e36b76fdaad77386f024f"><code>bbbca2d</code></a>
Support direct file uploads (<a
href="https://redirect.github.com/actions/upload-artifact/issues/764">#764</a>)</li>
<li><a
href="https://github.com/actions/upload-artifact/commit/589182c5a4cec8920b8c1bce3e2fab1c97a02296"><code>589182c</code></a>
Upgrade the module to ESM and bump dependencies (<a
href="https://redirect.github.com/actions/upload-artifact/issues/762">#762</a>)</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/upload-artifact/compare/v4...v7">compare
view</a></li>
</ul>
</details>
<br />

Updates `actions/download-artifact` from 4 to 8
<details>
<summary>Release notes</summary>
<p><em>Sourced from <a
href="https://github.com/actions/download-artifact/releases">actions/download-artifact's
releases</a>.</em></p>
<blockquote>
<h2>v8.0.0</h2>
<h2>v8 - What's new</h2>
<blockquote>
<p>[!IMPORTANT]
actions/download-artifact@v8 has been migrated to an ESM module. This
should be transparent to the caller but forks might need to make
significant changes.</p>
</blockquote>
<blockquote>
<p>[!IMPORTANT]
Hash mismatches will now error by default. Users can override this
behavior with a setting change (see below).</p>
</blockquote>
<h3>Direct downloads</h3>
<p>To support direct uploads in <code>actions/upload-artifact</code>,
the action will no longer attempt to unzip all downloaded files.
Instead, the action checks the <code>Content-Type</code> header ahead of
unzipping and skips non-zipped files. Callers wishing to download a
zipped file as-is can also set the new <code>skip-decompress</code>
parameter to <code>true</code>.</p>
<h3>Enforced checks (breaking)</h3>
<p>A previous release introduced digest checks on the download. If a
download hash didn't match the expected hash from the server, the action
would log a warning. Callers can now configure the behavior on mismatch
with the <code>digest-mismatch</code> parameter. To be secure by
default, we are now defaulting the behavior to <code>error</code> which
will fail the workflow run.</p>
<h3>ESM</h3>
<p>To support new versions of the @actions/* packages, we've upgraded
the package to ESM.</p>
<h2>What's Changed</h2>
<ul>
<li>Don't attempt to un-zip non-zipped downloads by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/download-artifact/pull/460">actions/download-artifact#460</a></li>
<li>Add a setting to specify what to do on hash mismatch and default it
to <code>error</code> by <a
href="https://github.com/danwkennedy"><code>@​danwkennedy</code></a> in
<a
href="https://redirect.github.com/actions/download-artifact/pull/461">actions/download-artifact#461</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/download-artifact/compare/v7...v8.0.0">https://github.com/actions/download-artifact/compare/v7...v8.0.0</a></p>
<h2>v7.0.0</h2>
<h2>v7 - What's new</h2>
<blockquote>
<p>[!IMPORTANT]
actions/download-artifact@v7 now runs on Node.js 24 (<code>runs.using:
node24</code>) and requires a minimum Actions Runner version of 2.327.1.
If you are using self-hosted runners, ensure they are updated before
upgrading.</p>
</blockquote>
<h3>Node.js 24</h3>
<p>This release updates the runtime to Node.js 24. v6 had preliminary
support for Node 24, however this action was by default still running on
Node.js 20. Now this action by default will run on Node.js 24.</p>
<h2>What's Changed</h2>
<ul>
<li>Update GHES guidance to include reference to Node 20 version by <a
href="https://github.com/patrikpolyak"><code>@​patrikpolyak</code></a>
in <a
href="https://redirect.github.com/actions/download-artifact/pull/440">actions/download-artifact#440</a></li>
<li>Download Artifact Node24 support by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/download-artifact/pull/415">actions/download-artifact#415</a></li>
<li>fix: update <code>@​actions/artifact</code> to fix Node.js 24
punycode deprecation by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/download-artifact/pull/451">actions/download-artifact#451</a></li>
<li>prepare release v7.0.0 for Node.js 24 support by <a
href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a> in <a
href="https://redirect.github.com/actions/download-artifact/pull/452">actions/download-artifact#452</a></li>
</ul>
<h2>New Contributors</h2>
<ul>
<li><a
href="https://github.com/patrikpolyak"><code>@​patrikpolyak</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/download-artifact/pull/440">actions/download-artifact#440</a></li>
<li><a href="https://github.com/salmanmkc"><code>@​salmanmkc</code></a>
made their first contribution in <a
href="https://redirect.github.com/actions/download-artifact/pull/415">actions/download-artifact#415</a></li>
</ul>
<p><strong>Full Changelog</strong>: <a
href="https://github.com/actions/download-artifact/compare/v6.0.0...v7.0.0">https://github.com/actions/download-artifact/compare/v6.0.0...v7.0.0</a></p>
<h2>v6.0.0</h2>
<!-- raw HTML omitted -->
</blockquote>
<p>... (truncated)</p>
</details>
<details>
<summary>Commits</summary>
<ul>
<li><a
href="https://github.com/actions/download-artifact/commit/9000827ccba6bdab643e8b6fd33ac0654aef8333"><code>9000827</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/493">#493</a>
from actions/tunc-d-prepare-download-v8-0-2</li>
<li><a
href="https://github.com/actions/download-artifact/commit/e455989ae00f5bdd4054f05d3e972c6bf8a62dc5"><code>e455989</code></a>
Prepare download-artifact v8.0.2</li>
<li><a
href="https://github.com/actions/download-artifact/commit/a00b90c4a2f01d73a8c89552caabca6ea3b43137"><code>a00b90c</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/492">#492</a>
from actions/tunc-d-download-artifact-package-bump</li>
<li><a
href="https://github.com/actions/download-artifact/commit/29bb7ce6af5345aeab5bb4774ce7249ee3ccc27f"><code>29bb7ce</code></a>
Update <code>@​actions/artifact</code> to 6.3.1</li>
<li><a
href="https://github.com/actions/download-artifact/commit/484a0b528fb4d7bd804637ccb632e47a0e638317"><code>484a0b5</code></a>
Update the readme (<a
href="https://redirect.github.com/actions/download-artifact/issues/473">#473</a>)</li>
<li><a
href="https://github.com/actions/download-artifact/commit/3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c"><code>3e5f45b</code></a>
Add regression tests for CJK characters (<a
href="https://redirect.github.com/actions/download-artifact/issues/471">#471</a>)</li>
<li><a
href="https://github.com/actions/download-artifact/commit/e6d03f67377d4412c7aa56a8e2e4988e6ec479dd"><code>e6d03f6</code></a>
Add a regression test for artifact name + content-type mismatches (<a
href="https://redirect.github.com/actions/download-artifact/issues/472">#472</a>)</li>
<li><a
href="https://github.com/actions/download-artifact/commit/70fc10c6e5e1ce46ad2ea6f2b72d43f7d47b13c3"><code>70fc10c</code></a>
Merge pull request <a
href="https://redirect.github.com/actions/download-artifact/issues/461">#461</a>
from actions/danwkennedy/digest-mismatch-behavior</li>
<li><a
href="https://github.com/actions/download-artifact/commit/f258da9a506b755b84a09a531814700b86ccfc62"><code>f258da9</code></a>
Add change docs</li>
<li><a
href="https://github.com/actions/download-artifact/commit/ccc058e5fbb0bb2352213eaec3491e117cbc4a5c"><code>ccc058e</code></a>
Fix linting issues</li>
<li>Additional commits viewable in <a
href="https://github.com/actions/download-artifact/compare/v4...v8">compare
view</a></li>
</ul>
</details>
<br />


Dependabot will resolve any conflicts with this PR as long as you don't
alter it yourself. You can also trigger a rebase manually by commenting
`@dependabot rebase`.

[//]: # (dependabot-automerge-start)
[//]: # (dependabot-automerge-end)

---

<details>
<summary>Dependabot commands and options</summary>
<br />

You can trigger Dependabot actions by commenting on this PR:
- `@dependabot rebase` will rebase this PR
- `@dependabot recreate` will recreate this PR, overwriting any edits
that have been made to it
- `@dependabot show <dependency name> ignore conditions` will show all
of the ignore conditions of the specified dependency
- `@dependabot ignore <dependency name> major version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's major version (unless you unignore this specific
dependency's major version or upgrade to it yourself)
- `@dependabot ignore <dependency name> minor version` will close this
group update PR and stop Dependabot creating any more for the specific
dependency's minor version (unless you unignore this specific
dependency's minor version or upgrade to it yourself)
- `@dependabot ignore <dependency name>` will close this group update PR
and stop Dependabot creating any more for the specific dependency
(unless you unignore this specific dependency or upgrade to it yourself)
- `@dependabot unignore <dependency name>` will remove all of the ignore
conditions of the specified dependency
- `@dependabot unignore <dependency name> <ignore condition>` will
remove the ignore condition of the specified dependency and ignore
conditions


</details>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-11-06 21:31:01 -05:00
Dylan M. Taylor 4bf1b20a75 ci: add signed Linux release matrix, packaging lint, and FreeBSD CI (#2758)
Adds the complete signed Linux release matrix, FreeBSD CI, and packaging
lint to ArtCraft Studio, aligned with the Craft suite standards in
`craft-common-actions`.

| Architecture | Formats |

|:-------------|:------------------------------------------------------------------------|
| x86_64 | AppImage and zsync, DEB, RPM, tar.gz, Arch package, Flatpak
bundle, OCI |
| aarch64 | AppImage and zsync, DEB, RPM, tar.gz, Arch package, Flatpak
bundle, OCI |
| riscv64 | DEB, RPM, tar.gz |

### Changes

- **Linux Packaging Matrix**: Builds and packages ArtCraft Studio across
x86_64, aarch64, and riscv64 using `storytold/craft-common-actions`.
Produces DEB, RPM, AppImage, tarball, Arch package, and Flatpak
bundle/OCI artifacts. Cross-compiles riscv64 DEB and RPM via BoringSSL
CMake configuration. Signed Linux artifacts join existing
`artcraft-v<version>` release drafts while retaining macOS and Windows
assets (storytold/craft-common-actions#160 and #161).
- **Release Trigger Alignment**: Like macOS and Windows release
workflows, Linux release builds run on release branches (`release`,
`artcraft-release`, `artcraft-linux`) and `workflow_dispatch` rather
than on PRs.
- **Documentation**: Updated `README.md` to include Linux under download
options and reference `packaging/linux/README.md`. Added
`packaging/windows/README.md` placeholder.
- **Packaging Lint**: Added `.github/workflows/packaging-lint.yml`
calling `craft-common-actions/packaging-lint.yml@main`. Fixed
pre-existing actionlint and shellcheck issues in macOS and Windows
publish workflows (`shell: pwsh` defaults and variable quoting).
- **FreeBSD CI**: Added `.github/workflows/freebsd.yml` calling
`craft-common-actions/freebsd.yml@main` on scheduled nightly runs,
release branches, and manual dispatch (not on pull requests).
- **SQLx Build Fallback**: Updated
`crates/schema/database/sqlite_tasks/build.rs` to fall back to
`SQLX_OFFLINE=true` using committed `.sqlx/` query data when no live
`/tmp/tasks.sqlite` database file exists, fixing nextest coverage
archives under CI.

### Validation

- `actionlint` passes cleanly across all `.github/workflows/*.yml`
files.
- `shellcheck` passes across all scripts in `packaging/linux/*.sh`.
- `xmllint` validates `packaging/linux/ai.artcraft.app.metainfo.xml`.
- `cargo check -p sqlite_tasks`, `cargo test -p sqlite_tasks`, and
`cargo test --no-run -p sqlite_tasks` pass locally without database
environment variables.
- Packaging lint passes 100% green on CI.
- SonarCloud / nextest instrumented test archive build passes on CI.
- Linux package scripts (`package.sh`, `package-test.sh`, `flatpak.sh`)
verified locally and on CI runners.
2026-11-06 20:02:06 -05:00
FredApps dc6006f625 Copy the shared house rules into AGENTS.md (#2749)
## Summary
- Puts the shared house-rules blocks at the top of `AGENTS.md`, copied
from craft-contrib `dde09ea31e2e457f5b4653a6e76f7cdf55944bdf`.
- Adds `.github/workflows/house-rules.yml`, which calls the shared
check.

Artcraft is the Tauri product, so it gets the core block only. The Rust
rules are for native Craft apps and shared Rust libraries.
- `CLAUDE.md` was a symlink to `AGENTS.md` and is now `@AGENTS.md`.
- No local paragraph was removed. The Tauri and TypeScript description
stays.

## Sizes
- Core block: 3,052 bytes.
- Root `AGENTS.md`: 6,934 bytes.
2026-11-06 18:44:42 +01:00
Dylan Taylor 58f5944fe6 fix: align rfd backend with GTK dialog feature 2026-11-04 10:09:58 -05:00
Dylan M. Taylor cfdbbd1a3f ci: allow manual SonarCloud scans 2026-11-04 09:51:32 -05:00
Dylan M. Taylor 1f969a9224 ci(sonar): stop cancelling main-branch runs so coverage is published (#2574)
SonarCloud coverage upload began on 2026-10-23
(craft-common-actions#74), but `cancel-in-progress: true` cancels
superseded runs on `main` before the llvm-cov build finishes, so
coverage may never be published. Cancel superseded runs only on PR refs;
main runs queue and complete. Same change as storytold/vectorcraft#3881.
2026-10-23 17:01:44 -04:00
Dylan M. Taylor 800217ed04 ci: add sonar-project.properties for CI-based analysis
Marks the project ineligible for SonarCloud Automatic Analysis, resolving the
CI-vs-automatic-analysis conflict so the SonarCloud workflow succeeds.
2026-10-21 16:00:53 -04:00
Dylan M. Taylor 1790f9d562 ci: add SonarCloud scan via shared reusable workflow
Calls storytold/craft-common-actions/.github/workflows/sonarcloud.yml@main
with secrets: inherit. Adds SonarCloud badge to README where missing and
removes the redundant sonar-project.properties (config now passed by the
reusable workflow).
2026-10-21 15:47:11 -04:00
Brandon ThomasandClaude Opus 5.5 40e04793b5 Skip CodeRabbit reviews on Dependabot PRs (#2510)
Adds `.coderabbit.yaml` with `reviews.auto_review.ignore_usernames:
["dependabot[bot]"]` so CodeRabbit stops reviewing and commenting on
Dependabot PRs. Config-only change; same file is landing across the
craft repos.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-20 11:40:52 -04:00
Dylan M. Taylor 7a2935cf9e ci: configure Dependabot for Cargo and GitHub Actions (#2460)
Configures Dependabot to keep Rust / Cargo dependencies and GitHub
Actions workflows up to date.

- Schedules daily dependency checks at 06:00 America/New_York
- Groups Cargo updates into a single consolidated PR with `chore(deps)`
prefix
- Groups GitHub Actions updates into a single consolidated PR with
`ci(deps)` prefix
2026-10-18 15:10:55 -04:00
BFlat d4e4ac69bb artcraft-native: generate with Midjourney or fal (provider choice) (#2115)
Images can run on Midjourney (own account, via a sign-in window) or fal (own API key), chosen from the model selector's Select Provider submenu. Jobs finish into the library. The Midjourney credential manager and the router prompt helper move into shared crates used by both desktop apps.
2026-10-11 20:47:59 +07:00
BFlat 28a39a0de4 artcraft-native: Create Audio, in-app playback, animated previews and @Characters (#2079)
Create Audio (PromptBoxAudio) with waveform playback, in-app video playback, animated feed previews, and @Character mentions with the Characters dialog.
2026-10-11 02:03:53 +07:00
310 changed files with 9217 additions and 2325 deletions
+6
View File
@@ -0,0 +1,6 @@
# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
reviews:
auto_review:
# Dependabot bumps are checked by CI; skip CodeRabbit reviews on them.
ignore_usernames:
- "dependabot[bot]"
+27 -8
View File
@@ -1,11 +1,30 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/github/administering-a-repository/configuration-options-for-dependency-updates
version: 2
updates:
- package-ecosystem: "cargo" # See documentation for possible values
directory: "/" # Location of package manifests
# Maintain Rust / Cargo dependencies
- package-ecosystem: "cargo"
directory: "/"
schedule:
interval: "weekly"
interval: "daily"
time: "06:00"
timezone: "America/New_York"
# Consolidate updates into a single PR rather than one PR per crate
groups:
cargo-dependencies:
patterns:
- "*"
commit-message:
prefix: "chore(deps)"
# Maintain GitHub Actions workflows
- package-ecosystem: "github-actions"
directory: "/"
schedule:
interval: "daily"
time: "06:00"
timezone: "America/New_York"
groups:
actions:
patterns:
- "*"
commit-message:
prefix: "ci(deps)"
+93
View File
@@ -0,0 +1,93 @@
# This file is machine-generated by `gh actions-lock`.
# Do not edit by hand; run `gh actions-lock` to update.
# Docs: https://gh.io/actions-lockfile
version: 'v0.0.3'
workflows:
'.github/workflows/artcraft-linux-publish.yml':
- 'actions/checkout@v7.0.1'
- 'actions/download-artifact@v8.0.2'
- 'actions/setup-node@v7.1.0'
- 'actions/upload-artifact@v7.0.2'
- 'step-security/dtolnay-rust-toolchain@v1.0.0'
- 'step-security/harden-runner@v2.22.4'
- 'step-security/rust-cache@v2.9.2'
- 'step-security/tauri-action@v1.0.0'
- 'storytold/craft-common-actions@main'
'.github/workflows/artcraft-macos-publish.yml':
- 'actions/checkout@v7.0.1'
- 'actions/setup-node@v7.1.0'
- 'step-security/dtolnay-rust-toolchain@v1.0.0'
- 'step-security/harden-runner@v2.22.4'
- 'step-security/tauri-action@v1.0.0'
'.github/workflows/artcraft-windows-publish.yml':
- 'actions/checkout@v7.0.1'
- 'actions/setup-node@v7.1.0'
- 'step-security/dtolnay-rust-toolchain@v1.0.0'
- 'step-security/harden-runner@v2.22.4'
- 'step-security/tauri-action@v1.0.0'
'.github/workflows/freebsd.yml': []
'.github/workflows/house-rules.yml': []
'.github/workflows/lockfile.yml':
- 'actions/checkout@v7.0.1'
- 'step-security/harden-runner@v2.22.4'
'.github/workflows/packaging-lint.yml': []
'.github/workflows/sonarcloud.yml': []
dependencies:
'actions/cache@caa296126883cff596d87d8935842f9db880ef25':
ref: 'v5.1.0'
commit: 'sha1-caa296126883cff596d87d8935842f9db880ef25'
owner_id: 44036562
repo_id: 215566462
'actions/checkout@v7.0.1':
ref: 'v7.0.1'
commit: 'sha1-3d3c42e5aac5ba805825da76410c181273ba90b1'
owner_id: 44036562
repo_id: 197814629
'actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333':
ref: 'v8.0.2'
commit: 'sha1-9000827ccba6bdab643e8b6fd33ac0654aef8333'
owner_id: 44036562
repo_id: 192626254
'actions/download-artifact@v8.0.2':
ref: 'v8.0.2'
commit: 'sha1-9000827ccba6bdab643e8b6fd33ac0654aef8333'
owner_id: 44036562
repo_id: 192626254
'actions/setup-node@v7.1.0':
ref: 'v7.1.0'
commit: 'sha1-949feb2413d6458794dcd2491c4babbbce0c15c1'
owner_id: 44036562
repo_id: 189476904
'actions/upload-artifact@v7.0.2':
ref: 'v7.0.2'
commit: 'sha1-cf430e030ddbb5b0abf93d22962f4752f3646cd9'
owner_id: 44036562
repo_id: 192625955
'step-security/dtolnay-rust-toolchain@v1.0.0':
ref: 'v1.0.0'
commit: 'sha1-8dd607977d06adf4ed49b190381e74c17a9dd25f'
owner_id: 88700172
repo_id: 1249865941
'step-security/harden-runner@v2.22.4':
ref: 'v2.22.4'
commit: 'sha1-1d88384fd173af6ecb012dfd0264a306b711241a'
owner_id: 88700172
repo_id: 422287306
'step-security/rust-cache@v2.9.2':
ref: 'v2.9.2'
commit: 'sha1-8871c8ba063416da7c32246932ea6ae08700453e'
owner_id: 88700172
repo_id: 822636713
'step-security/tauri-action@v1.0.0':
ref: 'v1.0.0'
commit: 'sha1-dc662d498edbd35ec8ed564b45d44e955444d1bc'
owner_id: 88700172
repo_id: 1283860856
'storytold/craft-common-actions@main':
ref: 'main'
commit: 'sha1-293de72429d3ed5d9fa92aded5e500544c0fa28f'
owner_id: 76897702
repo_id: 1414429108
uses:
- 'actions/cache@caa296126883cff596d87d8935842f9db880ef25'
- 'actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333'
@@ -0,0 +1,215 @@
name: ArtCraft Linux release
on:
push:
branches: [release, artcraft-release, artcraft-linux]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: linux-release-${{ github.ref }}
cancel-in-progress: false
jobs:
packaging-check:
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: rpm jq binutils zsync
- run: packaging/linux/package-test.sh
build:
strategy:
fail-fast: false
matrix:
include:
- {arch: x86_64, runner: ubuntu-22.04, target: x86_64-unknown-linux-gnu, bundles: "deb,rpm,appimage"}
- {arch: aarch64, runner: ubuntu-22.04-arm, target: aarch64-unknown-linux-gnu, bundles: "deb,rpm,appimage"}
- {arch: riscv64, runner: ubuntu-24.04, target: riscv64gc-unknown-linux-gnu, bundles: "deb,rpm"}
runs-on: ${{ matrix.runner }}
timeout-minutes: 90
env:
SQLX_OFFLINE: 'true'
VITE_ENVIRONMENT_TYPE: production
TAURI_FRONTEND_PATH: ${{ github.workspace }}/frontend
TAURI_APP_PATH: ${{ github.workspace }}/crates/desktop/artcraft
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf libssl-dev libclang-dev libasound2-dev libxdo-dev libdbus-1-dev build-essential clang file jq perl cmake nasm rpm zsync gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-libav
- uses: storytold/craft-common-actions/actions/install-riscv64-cross@main
if: matrix.arch == 'riscv64'
- name: Remove conflicting host Pango development files
if: matrix.arch == 'riscv64'
# Ubuntu's Pango GIR differs by architecture despite being in a shared path.
# Only the target GTK development packages are needed for this cross build.
run: sudo apt-get remove -y libpango1.0-dev:amd64
- name: Install RISC-V WebKitGTK dependencies
if: matrix.arch == 'riscv64'
uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: libwebkit2gtk-4.1-dev:riscv64 libappindicator3-dev:riscv64 libssl-dev:riscv64 libxdo-dev:riscv64 libdbus-1-dev:riscv64 g++-riscv64-linux-gnu
- name: Configure RISC-V cross compilation
if: matrix.arch == 'riscv64'
run: |
cat >> "$GITHUB_ENV" <<'ENV'
CARGO_TARGET_RISCV64GC_UNKNOWN_LINUX_GNU_LINKER=riscv64-linux-gnu-gcc
CC_riscv64gc_unknown_linux_gnu=riscv64-linux-gnu-gcc
CXX_riscv64gc_unknown_linux_gnu=riscv64-linux-gnu-g++
AR_riscv64gc_unknown_linux_gnu=riscv64-linux-gnu-ar
PKG_CONFIG_ALLOW_CROSS=1
PKG_CONFIG_PATH_riscv64gc_unknown_linux_gnu=/usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig
PKG_CONFIG_LIBDIR_riscv64gc_unknown_linux_gnu=/usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig
ENV
echo "CMAKE_TOOLCHAIN_FILE_riscv64gc_unknown_linux_gnu=$GITHUB_WORKSPACE/packaging/linux/riscv64.cmake" >> "$GITHUB_ENV"
- uses: actions/setup-node@v7.1.0
with:
node-version: '22'
cache: npm
cache-dependency-path: frontend/package-lock.json
- uses: step-security/dtolnay-rust-toolchain@v1.0.0
with:
toolchain: stable
targets: ${{ matrix.target }}
- uses: step-security/rust-cache@v2.9.2
with:
key: tauri-linux-${{ matrix.arch }}
cache-on-failure: true
- run: npm ci --ignore-scripts
working-directory: frontend
- run: ./node_modules/.bin/nx run artcraft:build
working-directory: frontend
- uses: step-security/tauri-action@v1.0.0
with:
projectPath: crates/desktop/artcraft
args: --target ${{ matrix.target }} --bundles ${{ matrix.bundles }} --config ${{ github.workspace }}/packaging/linux/tauri.conf.json
- name: Package and verify Linux outputs
env:
ARCH: ${{ matrix.arch }}
TARGET: ${{ matrix.target }}
run: packaging/linux/package.sh "$ARCH" "$TARGET"
- uses: actions/upload-artifact@v7.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/release/*
if-no-files-found: error
arch:
if: always() && !cancelled()
needs: build
runs-on: ubuntu-24.04
strategy:
matrix:
arch: [x86_64, aarch64]
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- uses: actions/download-artifact@v8.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- name: Find tarball
id: tarball
env:
ARCH: ${{ matrix.arch }}
run: |
shopt -s nullglob
files=(dist/linux/artcraft-*-linux-"$ARCH".tar.gz)
test "${#files[@]}" -eq 1
echo "path=${files[0]}" >> "$GITHUB_OUTPUT"
- uses: storytold/craft-common-actions/actions/arch-package@main # extra-depends support (#161)
with:
tarball: ${{ steps.tarball.outputs.path }}
arch: ${{ matrix.arch }}
extra-depends: webkit2gtk-4.1 gtk3 openssl
- name: Verify Arch runtime dependencies
run: |
for package in dist/release/*.pkg.tar.zst; do
tar --zstd -xOf "$package" .PKGINFO | grep -Fx 'depend = webkit2gtk-4.1'
done
- uses: actions/upload-artifact@v7.0.2
with:
name: release-arch-${{ matrix.arch }}
path: dist/release/*.pkg.tar.zst
if-no-files-found: error
flatpak:
if: always() && !cancelled()
needs: build
runs-on: ${{ matrix.runner }}
permissions:
contents: read
packages: write
strategy:
fail-fast: false
matrix:
include:
- {arch: x86_64, runner: ubuntu-24.04}
- {arch: aarch64, runner: ubuntu-24.04-arm}
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- uses: actions/download-artifact@v8.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
- uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: flatpak flatpak-builder elfutils librsvg2-common skopeo
- name: Allow Flatpak build sandboxes
run: sudo sysctl -w kernel.apparmor_restrict_unprivileged_userns=0
- uses: storytold/craft-common-actions/actions/import-signing-key@main
with:
key: ${{ secrets.LINUX_SIGNING_KEY }}
- name: Build Flatpak and publish OCI
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: packaging/linux/flatpak.sh
- uses: actions/upload-artifact@v7.0.2
with:
name: release-flatpak-${{ matrix.arch }}
path: |
dist/release/*.flatpak
dist/release/*.oci-manifest.json
if-no-files-found: error
release:
needs: [packaging-check, build, arch, flatpak]
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- name: Read version
id: version
run: echo "value=$(jq -r .version crates/desktop/artcraft/tauri.conf.json)" >> "$GITHUB_OUTPUT"
- name: Require signing key
env:
SIGNING_KEY: ${{ secrets.LINUX_SIGNING_KEY }}
run: test -n "$SIGNING_KEY"
- uses: storytold/craft-common-actions/actions/draft-release@main
with:
version: ${{ steps.version.outputs.value }}
app-title: ArtCraft
tag-prefix: artcraft-v
retain: "true"
github-token: ${{ secrets.GITHUB_TOKEN }}
signing-key: ${{ secrets.LINUX_SIGNING_KEY }}
+31 -16
View File
@@ -10,6 +10,13 @@ on:
# This workflow creates Universal macOS builds with signing certificate support.
# On each push to the release branches it will create or update a GitHub release, build your app, and upload the artifacts to the release.
# One release build per branch at a time. A run in progress is never cancelled, because it may
# already have created or uploaded to the draft release. If more pushes arrive while one runs,
# GitHub keeps only the newest as pending, so the next build is of the latest commit.
concurrency:
group: macos-release-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-artcraft:
permissions:
@@ -28,17 +35,24 @@ jobs:
# args: ''
runs-on: ${{ matrix.platform }}
# Successful builds took 10-22 min (2026-04 to 2026-11).
timeout-minutes: 60
steps:
- uses: actions/checkout@v4
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- name: setup node
uses: actions/setup-node@v4
uses: actions/setup-node@v7.1.0
with:
node-version: lts/*
- name: install Rust stable
uses: dtolnay/rust-toolchain@stable
uses: step-security/dtolnay-rust-toolchain@v1.0.0
with:
toolchain: stable
targets: ${{ matrix.platform == 'macos-latest-xlarge' && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
- name: install dependencies (ubuntu only)
@@ -63,7 +77,7 @@ jobs:
npm cache clean --force
echo "Installing all workspace dependencies..."
npm ci
npm ci --ignore-scripts
echo "=== Version Information ==="
echo "Node version:"
@@ -84,7 +98,7 @@ jobs:
echo "Starting certificate import process..."
# Create certificate file
echo $APPLE_CERTIFICATE | base64 --decode > certificate.p12
echo "$APPLE_CERTIFICATE" | base64 --decode > certificate.p12
# Debug: Check if certificate file exists and has content
echo "Certificate file info:"
@@ -93,25 +107,25 @@ jobs:
# Debug: Check certificate contents without password (just structure)
echo "Certificate structure (may show error if password protected):"
openssl pkcs12 -info -in certificate.p12 -passin pass:$APPLE_CERTIFICATE_PASSWORD -noout || echo "Could not read certificate structure"
openssl pkcs12 -info -in certificate.p12 -passin pass:"$APPLE_CERTIFICATE_PASSWORD" -noout || echo "Could not read certificate structure"
# Debug: Try to list what's in the p12 file
echo "Attempting to list certificate contents:"
security cms -D -i certificate.p12 || echo "Not a CMS file, trying PKCS12..."
openssl pkcs12 -in certificate.p12 -passin pass:$APPLE_CERTIFICATE_PASSWORD -info -nodes 2>&1 | head -20 || echo "Failed to read PKCS12 contents"
openssl pkcs12 -in certificate.p12 -passin pass:"$APPLE_CERTIFICATE_PASSWORD" -info -nodes 2>&1 | head -20 || echo "Failed to read PKCS12 contents"
# Debug: Extract certificate details to verify type and expiration
echo "Extracting certificate details..."
# Extract just the certificate portion and examine it
openssl pkcs12 -in certificate.p12 -passin pass:$APPLE_CERTIFICATE_PASSWORD -nokeys 2>/dev/null | openssl x509 -text -noout | grep -E "(Subject:|Issuer:|Not Before|Not After)" || echo "Failed to parse certificate details"
openssl pkcs12 -in certificate.p12 -passin pass:"$APPLE_CERTIFICATE_PASSWORD" -nokeys 2>/dev/null | openssl x509 -text -noout | grep -E "(Subject:|Issuer:|Not Before|Not After)" || echo "Failed to parse certificate details"
# Also try to see what type of certificate this is
echo "Certificate type check:"
openssl pkcs12 -in certificate.p12 -passin pass:$APPLE_CERTIFICATE_PASSWORD -nokeys 2>/dev/null | openssl x509 -text -noout | grep -E "(CN=|OU=)" || echo "Could not determine certificate type"
openssl pkcs12 -in certificate.p12 -passin pass:"$APPLE_CERTIFICATE_PASSWORD" -nokeys 2>/dev/null | openssl x509 -text -noout | grep -E "(CN=|OU=)" || echo "Could not determine certificate type"
# Debug: Check if private key exists
echo "Checking for private key..."
openssl pkcs12 -in certificate.p12 -passin pass:$APPLE_CERTIFICATE_PASSWORD -nocerts -nodes 2>&1 | grep -E "(PRIVATE KEY|RSA KEY)" | head -5 || echo "No private key found"
openssl pkcs12 -in certificate.p12 -passin pass:"$APPLE_CERTIFICATE_PASSWORD" -nocerts -nodes 2>&1 | grep -E "(PRIVATE KEY|RSA KEY)" | head -5 || echo "No private key found"
# Create temporary keychain
security create-keychain -p "$KEYCHAIN_PASSWORD" build.keychain
@@ -139,6 +153,7 @@ jobs:
security unlock-keychain -p "$KEYCHAIN_PASSWORD" build.keychain
# Add the keychain to the search list
# shellcheck disable=SC2046
security list-keychains -d user -s build.keychain $(security list-keychains -d user | sed 's/^[[:space:]]*"//;s/"$//')
# List all certificates to debug
@@ -181,11 +196,11 @@ jobs:
security find-certificate -c "Learning Machines" build.keychain -p | openssl x509 -text -noout | grep -E "(Subject:|CN=)" || echo "Could not parse certificate"
# Set the correct signing identity
echo "APPLE_SIGNING_IDENTITY=Developer ID Application: Learning Machines LLC (DJ6XS33FX8)" >> $GITHUB_ENV
echo "APPLE_SIGNING_IDENTITY=Developer ID Application: Learning Machines LLC (DJ6XS33FX8)" >> "$GITHUB_ENV"
echo "Proceeding with identity: Developer ID Application: Learning Machines LLC (DJ6XS33FX8)"
# Also set the team ID
echo "APPLE_DEVELOPMENT_TEAM=DJ6XS33FX8" >> $GITHUB_ENV
echo "APPLE_DEVELOPMENT_TEAM=DJ6XS33FX8" >> "$GITHUB_ENV"
- name: Pre-build frontend to avoid Tauri build issues
run: |
@@ -195,11 +210,11 @@ jobs:
# Increase Node.js memory limit for the build
export NODE_OPTIONS="--max-old-space-size=8192"
# Use npx to ensure we use the local nx version
echo "Building with npx nx..."
npx nx run artcraft:build
echo "Building with nx..."
./node_modules/.bin/nx run artcraft:build
echo "Checking build output..."
# shellcheck disable=SC2012
ls -la apps/artcraft/dist/ | head -10
# Update Tauri config to skip the build (since we already built)
@@ -215,7 +230,7 @@ jobs:
- name: build and publish
uses: tauri-apps/tauri-action@v0
uses: step-security/tauri-action@v1.0.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
APPLE_ID: ${{ secrets.APPLE_ID }}
+22 -5
View File
@@ -10,6 +10,13 @@ on:
# This workflow creates Windows builds.
# On each push to the release branches it will create or update a GitHub release, build your app, and upload the artifacts to the release.
# One release build per branch at a time. A run in progress is never cancelled, because it may
# already have created or uploaded to the draft release. If more pushes arrive while one runs,
# GitHub keeps only the newest as pending, so the next build is of the latest commit.
concurrency:
group: windows-release-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-artcraft-windows:
permissions:
@@ -23,11 +30,19 @@ jobs:
args: ''
runs-on: ${{ matrix.platform }}
# Successful builds took 17-36 min (2026-04 to 2026-11); one hung run used the full 6-hour default.
timeout-minutes: 90
steps:
- uses: actions/checkout@v4
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
- name: install Rust stable
uses: dtolnay/rust-toolchain@stable
uses: step-security/dtolnay-rust-toolchain@v1.0.0
with:
toolchain: stable
- name: install wreq dependencies
run: |
@@ -39,11 +54,12 @@ jobs:
shell: cmd
- name: setup node
uses: actions/setup-node@v4
uses: actions/setup-node@v7.1.0
with:
node-version: lts/*
- name: install frontend dependencies
shell: pwsh
run: |
# Navigate to the frontend directory to install dependencies
cd ./frontend
@@ -51,7 +67,7 @@ jobs:
npm cache clean --force
echo "Installing all workspace dependencies..."
npm ci
npm ci --ignore-scripts
echo "=== Version Information ==="
echo "Node version:"
@@ -63,6 +79,7 @@ jobs:
npm list --depth=0 | findstr nx
- name: Pre-build frontend to avoid Tauri build issues
shell: pwsh
run: |
# Build the frontend app from the correct directory
cd ./frontend
@@ -90,7 +107,7 @@ jobs:
$config | ConvertTo-Json -Depth 10 | Set-Content tauri.conf.json
- name: build and publish
uses: tauri-apps/tauri-action@v0
uses: step-security/tauri-action@v1.0.0
env:
# NB(bt): This is needed by wreq. Hopefully it doesn't kill Tauri
RUSTFLAGS: "-C target-feature=+crt-static"
+24
View File
@@ -0,0 +1,24 @@
# Builds and tests the workspace on FreeBSD, inside a FreeBSD VM on a Linux runner
# (GitHub has no FreeBSD runners).
name: FreeBSD
# The VM build is slow, so it runs nightly on main and on pushes to `release`, not on pull requests.
on:
schedule:
- cron: "27 4 * * *"
push:
branches: [release]
workflow_dispatch:
concurrency:
group: freebsd-${{ github.ref }}
cancel-in-progress: true
permissions:
contents: read
jobs:
freebsd:
# Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos.
uses: storytold/craft-common-actions/.github/workflows/freebsd.yml@main
+28
View File
@@ -0,0 +1,28 @@
name: House rules
on:
push:
branches: [main]
paths:
- AGENTS.md
- CLAUDE.md
- .github/workflows/house-rules.yml
pull_request:
paths:
- AGENTS.md
- CLAUDE.md
- .github/workflows/house-rules.yml
workflow_dispatch:
permissions:
contents: read
concurrency:
group: house-rules-${{ github.ref }}
cancel-in-progress: true
jobs:
house-rules:
uses: storytold/craft-common-actions/.github/workflows/house-rules.yml@main
with:
rust: false
+34
View File
@@ -0,0 +1,34 @@
name: Lockfile
# Fails when .github/workflows/actions.lock is out of sync with the workflows, or when a locked pin
# no longer verifies upstream, so a moved tag cannot slip a different commit into a run.
# Docs: https://gh.io/actions-lockfile
on:
pull_request:
push:
branches: [main]
workflow_dispatch:
permissions:
contents: read
env:
GH_TOKEN: ${{ github.token }}
jobs:
lockfile:
name: Lockfile is in sync
runs-on: ubuntu-24.04
timeout-minutes: 10
steps:
- name: Harden runner
uses: step-security/harden-runner@v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@v7.0.1
with:
persist-credentials: false
- name: Install gh-actions-lock
run: gh extension install github/gh-actions-lock --pin v0.4.1
- name: Verify the lockfile
run: gh actions-lock --verify
+23
View File
@@ -0,0 +1,23 @@
# Quick static checks for the release pipeline (seconds, no Rust build): workflow syntax,
# packaging shell/PowerShell scripts, and the Linux desktop/AppStream/Flatpak metadata.
name: Packaging lint
on:
push:
branches: [main, release]
paths: [".github/workflows/**", "packaging/**"]
pull_request:
paths: [".github/workflows/**", "packaging/**"]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: packaging-lint-${{ github.ref }}
cancel-in-progress: true
jobs:
lint:
# Intentionally tracked at @main (not SHA-pinned) so org-wide CI improvements, security fixes, and build optimizations propagate without PR churn across all craftapp repos.
uses: storytold/craft-common-actions/.github/workflows/packaging-lint.yml@main
+22
View File
@@ -0,0 +1,22 @@
name: SonarCloud
on:
push:
branches: [main]
pull_request:
types: [opened, synchronize, reopened]
workflow_dispatch:
permissions:
contents: read
concurrency:
group: sonarcloud-${{ github.ref }}
# A push to main is the run that publishes coverage; superseded runs there would never finish.
cancel-in-progress: ${{ github.ref != 'refs/heads/main' }}
jobs:
sonarcloud:
# Tracked at @main so org-wide CI improvements propagate without PR churn.
uses: storytold/craft-common-actions/.github/workflows/sonarcloud.yml@main
secrets: inherit
+22
View File
@@ -1,3 +1,25 @@
<!-- craft-house-rules:start core source=dde09ea31e2e457f5b4653a6e76f7cdf55944bdf sha256=540e6cea75d12d0710c3dca89658328e81e508ecef102446ca5bb95b7c0b92bb -->
## House rules
These apply in every Craft repository. A later rule may be stricter, not weaker. The source is `standards/house-rules-core.md` in `storytold/craft-contrib`. Propose changes there, not in a copy.
1. **A person is accountable.** Whoever runs an agent answers for what it opens and for its review comments.
2. **Check, then claim.** Before writing code, read the issue comments, search pull requests for `#N` and the keywords, and check recent commits on `main`. Comment on the issue first. Release a claim you will not finish.
3. **Verify, don't guess.** Open a definition before using it. Confirm the cause in the code, even when someone names a location.
4. **Keep fixes narrow and tested.** Fix what the issue describes, where the bug is. Add a test that fails without the fix.
5. **Review the diff before opening a pull request.** Identifiers exist as used, the test fails without the fix, nothing unrelated changed, and the description is true.
6. **Look for a better pull request.** If another fixes the same thing, close yours when it is better or equal, and say why.
7. **Report only what ran.** Say what you tested and what you could not run. Without a local build or test, say so and rely on CI before marking ready.
8. **`main` changes through a pull request.** Do not push to it. Branch protection enforces this. There is no exception.
9. **Stop and ask only about** licensing outside the allowlist, publishing, releases, a new remote, secrets, spending money, destructive git, and anything else hard to undo. Otherwise keep working. `plan/` is a gitignored notebook, not rules. Without it, do not ask for it and do not stop.
10. **Keep secrets and personal details out** of the repository, logs and requests.
11. **CI stays shared.** Workflows call `storytold/craft-common-actions`. Do not change CI in a feature or fix pull request. If CI cannot cover a change, open an issue there.
12. **Guards say why.** A check that looks removable says what it prevents. Read that before changing it.
13. **Instructions must be readable by everyone.** No notes about one machine, and no links to files a reader cannot open. `craftrules` is retired. Do not link to it.
14. **Keep pull requests moving.** Concurrent pull requests are fine; unanswered actionable feedback is not. Respond in 7 days, a reminder at 14, closure eligible at 30, from when an actionable request waits on the author. Suggestions and unconfirmed bot findings do not start the clock. A fix, explanation or agreed plan resets it; unrelated activity does not. Waiting on a reviewer or maintainer pauses it and does not count against the author. A maintainer decides closure; it is never automatic. Reuse search results, do not poll, and stop at a rate limit.
15. **No agent attribution.** A pull request does not say an agent wrote it. A repository rule may not require that.
<!-- craft-house-rules:end -->
# AGENTS.md
This file provides guidance for coding agents when working with the Artcraft monorepo.
-1
View File
@@ -1 +0,0 @@
AGENTS.md
+1
View File
@@ -0,0 +1 @@
@AGENTS.md
Generated
+511 -75
View File
@@ -184,6 +184,28 @@ version = "0.2.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c6cb57a04249c6480766f7f7cef5467412af1490f8d1e243141daddada3264f"
[[package]]
name = "alsa"
version = "0.9.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed7572b7ba83a31e20d1b48970ee402d2e3e0537dcfe0a3ff4d6eb7508617d43"
dependencies = [
"alsa-sys",
"bitflags 2.13.2",
"cfg-if",
"libc",
]
[[package]]
name = "alsa-sys"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db8fee663d06c4e303404ef5f40488a53e062f89ba8bfed81f42325aafad1527"
dependencies = [
"libc",
"pkg-config",
]
[[package]]
name = "android-activity"
version = "0.6.1"
@@ -196,9 +218,9 @@ dependencies = [
"jni 0.22.4",
"libc",
"log",
"ndk",
"ndk 0.9.0",
"ndk-context",
"ndk-sys",
"ndk-sys 0.6.0+11769913",
"num_enum",
"thiserror 2.0.18",
]
@@ -282,7 +304,7 @@ checksum = "291e6a250ff86cd4a820112fb8898808a366d8f9f58ce16d1f538353ad55747d"
dependencies = [
"anstyle",
"once_cell_polyfill",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -361,6 +383,7 @@ dependencies = [
"chrono",
"cookie_store 0.0.1",
"cookie_store_wrapper",
"craft-update",
"directories",
"dirs",
"discord-rich-presence",
@@ -465,8 +488,10 @@ dependencies = [
"arboard",
"artcraft_api_defs",
"artcraft_client",
"artcraft_router",
"chrono",
"cookie",
"cookie_store_wrapper",
"directories",
"eframe",
"egui",
@@ -474,17 +499,23 @@ dependencies = [
"egui_kittest",
"enums",
"env_logger",
"fal_client",
"image",
"log",
"midjourney_client",
"open",
"reqwest 0.12.23",
"resvg",
"rfd",
"rodio",
"serde",
"serde_json",
"tempfile",
"tokens",
"tokio",
"url",
"uuid 1.22.0",
"wry",
]
[[package]]
@@ -523,28 +554,6 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "175571dd1d178ced59193a6fc02dde1b972eb0bc56c892cde9beeceac5bf0f6b"
[[package]]
name = "ashpd"
version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f3f79755c74fd155000314eb349864caa787c6592eace6c6882dad873d9c39"
dependencies = [
"async-fs",
"async-net",
"enumflags2",
"futures-channel",
"futures-util",
"rand 0.9.2",
"raw-window-handle",
"serde",
"serde_repr",
"url",
"wayland-backend",
"wayland-client",
"wayland-protocols",
"zbus",
]
[[package]]
name = "assertor"
version = "0.0.3"
@@ -608,17 +617,6 @@ dependencies = [
"slab",
]
[[package]]
name = "async-fs"
version = "2.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8034a681df4aed8b8edbd7fbe472401ecf009251c8b40556b304567052e294c5"
dependencies = [
"async-lock",
"blocking",
"futures-lite",
]
[[package]]
name = "async-io"
version = "2.6.0"
@@ -648,17 +646,6 @@ dependencies = [
"pin-project-lite",
]
[[package]]
name = "async-net"
version = "2.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b948000fad4873c1c9339d60f2623323a0cfd3816e5181033c6a5cb68b2accf7"
dependencies = [
"async-io",
"blocking",
"futures-lite",
]
[[package]]
name = "async-process"
version = "2.5.0"
@@ -852,6 +839,12 @@ version = "0.22.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72b3254f16251a8381aa12e40e3c4d2f0199f8c6508fbecb9d91f575e0fbb8c6"
[[package]]
name = "base64"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "base64ct"
version = "1.6.0"
@@ -1438,9 +1431,9 @@ dependencies = [
[[package]]
name = "chrono"
version = "0.4.44"
version = "0.4.45"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c673075a2e0e5f4a1dde27ce9dee1ea4558c7ffe648f576438a20ca1d2acc4b0"
checksum = "1aa79e62e7697b8e29b513a68abacf485adcd1fe8284a4316c5ae868e6633327"
dependencies = [
"iana-time-zone",
"js-sys",
@@ -1730,6 +1723,49 @@ dependencies = [
"libm",
]
[[package]]
name = "coreaudio-rs"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "321077172d79c662f64f5071a03120748d5bb652f5231570141be24cfcd2bace"
dependencies = [
"bitflags 1.3.2",
"core-foundation-sys",
"coreaudio-sys",
]
[[package]]
name = "coreaudio-sys"
version = "0.2.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9b4739a805a62757a83e5654fa3faabec0442666b263bb2287d5a8185bfd953"
dependencies = [
"bindgen",
]
[[package]]
name = "cpal"
version = "0.15.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "873dab07c8f743075e57f524c583985fbaf745602acbe916a01539364369a779"
dependencies = [
"alsa",
"core-foundation-sys",
"coreaudio-rs",
"dasp_sample",
"jni 0.21.1",
"js-sys",
"libc",
"mach2",
"ndk 0.8.0",
"ndk-context",
"oboe",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
"windows 0.54.0",
]
[[package]]
name = "cpufeatures"
version = "0.2.12"
@@ -1739,6 +1775,31 @@ dependencies = [
"libc",
]
[[package]]
name = "craft-update"
version = "0.1.0"
source = "git+https://github.com/storytold/craft-libs?rev=2a24c10817173d26d0ea4689acbbade36f91b5dc#2a24c10817173d26d0ea4689acbbade36f91b5dc"
dependencies = [
"chrono",
"dunce",
"flate2",
"getrandom 0.3.1",
"log",
"plist",
"rustix 1.1.5",
"semver",
"serde",
"serde_json",
"sha2 0.10.8",
"tar",
"tempfile",
"ureq",
"verifysign",
"winreg 0.55.0",
"winsafe",
"zip",
]
[[package]]
name = "crc"
version = "3.0.1"
@@ -1756,9 +1817,9 @@ checksum = "19d374276b40fb8bbdee95aef7c7fa6b5316ec764510eb64b8dd0e2ed0d7e7f5"
[[package]]
name = "crc32fast"
version = "1.4.2"
version = "1.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a97769d94ddab943e4510d138150169a2758b5ef3eb191a9ee688de3e23ef7b3"
checksum = "2b929dd2a135771f40afabd1073f85eed59c800b7c61901e5c254270497dad64"
dependencies = [
"cfg-if",
]
@@ -1930,6 +1991,12 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "dasp_sample"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c87e182de0887fd5361989c677c4e8f5000cd9491d6d563161a8f3a5519fc7f"
[[package]]
name = "data-encoding"
version = "2.5.0"
@@ -2057,7 +2124,7 @@ dependencies = [
"libc",
"option-ext",
"redox_users",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -2429,7 +2496,7 @@ dependencies = [
"rustc_version",
"toml 0.8.23",
"vswhom",
"winreg",
"winreg 0.52.0",
]
[[package]]
@@ -2619,7 +2686,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -2696,6 +2763,12 @@ dependencies = [
"zune-inflate",
]
[[package]]
name = "extended"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "af9673d8203fcb076b19dfd17e38b3d4ae9f44959416ea532ce72415a6020365"
[[package]]
name = "fal_client"
version = "0.0.1"
@@ -2777,6 +2850,16 @@ dependencies = [
"walkdir",
]
[[package]]
name = "filetime"
version = "0.2.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5c287a33c7f0a620c38e641e7f60827713987b3c0f26e8ddc9462cc69cf75759"
dependencies = [
"cfg-if",
"libc",
]
[[package]]
name = "find-msvc-tools"
version = "0.1.9"
@@ -4683,7 +4766,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fc2f4eb4bc735547cfed7c0a4922cbd04a4655978c09b54f1f7b228750664c34"
dependencies = [
"cfg-if",
"windows-targets 0.52.6",
"windows-targets 0.48.5",
]
[[package]]
@@ -4832,6 +4915,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c41e0c4fef86961ac6d6f8a82609f55f31b05e4fce149ac5710e439df7619ba4"
[[package]]
name = "mach2"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d640282b302c0bb0a2a8e0233ead9035e3bed871f0b7e81fe4a1ec829765db44"
dependencies = [
"libc",
]
[[package]]
name = "markup5ever"
version = "0.35.0"
@@ -4922,7 +5014,9 @@ name = "midjourney_client"
version = "0.0.1"
dependencies = [
"anyhow",
"base64 0.22.1",
"browser_emulation",
"chrono",
"ciborium",
"cloudflare_errors",
"cookie_store_wrapper",
@@ -4933,10 +5027,13 @@ dependencies = [
"log",
"serde",
"serde_json",
"tempfile",
"test_data",
"test_utils",
"tokio",
"toml 1.1.8+spec-1.1.0",
"url",
"uuid_utils",
"wreq",
"wreq-util",
]
@@ -5018,7 +5115,7 @@ dependencies = [
"png 0.18.1",
"serde",
"thiserror 2.0.18",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -5075,6 +5172,20 @@ dependencies = [
"tempfile",
]
[[package]]
name = "ndk"
version = "0.8.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2076a31b7010b17a38c01907c45b945e8f11495ee4dd588309718901b1f7a5b7"
dependencies = [
"bitflags 2.13.2",
"jni-sys 0.3.1",
"log",
"ndk-sys 0.5.0+25.2.9519653",
"num_enum",
"thiserror 1.0.63",
]
[[package]]
name = "ndk"
version = "0.9.0"
@@ -5084,7 +5195,7 @@ dependencies = [
"bitflags 2.13.2",
"jni-sys 0.3.1",
"log",
"ndk-sys",
"ndk-sys 0.6.0+11769913",
"num_enum",
"raw-window-handle",
"thiserror 1.0.63",
@@ -5096,6 +5207,15 @@ version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "27b02d87554356db9e9a873add8782d4ea6e3e58ea071a9adb9a2e8ddb884a8b"
[[package]]
name = "ndk-sys"
version = "0.5.0+25.2.9519653"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c196769dd60fd4f363e11d948139556a344e79d451aeb2fa2fd040738ef7691"
dependencies = [
"jni-sys 0.3.1",
]
[[package]]
name = "ndk-sys"
version = "0.6.0+11769913"
@@ -5642,6 +5762,29 @@ dependencies = [
"objc2-foundation 0.3.2",
]
[[package]]
name = "oboe"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8b61bebd49e5d43f5f8cc7ee2891c16e0f41ec7954d36bcb6c14c5e0de867fb"
dependencies = [
"jni 0.21.1",
"ndk 0.8.0",
"ndk-context",
"num-derive",
"num-traits",
"oboe-sys",
]
[[package]]
name = "oboe-sys"
version = "0.6.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c8bb09a4a2b1d668170cfe0a7d5bc103f8999fb316c98099b6a9939c9f2e79d"
dependencies = [
"cc",
]
[[package]]
name = "once_cell"
version = "1.21.4"
@@ -6127,12 +6270,6 @@ dependencies = [
"windows-sys 0.61.2",
]
[[package]]
name = "pollster"
version = "0.4.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2f3a9f18d041e6d0e102a0a46750538147e5e8992d3b4873aaafee2520b00ce3"
[[package]]
name = "polycool"
version = "0.4.0"
@@ -6843,7 +6980,6 @@ version = "0.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a15ad77d9e70a92437d8f74c35d99b4e4691128df018833e99f90bcd36152672"
dependencies = [
"ashpd",
"block2 0.6.2",
"dispatch2",
"glib-sys",
@@ -6855,9 +6991,7 @@ dependencies = [
"objc2-app-kit 0.3.2",
"objc2-core-foundation",
"objc2-foundation 0.3.2",
"pollster",
"raw-window-handle",
"urlencoding",
"wasm-bindgen",
"wasm-bindgen-futures",
"web-sys",
@@ -6917,6 +7051,16 @@ dependencies = [
"syn 1.0.109",
]
[[package]]
name = "rodio"
version = "0.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7ceb6607dd738c99bc8cb28eff249b7cd5c8ec88b9db96c0608c1480d140fb1"
dependencies = [
"cpal",
"symphonia",
]
[[package]]
name = "ron"
version = "0.12.2"
@@ -7017,7 +7161,7 @@ dependencies = [
"errno",
"libc",
"linux-raw-sys 0.12.1",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -7037,6 +7181,7 @@ version = "0.23.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c665f33d38cea657d9614f766881e4d510e0eda4239891eea56b4cadcf01801b"
dependencies = [
"log",
"once_cell",
"ring",
"rustls-pki-types",
@@ -7087,6 +7232,33 @@ dependencies = [
"zeroize",
]
[[package]]
name = "rustls-platform-verifier"
version = "0.7.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1167586491e2b18b8bfbb293e8180ec17c201c4f076d7cb3070ca964e7598f98"
dependencies = [
"core-foundation 0.10.1",
"core-foundation-sys",
"jni 0.22.4",
"log",
"once_cell",
"rustls 0.23.36",
"rustls-native-certs",
"rustls-platform-verifier-android",
"rustls-webpki 0.103.9",
"security-framework 3.7.0",
"security-framework-sys",
"webpki-root-certs",
"windows-sys 0.60.2",
]
[[package]]
name = "rustls-platform-verifier-android"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "eec689c0bc40ff2458a5977b6619cb718087084a18e02a131c599b62d05e1a5f"
[[package]]
name = "rustls-webpki"
version = "0.101.7"
@@ -7920,6 +8092,7 @@ dependencies = [
"sqlx",
"tempfile",
"tokens",
"tokio",
]
[[package]]
@@ -8258,6 +8431,151 @@ dependencies = [
"serde_json",
]
[[package]]
name = "symphonia"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5773a4c030a19d9bfaa090f49746ff35c75dfddfa700df7a5939d5e076a57039"
dependencies = [
"lazy_static",
"symphonia-bundle-flac",
"symphonia-bundle-mp3",
"symphonia-codec-aac",
"symphonia-codec-adpcm",
"symphonia-codec-pcm",
"symphonia-codec-vorbis",
"symphonia-core",
"symphonia-format-isomp4",
"symphonia-format-riff",
"symphonia-metadata",
]
[[package]]
name = "symphonia-bundle-flac"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c91565e180aea25d9b80a910c546802526ffd0072d0b8974e3ebe59b686c9976"
dependencies = [
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-bundle-mp3"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4872dd6bb56bf5eac799e3e957aa1981086c3e613b27e0ac23b176054f7c57ed"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-codec-aac"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4c263845aa86881416849c1729a54c7f55164f8b96111dba59de46849e73a790"
dependencies = [
"lazy_static",
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-adpcm"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2dddc50e2bbea4cfe027441eece77c46b9f319748605ab8f3443350129ddd07f"
dependencies = [
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-pcm"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e89d716c01541ad3ebe7c91ce4c8d38a7cf266a3f7b2f090b108fb0cb031d95"
dependencies = [
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-codec-vorbis"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f025837c309cd69ffef572750b4a2257b59552c5399a5e49707cc5b1b85d1c73"
dependencies = [
"log",
"symphonia-core",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-core"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ea00cc4f79b7f6bb7ff87eddc065a1066f3a43fe1875979056672c9ef948c2af"
dependencies = [
"arrayvec",
"bitflags 1.3.2",
"bytemuck",
"lazy_static",
"log",
]
[[package]]
name = "symphonia-format-isomp4"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "243739585d11f81daf8dac8d9f3d18cc7898f6c09a259675fc364b382c30e0a5"
dependencies = [
"encoding_rs",
"log",
"symphonia-core",
"symphonia-metadata",
"symphonia-utils-xiph",
]
[[package]]
name = "symphonia-format-riff"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c2d7c3df0e7d94efb68401d81906eae73c02b40d5ec1a141962c592d0f11a96f"
dependencies = [
"extended",
"log",
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "symphonia-metadata"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "36306ff42b9ffe6e5afc99d49e121e0bd62fe79b9db7b9681d48e29fa19e6b16"
dependencies = [
"encoding_rs",
"lazy_static",
"log",
"symphonia-core",
]
[[package]]
name = "symphonia-utils-xiph"
version = "0.5.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee27c85ab799a338446b68eec77abf42e1a6f1bb490656e121c6e27bfbab9f16"
dependencies = [
"symphonia-core",
"symphonia-metadata",
]
[[package]]
name = "syn"
version = "1.0.109"
@@ -8366,8 +8684,8 @@ dependencies = [
"jni 0.21.1",
"libc",
"log",
"ndk",
"ndk-sys",
"ndk 0.9.0",
"ndk-sys 0.6.0+11769913",
"objc2 0.6.4",
"objc2-app-kit 0.3.2",
"objc2-foundation 0.3.2",
@@ -8402,6 +8720,16 @@ version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "55937e1799185b12863d447f42597ed69d9928686b8d88a1df17376a097d8369"
[[package]]
name = "tar"
version = "0.4.46"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3f6221d9a6003c78398e3b239969f352578258df48c8eb051caadae0015bc840"
dependencies = [
"filetime",
"libc",
]
[[package]]
name = "target-lexicon"
version = "0.12.14"
@@ -8786,7 +9114,7 @@ dependencies = [
"getrandom 0.4.1",
"once_cell",
"rustix 1.1.5",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -9296,7 +9624,7 @@ dependencies = [
"png 0.18.1",
"serde",
"thiserror 2.0.18",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -9360,6 +9688,12 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "typed-path"
version = "0.12.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "54ec2755506471f54baf5e4bb5499d8e510cc1caf271270e778a736bd2d7336c"
[[package]]
name = "typeid"
version = "1.0.2"
@@ -9389,7 +9723,7 @@ checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e"
dependencies = [
"memoffset",
"tempfile",
"windows-sys 0.61.2",
"windows-sys 0.60.2",
]
[[package]]
@@ -9541,6 +9875,35 @@ version = "0.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8ecb6da28b8a351d773b68d5825ac39017e680750f980f3a1a85cd8dd28a47c1"
[[package]]
name = "ureq"
version = "3.4.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9a7ac20be9b7726e0bbdbf974c059676d9acb1cd414961f570a4e8231cacd7fc"
dependencies = [
"base64 0.23.1",
"flate2",
"log",
"percent-encoding",
"rustls 0.23.36",
"rustls-pki-types",
"rustls-platform-verifier",
"ureq-proto",
"utf8-zero",
]
[[package]]
name = "ureq-proto"
version = "0.6.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "841583fc36089d7d59419568ae157f43a5a03a2e5a81ba87c77d1596ef46e922"
dependencies = [
"base64 0.23.1",
"http",
"httparse",
"log",
]
[[package]]
name = "url"
version = "2.5.8"
@@ -9630,6 +9993,12 @@ version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "86bd8d4e895da8537e5315b8254664e6b769c4ff3db18321b297a1e7004392e3"
[[package]]
name = "utf8-zero"
version = "0.8.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8c0a043c9540bae7c578c88f91dda8bd82e59ae27c21baca69c8b191aaf5a6e"
[[package]]
name = "utf8_iter"
version = "1.0.4"
@@ -9745,6 +10114,16 @@ dependencies = [
"vello_common",
]
[[package]]
name = "verifysign"
version = "1.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6efd25aadc22c756a9012267fae0b130278d335d6ceb512ec02ce82029b5ca9"
dependencies = [
"core-foundation 0.10.1",
"windows-sys 0.61.2",
]
[[package]]
name = "version-compare"
version = "0.2.0"
@@ -10447,6 +10826,16 @@ dependencies = [
"windows-version",
]
[[package]]
name = "windows"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9252e5725dbed82865af151df558e754e4a3c2c30818359eb17465f1346a1b49"
dependencies = [
"windows-core 0.54.0",
"windows-targets 0.52.6",
]
[[package]]
name = "windows"
version = "0.61.3"
@@ -10490,6 +10879,16 @@ dependencies = [
"windows-core 0.62.2",
]
[[package]]
name = "windows-core"
version = "0.54.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "12661b9c89351d684a50a8a643ce5f608e20243b9fb84687800163429f161d65"
dependencies = [
"windows-result 0.1.2",
"windows-targets 0.52.6",
]
[[package]]
name = "windows-core"
version = "0.61.2"
@@ -10603,6 +11002,15 @@ dependencies = [
"windows-strings 0.4.2",
]
[[package]]
name = "windows-result"
version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5e383302e8ec8515204254685643de10811af0ed97ea37210dc26fb0032647f8"
dependencies = [
"windows-targets 0.52.6",
]
[[package]]
name = "windows-result"
version = "0.3.4"
@@ -10985,7 +11393,7 @@ dependencies = [
"js-sys",
"libc",
"memmap2",
"ndk",
"ndk 0.9.0",
"objc2 0.5.2",
"objc2-app-kit 0.2.2",
"objc2-foundation 0.2.2",
@@ -11051,6 +11459,22 @@ dependencies = [
"windows-sys 0.48.0",
]
[[package]]
name = "winreg"
version = "0.55.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb5a765337c50e9ec252c2069be9bf91c7df47afb103b642ba3a53bf8101be97"
dependencies = [
"cfg-if",
"windows-sys 0.59.0",
]
[[package]]
name = "winsafe"
version = "0.0.29"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9ef0ffc427f045c0cc9ebffd6f4f91153dcd2a1547b5c3c29ee3f541e16e95c6"
[[package]]
name = "wit-bindgen"
version = "0.51.0"
@@ -11296,7 +11720,7 @@ dependencies = [
"javascriptcore-rs",
"jni 0.21.1",
"libc",
"ndk",
"ndk 0.9.0",
"objc2 0.6.4",
"objc2-app-kit 0.3.2",
"objc2-core-foundation",
@@ -11613,6 +12037,19 @@ dependencies = [
"syn 2.0.117",
]
[[package]]
name = "zip"
version = "9.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "711e8c9646791c49b7eb799a19c4ae70a9b8372ddc6a1fa4778defd1e2aa5d5e"
dependencies = [
"crc32fast",
"flate2",
"indexmap 2.13.0",
"memchr",
"typed-path",
]
[[package]]
name = "zmij"
version = "1.0.21"
@@ -11680,7 +12117,6 @@ dependencies = [
"endi",
"enumflags2",
"serde",
"url",
"winnow 1.0.0",
"zcheapstr",
"zvariant_derive",
+9 -2
View File
@@ -1,4 +1,9 @@
<h1 align="center">ArtCraft</h1>
<p align="center">
<a href="https://sonarcloud.io/project/overview?id=storytold_artcraft"><img alt="Scanned by SonarCloud" src="https://img.shields.io/badge/Scanned_by-SonarCloud-4b9fd5?style=flat-square&logo=sonarqubecloud&logoColor=white"></a>
<a href="https://coderabbit.ai"><img alt="Reviewed by CodeRabbit" src="https://img.shields.io/coderabbit/prs/github/storytold/artcraft?style=flat-square&label=Reviewed%20by%20CodeRabbit&logo=coderabbit"></a>
</p>
<p align="center"><strong>The IDE for artists.</strong></p>
<!-- Main video: https://www.youtube.com/watch?v=kzvQMdg66Go
@@ -126,13 +131,15 @@ Additional provider integrations are planned for Kling, Google, Runway, and Luma
## Downloads
- [Stable releases for Windows and macOS](https://getartcraft.com/)
- [Latest builds on GitHub](https://github.com/storytold/artcraft/releases)
- [Stable releases for Windows, macOS, and Linux](https://getartcraft.com/)
- [Latest builds on GitHub](https://github.com/storytold/artcraft/releases) (including AppImage, DEB, RPM, Arch, and Flatpak packages)
- [Linux packaging notes](./packaging/linux/README.md)
- [Build from source](./_docs/dev_setup.md), including Linux
## Documentation
- [Development setup](./_docs/dev_setup.md)
- [Linux packaging](./packaging/linux/README.md)
- [Developer documentation](./_docs)
- [Desktop build and development scripts](./script/artcraft)
- [Roadmap](./ROADMAP.md)
+11 -1
View File
@@ -13,6 +13,10 @@ path = "src/main.rs"
# Internal (Tauri-free) API crates shared with the Tauri app.
artcraft_api_defs.workspace = true
artcraft_client.workspace = true
artcraft_router.workspace = true
cookie_store_wrapper.workspace = true
fal_client.workspace = true
midjourney_client.workspace = true
tokens.workspace = true
# External
@@ -30,14 +34,20 @@ log.workspace = true
open = "5"
reqwest = { workspace = true, features = ["multipart"] }
resvg = "0.45"
rfd = "0.16"
rfd = { version = "0.16", default-features = false, features = ["gtk3"] }
rodio = { version = "0.20", default-features = false, features = ["symphonia-all"] }
serde.workspace = true
serde_json.workspace = true
tempfile.workspace = true
tokio = { workspace = true, features = ["rt-multi-thread", "sync", "time", "fs"] }
url.workspace = true
uuid = { workspace = true, features = ["v4"] }
# Pasting images from the clipboard (egui's own clipboard support is text only).
arboard = { version = "3", default-features = false, features = ["image-data"] }
# The Midjourney sign-in window (its login cookies can only be captured inside a browser).
wry = "0.55"
[dev-dependencies]
egui_kittest = "0.36"
+32 -16
View File
@@ -1,14 +1,15 @@
# ArtCraft Native
ArtCraft's desktop app in pure Rust: an [egui](https://github.com/emilk/egui) UI over the same
ArtCraft API client crates the Tauri app uses. There is no Tauri, no webview and no JavaScript.
ArtCraft API client crates the Tauri app uses. There is no Tauri and no JavaScript; the only
webview is the Midjourney sign-in window, which exists to capture Midjourney's login cookies.
The look follows the brutalist webapp (`artcraft-services/frontend/apps/artcraft-webapp`). That
means near-black surfaces with white/15 hairlines and 3 px corners on controls. Type is Archivo for
display, Inter for body text and Geist Mono for labels. The brand blue marks the primary actions.
Create Image and Create Video are fully built. The other tools show a "coming soon" page that
links to the same tool in the web app.
Create Image, Create Video and Create Audio are fully built. The other tools show a "coming soon"
page that links to the same tool in the web app.
## Running
@@ -32,24 +33,32 @@ The workspace's default member is the Tauri app, so pass `-p artcraft_native`.
| `backend::wire` | Lenient response shapes; new server enum values can't break decoding |
| `backend::session` | Credentials: own file, Tauri cookie import, env config |
| `backend::media_cache` | Thumbnails and previews: downloaded and decoded off-thread, evicted when unused |
| `backend::audio` | One clip at a time: reference previews, feed tracks (pause, seek, volume), waveforms |
| `backend::providers` | Midjourney and fal: accounts, submitting, and a worker that finishes jobs into the library |
| `models` | Model catalog from OmniGen listings, plus names, families and limits |
| `prompt_box` | The shared prompt box: editor, reference deck, keyframes, pickers, model selector |
| `pages` | Create Image, Create Video, Home, Library, and the shared page shell |
| `pages` | Create Image, Video and Audio, Home, Library, and the shared page shell |
| `feed` | Generation feed: running, failed and finished jobs, library paging, grid and list |
| `overlays` | Lightbox, library picker, sign-in and settings dialogs |
| `overlays` | Lightbox, library picker, sign-in, settings, provider setup and the Midjourney sign-in window |
| `shell` | Custom title bar, sidebar flush to the window edge, content panel header |
| `theme`, `ui` | Tokens, Lucide-style icons, widgets, toasts, window chrome (drag, resize, caption buttons) |
How the backend works:
- **Generation.** It goes through OmniGen: `/v1/omni_gen/{models,cost,generate}/{image,video}`.
- **Generation.** It goes through OmniGen: `/v1/omni_gen/{models,cost,generate}/{image,video,audio}`.
Model capabilities come from the server, so new models and options appear without an app update.
- **Typed client functions.** Model lists, uploads, credits, login and deletes use
`artcraft_client`.
- **Lenient decoding.** Generate and cost requests are JSON maps, as in the Tauri app. Job polling
(`/v1/jobs/session`), the library list, batches and prompts are decoded with lenient structs.
(`/v1/jobs/session`), the audio model list, the library list, batches and prompts are decoded
with lenient structs.
- **Identity headers.** Every request sends the desktop identity headers: `Origin` and
`User-Agent`.
- **Other providers.** Images can also run on Midjourney (your own account) or fal (your own API
key), through `artcraft_router` like the Tauri app. The accounts live in the same files
(`~/Artcraft/credentials`), so connecting in either app works in both. Those jobs are tracked in
`~/Artcraft/state/native_provider_jobs.json` and their results uploaded into your library, resuming
after a restart.
## Prompt box parity
@@ -69,24 +78,31 @@ How the backend works:
| Live credit cost on the generate button | Done |
| Validation: starting frame, text-only banner, limits, uploads | Done |
| Clear all (confirms when references are attached) | Done |
| `@Character` mentions and the Characters modal (Seedance 2.0) | Not yet |
| Provider choice (Midjourney direct, fal) | Not yet; everything runs via ArtCraft |
| In-app video and audio playback | Opens in the system player instead |
| Animated video previews in the feed | Still frames only |
| `@Character` mentions and the Characters modal (create/edit/delete) | Done |
| Provider choice (Midjourney direct, fal), "Select Provider" | Done (images; video runs on ArtCraft) |
| In-app video playback | Done with FFmpeg on the PATH; otherwise opens in the system player |
| Audio reference playback (click to play/stop) | Done |
| Animated video previews in the feed (with a stills toggle) | Done |
| Audio: style, instrumental, keep lyrics, loop | Done |
| Audio: Beat & Key (BPM, musical key) and Tuning (sample rate, speed, volume, pitch) | Done |
| Audio references (play, 600 s cap) or one image, never both | Done |
Feed and lightbox:
- **Feed.** Pending cards show progress, time left and a batch banner. Failed cards show the reason
and can be dismissed. The grid is masonry, with a list view alongside. Hover actions are Recreate,
Make Video, Share and Download. Select mode does batch downloads. The last viewed item is marked.
- **Audio.** Audio cards play in place: a waveform (click to seek), the time and a volume flyout.
One track plays at a time and stops when you leave the page. Audio offers Share and Download.
- **Lightbox.** Prev/next with the arrow keys, prompt copy and details. Actions are Recreate, Make
Video, Download, Share and Delete.
Video, Download, Share and Delete. Audio plays in a panel with the full-size player.
## Diagnostics
Set `ARTCRAFT_SCREENSHOT=<file.png>` to save a screenshot once the window has settled.
`ARTCRAFT_PAGE` (`home`, `create-image`, `create-video` or `library`) picks the page, and
`ARTCRAFT_SCREENSHOT_EXIT=1` closes the app afterwards.
`ARTCRAFT_PAGE` (`home`, `create-image`, `create-video`, `create-audio` or `library`) picks the page,
`ARTCRAFT_OPEN` (`settings` or `midjourney-login`) opens a dialog, and `ARTCRAFT_SCREENSHOT_EXIT=1`
closes the app afterwards. The screenshot shows egui only, not the sign-in webview.
## Tests
@@ -95,8 +111,8 @@ cargo test -p artcraft_native
```
There are unit tests for the catalog, feed store, wire decoding and backend helpers. Headless UI
tests (`egui_kittest`) drive the prompt box: generate gating, Enter to generate, the pickers,
`@` mentions and clear-all.
tests (`egui_kittest`) drive the prompt box (generate gating, Enter to generate, the pickers, `@`
mentions, clear-all), the Characters dialog and the audio toolbar (toggles, Beat & Key, Tuning).
## Fonts and brand assets
+318 -100
View File
@@ -7,22 +7,28 @@ use std::time::Duration;
use egui::{Id, Ui};
use serde::{Deserialize, Serialize};
use crate::backend::audio::AudioPlayer;
use crate::backend::media_cache::MediaCache;
use crate::backend::wire::{Prompt, SessionUser};
use crate::backend::providers::Provider;
use crate::backend::{Backend, Event, Modality, SHARE_URL_BASE};
use crate::feed::grid::{RatioCache, ViewMode};
use crate::feed::store::{FeedNotice, FeedStore};
use crate::feed::types::{FeedItem, MediaKind};
use crate::models::Catalog;
use crate::backend::wire::{Character, JobPhase};
use crate::overlays::account::{LoginAction, LoginDialog, SettingsAction, settings_dialog};
use crate::overlays::characters::{CharactersAction, CharactersModal};
use crate::overlays::library_picker::{LibraryPicker, PickerAction};
use crate::overlays::lightbox::{Lightbox, LightboxAction, fit_inside};
use crate::overlays::providers::{AccountAction, AccountsView, LoginEvent, MidjourneyLogin, SetupAction, setup_dialog};
use crate::pages::common::{self, AppRequest, Env, FeedKey};
use crate::pages::create_audio::{AudioSettings, CreateAudioPage};
use crate::pages::create_image::{CreateImagePage, ImageSettings};
use crate::pages::create_video::{CreateVideoPage, VideoSettings};
use crate::pages::other;
use crate::prompt_box::refs::{RefKind, RefMedia};
use crate::shell::{self, Account, Page, ShellAction};
use crate::prompt_box::refs::{RefKind, RefMedia, References};
use crate::shell::{self, Account, FeedToggles, Page, ShellAction};
use crate::theme;
use crate::ui::toast::Toasts;
use crate::ui::{creator_icons, widgets, window};
@@ -31,7 +37,7 @@ use crate::ui::{creator_icons, widgets, window};
/// borrowed mutably alongside it).
macro_rules! env {
($app:ident, $ctx:expr) => {
Env { ctx: $ctx.clone(), backend: &$app.backend, cache: &mut $app.cache, toasts: &mut $app.toasts, catalog: &$app.catalog, signed_in: $app.user.is_some(), enter_to_generate: $app.enter_to_generate, view_mode: $app.view_mode, ratios: &mut $app.ratios, prompts: &$app.prompts, requests: &mut $app.requests }
Env { ctx: $ctx.clone(), backend: &$app.backend, cache: &mut $app.cache, audio: &mut $app.player, toasts: &mut $app.toasts, catalog: &$app.catalog, signed_in: $app.user.is_some(), enter_to_generate: $app.enter_to_generate, view_mode: $app.view_mode, autoplay: $app.autoplay, ratios: &mut $app.ratios, prompts: &$app.prompts, requests: &mut $app.requests, characters: &$app.characters }
};
}
@@ -41,6 +47,8 @@ const CREDITS_INTERVAL: f64 = 60.0;
const MODEL_RETRY_INTERVAL: f64 = 15.0;
/// List view: prompt records fetched per frame at most.
const PROMPT_FETCHES_PER_FRAME: usize = 8;
/// Every create page's model listing.
const MODALITIES: [Modality; 3] = [Modality::Image, Modality::Video, Modality::Audio];
/// Preferences that survive restarts.
#[derive(Clone, Debug, Serialize, Deserialize)]
@@ -49,20 +57,23 @@ struct Saved {
page: Page,
enter_to_generate: bool,
view_mode: ViewMode,
autoplay: bool,
sidebar_open: bool,
image: ImageSettings,
video: VideoSettings,
audio: AudioSettings,
}
impl Default for Saved {
fn default() -> Self {
Self { page: Page::CreateImage, enter_to_generate: false, view_mode: ViewMode::Grid, sidebar_open: true, image: ImageSettings { count: 1, ..Default::default() }, video: VideoSettings::default() }
Self { page: Page::CreateImage, enter_to_generate: false, view_mode: ViewMode::Grid, autoplay: true, sidebar_open: true, image: ImageSettings { count: 1, ..Default::default() }, video: VideoSettings::default(), audio: AudioSettings::default() }
}
}
pub struct ArtcraftApp {
backend: Backend,
cache: MediaCache,
player: AudioPlayer,
toasts: Toasts,
catalog: Catalog,
user: Option<SessionUser>,
@@ -70,9 +81,11 @@ pub struct ArtcraftApp {
page: Page,
enter_to_generate: bool,
view_mode: ViewMode,
autoplay: bool,
sidebar_open: bool,
image: CreateImagePage,
video: CreateVideoPage,
audio: CreateAudioPage,
library: FeedStore,
ratios: RatioCache,
/// Prompt texts by prompt token (list view, lightbox).
@@ -86,7 +99,16 @@ pub struct ArtcraftApp {
lightbox: Option<Lightbox>,
picker: Option<LibraryPicker>,
login: Option<LoginDialog>,
characters_modal: Option<CharactersModal>,
characters: Vec<Character>,
/// Characters being made: (job token, name).
pending_characters: Vec<(String, String)>,
settings_open: bool,
/// "Set up {provider}", after generating needed an account that isn't connected.
provider_setup: Option<Provider>,
midjourney_login: Option<MidjourneyLogin>,
/// The fal key being typed (setup dialog and Settings).
fal_key_input: String,
preview: Option<String>,
requests: Vec<AppRequest>,
screenshot: Option<ScreenshotRequest>,
@@ -105,10 +127,25 @@ impl ArtcraftApp {
}
let backend = Backend::new(cc.egui_ctx.clone());
let cache = MediaCache::new(backend.runtime(), backend.http());
backend.load_models(Modality::Image);
backend.load_models(Modality::Video);
let player = AudioPlayer::new(backend.runtime(), backend.http());
for modality in MODALITIES {
backend.load_models(modality);
}
backend.refresh_session();
Self { cache, toasts: Toasts::default(), catalog: Catalog::default(), user: None, credits: None, page: saved.page, enter_to_generate: saved.enter_to_generate, view_mode: saved.view_mode, sidebar_open: saved.sidebar_open, image: CreateImagePage::new(saved.image), video: CreateVideoPage::new(saved.video), library: FeedStore::default(), ratios: RatioCache::default(), prompts: HashMap::new(), prompt_records: HashMap::new(), requested_prompts: HashSet::new(), pending_recreate: None, batch_kinds: HashMap::new(), lightbox: None, picker: None, login: None, settings_open: false, preview: None, requests: Vec::new(), screenshot: ScreenshotRequest::from_env(), next_poll: 0.0, next_credits: CREDITS_INTERVAL, models_retry_at: None, backend }
let mut app = Self { cache, player, toasts: Toasts::default(), catalog: Catalog::default(), user: None, credits: None, page: saved.page, enter_to_generate: saved.enter_to_generate, view_mode: saved.view_mode, autoplay: saved.autoplay, sidebar_open: saved.sidebar_open, image: CreateImagePage::new(saved.image), video: CreateVideoPage::new(saved.video), audio: CreateAudioPage::new(saved.audio), library: FeedStore::default(), ratios: RatioCache::default(), prompts: HashMap::new(), prompt_records: HashMap::new(), requested_prompts: HashSet::new(), pending_recreate: None, batch_kinds: HashMap::new(), lightbox: None, picker: None, login: None, characters_modal: None, characters: Vec::new(), pending_characters: Vec::new(), settings_open: false, provider_setup: None, midjourney_login: None, fal_key_input: String::new(), preview: None, requests: Vec::new(), screenshot: ScreenshotRequest::from_env(), next_poll: 0.0, next_credits: CREDITS_INTERVAL, models_retry_at: None, backend };
app.restore_provider_jobs();
// Diagnostics: `ARTCRAFT_OPEN=settings` or `midjourney-login` opens that dialog at launch.
match std::env::var("ARTCRAFT_OPEN").as_deref() {
Ok("settings") => app.settings_open = true,
Ok("midjourney-login") => app.account_action(AccountAction::ConnectMidjourney),
_ => {},
}
app
}
/// Pending cards for the Midjourney and fal jobs still running.
fn restore_provider_jobs(&mut self) {
self.image.feed.restore_local(&self.backend.provider_jobs(), &self.catalog);
}
// --- Events from the backend -------------------------------------------------------------
@@ -119,13 +156,17 @@ impl ArtcraftApp {
let changed = self.user.as_ref().map(|u| &u.username) != user.as_ref().map(|u| &u.username);
self.user = user;
if changed {
self.image.feed.reset();
self.video.feed.reset();
self.library.reset();
for feed in self.feeds_mut() {
feed.reset();
}
self.restore_provider_jobs();
self.next_poll = 0.0;
}
if self.user.is_some() {
self.login = None;
if changed {
self.backend.load_characters();
}
} else {
self.credits = None;
if let Some(login) = &mut self.login {
@@ -149,6 +190,7 @@ impl ArtcraftApp {
Event::Models(modality, Ok(models)) => match modality {
Modality::Image => self.catalog.set_image(models),
Modality::Video => self.catalog.set_video(models),
Modality::Audio => self.catalog.set_audio(models),
},
Event::Models(_, Err(message)) => {
if self.models_retry_at.is_none() {
@@ -157,34 +199,65 @@ impl ArtcraftApp {
self.models_retry_at = Some(ctx.input(|i| i.time) + MODEL_RETRY_INTERVAL);
},
Event::Cost { key, credits } => {
self.image.cost.apply(&key, credits);
self.video.cost.apply(&key, credits);
for cost in [&mut self.image.cost, &mut self.video.cost, &mut self.audio.cost] {
cost.apply(&key, credits);
}
},
Event::Enqueued { modality, job_tokens, meta } => {
let mut env = env!(self, ctx);
match modality {
Modality::Image => self.image.on_enqueued(&mut env, &job_tokens, &meta),
Modality::Video => self.video.on_enqueued(&mut env, &job_tokens, &meta),
Modality::Audio => self.audio.on_enqueued(&mut env, &job_tokens, &meta),
}
self.next_poll = ctx.input(|i| i.time) + 2.0;
},
Event::EnqueueFailed { modality, message } => {
Event::EnqueueFailed { modality, message, status } => {
let mut env = env!(self, ctx);
match modality {
Modality::Image => self.image.on_enqueue_failed(&mut env, &message),
Modality::Video => self.video.on_enqueue_failed(&mut env, &message),
Modality::Audio => self.audio.on_enqueue_failed(&mut env, &message, status),
}
},
Event::Uploaded { ref_id, result } => {
if let Some(modal) = &mut self.characters_modal {
match modal.apply_upload(ref_id, result.clone()) {
Some(Err(err)) => {
self.toasts.error(format!("Failed to upload the reference image: {err}"));
return;
},
Some(Ok(())) => return,
None => {},
}
}
if self.image.refs.find_mut(ref_id).is_some() {
common::apply_upload(&mut self.toasts, &mut self.image.refs, ref_id, result);
} else if self.audio.refs.find_mut(ref_id).is_some() {
if common::apply_upload(&mut self.toasts, &mut self.audio.refs, ref_id, result) {
self.audio.check_durations(&mut self.toasts, ref_id);
}
} else if common::apply_upload(&mut self.toasts, &mut self.video.refs, ref_id, result) {
let mut env = env!(self, ctx);
self.video.check_durations(&mut env, ref_id);
}
},
Event::Jobs(jobs) => {
for (kind, notices) in [(MediaKind::Image, self.image.feed.apply_jobs(MediaKind::Image, &jobs, &self.catalog)), (MediaKind::Video, self.video.feed.apply_jobs(MediaKind::Video, &jobs, &self.catalog))] {
// Characters being made finish as jobs too.
let before = self.pending_characters.len();
let toasts = &mut self.toasts;
self.pending_characters.retain(|(job, name)| match jobs.iter().find(|j| &j.job_token == job).map(|j| j.status.phase()) {
Some(JobPhase::Succeeded) => false,
Some(JobPhase::Failed) => {
toasts.error(format!("Character \u{201c}{name}\u{201d} failed to create"));
false
},
_ => true,
});
if self.pending_characters.len() != before {
self.backend.load_characters();
}
for (kind, notices) in [(MediaKind::Image, self.image.feed.apply_jobs(MediaKind::Image, &jobs, &self.catalog)), (MediaKind::Video, self.video.feed.apply_jobs(MediaKind::Video, &jobs, &self.catalog)), (MediaKind::Audio, self.audio.feed.apply_jobs(MediaKind::Audio, &jobs, &self.catalog))] {
for notice in notices {
match notice {
FeedNotice::Completed => {
@@ -200,33 +273,30 @@ impl ArtcraftApp {
}
}
},
Event::LibraryPage { key, page, items, total_pages } => match key.as_str() {
"picker" => {
// The picker's pages come back under its own name.
Event::LibraryPage { key, page, items, total_pages } => match FeedKey::from_name(&key) {
Some(feed) => self.feed_mut(feed).apply_library_page(feed.kind(), page, &items, total_pages),
None => {
if let Some(p) = &mut self.picker {
p.apply_page(page, items, total_pages);
}
},
"image" => self.image.feed.apply_library_page(MediaKind::Image, page, &items, total_pages),
"video" => self.video.feed.apply_library_page(MediaKind::Video, page, &items, total_pages),
_ => self.library.apply_library_page(MediaKind::Image, page, &items, total_pages),
},
Event::LibraryFailed { key, message } => {
match key.as_str() {
"picker" => {
match FeedKey::from_name(&key) {
Some(feed) => self.feed_mut(feed).library_failed(),
None => {
if let Some(p) = &mut self.picker {
p.loading = false;
p.has_more = false;
}
},
"image" => self.image.feed.library_failed(),
"video" => self.video.feed.library_failed(),
_ => self.library.library_failed(),
}
self.toasts.error(format!("Couldn't load your library: {message}"));
},
Event::Batch { job_token, items } => match self.batch_kinds.remove(&job_token) {
Some(MediaKind::Video) => self.video.feed.apply_batch(MediaKind::Video, &items),
_ => self.image.feed.apply_batch(MediaKind::Image, &items),
Event::Batch { job_token, items } => {
let kind = self.batch_kinds.remove(&job_token).unwrap_or(MediaKind::Image);
self.feed_mut(FeedKey::of(kind)).apply_batch(kind, &items);
},
Event::Prompt(prompt) => {
if let Some(text) = &prompt.maybe_positive_prompt {
@@ -239,8 +309,32 @@ impl ArtcraftApp {
}
self.prompt_records.insert(prompt.token.clone(), prompt);
},
Event::Characters(list) => self.characters = list,
Event::CharacterCreating { job_token, name } => {
self.toasts.success(format!("Character \u{201c}{name}\u{201d} is being created"));
self.pending_characters.push((job_token, name));
self.next_poll = 0.0;
},
Event::CharacterChanged => self.backend.load_characters(),
Event::ProviderSetupNeeded(provider) => {
self.image.generating = false;
self.provider_setup = Some(provider);
},
Event::ProviderJobFinished { job_id, batch_token } => {
if self.image.feed.finish_local(&job_id) {
self.toasts.success("Image generation complete!");
}
// The batch brings the images into the feed.
self.batch_kinds.insert(job_id.clone(), MediaKind::Image);
self.backend.load_batch(job_id, batch_token);
},
Event::ProviderJobFailed { job_id, reason } => {
self.image.feed.fail_local(&job_id, &reason);
self.toasts.error(reason);
},
Event::ProviderAccountsChanged => {},
Event::Deleted(token) => {
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
for feed in self.feeds_mut() {
feed.remove_item(&token);
}
if self.lightbox.as_ref().is_some_and(|l| l.token == token) {
@@ -274,7 +368,7 @@ impl ArtcraftApp {
self.open_login();
return;
};
let refs = if page == MediaKind::Image { &self.image.refs } else { &self.video.refs };
let refs = self.refs(page);
let attached: HashSet<String> = refs.list(kind).iter().chain(refs.first_frame.iter()).chain(refs.last_frame.iter()).filter_map(|r| r.token.clone()).collect();
let mut picker = LibraryPicker::new(kind, slot, page, max, attached);
picker.loading = true;
@@ -287,8 +381,7 @@ impl ArtcraftApp {
};
let image = RefMedia::from_library(RefKind::Image, item.token.clone(), item.thumbnail.clone(), Some(item.full_url.clone()), 0.0);
self.video.set_start_image(image);
self.lightbox = None;
self.page = Page::CreateVideo;
self.navigate(Page::CreateVideo);
},
AppRequest::Recreate { kind, token } => {
let Some(prompt_token) = self.find_item(&token).and_then(|i| i.prompt_token.clone()) else {
@@ -314,17 +407,27 @@ impl ArtcraftApp {
},
AppRequest::Download(tokens) => self.download(&tokens),
AppRequest::Preview(url) => self.preview = Some(url),
AppRequest::ToggleAudio { ref_id, url } => self.player.toggle(ctx, &format!("ref:{ref_id}"), &url),
AppRequest::OpenCharacters => {
if self.user.is_none() {
self.open_login();
return;
}
self.backend.load_characters();
self.characters_modal = Some(CharactersModal::new());
},
AppRequest::PlayExternally(url) => {
if let Err(err) = open::that(&url) {
self.toasts.error(format!("Couldn't open the file: {err}"));
}
},
AppRequest::LoadMoreLibrary(key) => {
let Some(user) = &self.user else {
return;
};
let (feed, name) = match key {
FeedKey::Image => (&mut self.image.feed, "image"),
FeedKey::Video => (&mut self.video.feed, "video"),
FeedKey::Library => (&mut self.library, "library"),
};
if let Some(page) = feed.next_library_page() {
self.backend.load_library(name.to_owned(), user.username.clone(), key.media_classes(), page);
let username = user.username.clone();
if let Some(page) = self.feed_mut(key).next_library_page() {
self.backend.load_library(key.name().to_owned(), username, key.media_classes(), page);
}
},
}
@@ -334,14 +437,15 @@ impl ArtcraftApp {
match kind {
MediaKind::Image => {
self.image.apply_prompt(prompt);
self.page = Page::CreateImage;
self.navigate(Page::CreateImage);
},
MediaKind::Video => {
self.video.apply_prompt(prompt);
self.page = Page::CreateVideo;
self.navigate(Page::CreateVideo);
},
// Audio isn't recreated yet (no Recreate button offers it).
MediaKind::Audio => {},
}
self.lightbox = None;
}
fn download(&mut self, tokens: &[String]) {
@@ -359,7 +463,7 @@ impl ArtcraftApp {
for item in many {
self.backend.save_url(item.full_url.clone(), dir.join(file_name_for(item)));
}
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
for feed in self.feeds_mut() {
feed.selecting = false;
feed.selected.clear();
}
@@ -369,7 +473,52 @@ impl ArtcraftApp {
}
fn find_item(&self, token: &str) -> Option<&FeedItem> {
self.image.feed.find(token).or_else(|| self.video.feed.find(token)).or_else(|| self.library.find(token))
self.feeds().into_iter().find_map(|f| f.find(token))
}
fn feeds(&self) -> [&FeedStore; 4] {
[&self.image.feed, &self.video.feed, &self.audio.feed, &self.library]
}
fn feeds_mut(&mut self) -> [&mut FeedStore; 4] {
[&mut self.image.feed, &mut self.video.feed, &mut self.audio.feed, &mut self.library]
}
fn feed(&self, key: FeedKey) -> &FeedStore {
match key {
FeedKey::Image => &self.image.feed,
FeedKey::Video => &self.video.feed,
FeedKey::Audio => &self.audio.feed,
FeedKey::Library => &self.library,
}
}
fn feed_mut(&mut self, key: FeedKey) -> &mut FeedStore {
match key {
FeedKey::Image => &mut self.image.feed,
FeedKey::Video => &mut self.video.feed,
FeedKey::Audio => &mut self.audio.feed,
FeedKey::Library => &mut self.library,
}
}
/// The references of the page that makes `kind`.
fn refs(&self, kind: MediaKind) -> &References {
match kind {
MediaKind::Image => &self.image.refs,
MediaKind::Video => &self.video.refs,
MediaKind::Audio => &self.audio.refs,
}
}
/// Opens `page`, closing the lightbox. Leaving a page stops its audio, like the webapp's
/// players unmounting.
fn navigate(&mut self, page: Page) {
if page != self.page {
self.player.stop();
}
self.page = page;
self.lightbox = None;
}
fn request_prompt(&mut self, prompt_token: Option<String>) {
@@ -386,12 +535,13 @@ impl ArtcraftApp {
}
}
/// The feed the top bar's toggles act on.
fn current_feed(&mut self) -> Option<&mut FeedStore> {
/// The feed the page shows (and the top bar's toggles act on).
fn page_feed(&self) -> Option<FeedKey> {
match self.page {
Page::CreateImage => Some(&mut self.image.feed),
Page::CreateVideo => Some(&mut self.video.feed),
Page::Library => Some(&mut self.library),
Page::CreateImage => Some(FeedKey::Image),
Page::CreateVideo => Some(FeedKey::Video),
Page::CreateAudio => Some(FeedKey::Audio),
Page::Library => Some(FeedKey::Library),
_ => None,
}
}
@@ -410,11 +560,10 @@ impl ArtcraftApp {
}
if self.models_retry_at.is_some_and(|t| now >= t) {
self.models_retry_at = None;
if self.catalog.image.is_empty() {
self.backend.load_models(Modality::Image);
}
if self.catalog.video.is_empty() {
self.backend.load_models(Modality::Video);
for (modality, loaded) in MODALITIES.into_iter().zip([&self.catalog.image, &self.catalog.video, &self.catalog.audio]) {
if loaded.is_empty() {
self.backend.load_models(modality);
}
}
}
ctx.request_repaint_after(Duration::from_secs(1));
@@ -441,10 +590,7 @@ impl ArtcraftApp {
fn apply_shell_action(&mut self, action: ShellAction) {
match action {
ShellAction::Navigate(page) => {
self.page = page;
self.lightbox = None;
},
ShellAction::Navigate(page) => self.navigate(page),
ShellAction::ToggleSidebar => self.sidebar_open = !self.sidebar_open,
ShellAction::SignIn => self.open_login(),
ShellAction::SignOut => {
@@ -453,8 +599,10 @@ impl ArtcraftApp {
},
ShellAction::OpenSettings => self.settings_open = true,
ShellAction::SetViewMode(mode) => self.view_mode = mode,
ShellAction::SetAutoplay(on) => self.autoplay = on,
ShellAction::ToggleSelect => {
if let Some(feed) = self.current_feed() {
if let Some(key) = self.page_feed() {
let feed = self.feed_mut(key);
feed.selecting = !feed.selecting;
feed.selected.clear();
}
@@ -468,13 +616,10 @@ impl ArtcraftApp {
let ctx = ui.ctx().clone();
let gap = shell::PANEL_GAP;
let margin = egui::Margin { left: if self.sidebar_open { 0 } else { gap }, right: gap, top: 0, bottom: gap };
let toggles = match self.page {
Page::CreateImage => Some((self.view_mode, self.image.feed.selecting)),
Page::CreateVideo => Some((self.view_mode, self.video.feed.selecting)),
Page::Library => Some((self.view_mode, self.library.selecting)),
_ => None,
};
// Only feeds with videos offer the preview autoplay toggle.
let toggles = self.page_feed().map(|key| FeedToggles { mode: self.view_mode, selecting: self.feed(key).selecting, autoplay: matches!(key, FeedKey::Video | FeedKey::Library).then_some(self.autoplay) });
let mut header_action = None;
let mut open_page = None;
egui::CentralPanel::default().frame(egui::Frame::NONE.fill(theme::CHROME).inner_margin(margin)).show(ui, |ui| {
let panel = egui::Frame::new().fill(theme::BG).stroke(theme::hairline()).corner_radius(shell::PANEL_RADIUS);
panel.show(ui, |ui| {
@@ -485,13 +630,10 @@ impl ArtcraftApp {
egui::CentralPanel::default().frame(egui::Frame::NONE).show(ui, |ui| {
let mut env = env!(self, ctx);
match self.page {
Page::Home => {
if let Some(page) = other::home(ui, env.signed_in, env.requests) {
self.page = page;
}
},
Page::Home => open_page = other::home(ui, env.signed_in, env.requests),
Page::CreateImage => self.image.ui(ui, &mut env),
Page::CreateVideo => self.video.ui(ui, &mut env),
Page::CreateAudio => self.audio.ui(ui, &mut env),
Page::Library => other::library(ui, &mut env, &mut self.library),
page => other::coming_soon(ui, page),
}
@@ -501,39 +643,33 @@ impl ArtcraftApp {
if let Some(action) = header_action {
self.apply_shell_action(action);
}
self.prefetch_list_prompts();
if let Some(page) = open_page {
self.navigate(page);
}
self.prefetch_prompts();
}
/// The list view shows prompt texts: fetch the records for the visible feed, a few at a time.
fn prefetch_list_prompts(&mut self) {
if self.view_mode != ViewMode::List {
/// The list view and audio cards show prompt texts: fetch the records for the visible feed, a
/// few at a time.
fn prefetch_prompts(&mut self) {
let Some(key) = self.page_feed() else {
return;
}
let feed = match self.page {
Page::CreateImage => &self.image.feed,
Page::CreateVideo => &self.video.feed,
Page::Library => &self.library,
_ => return,
};
let wanted: Vec<String> = feed.items.iter().filter_map(|i| i.prompt_token.clone()).filter(|t| !self.requested_prompts.contains(t)).take(PROMPT_FETCHES_PER_FRAME).collect();
let list = self.view_mode == ViewMode::List;
let wanted: Vec<String> = self.feed(key).items.iter().filter(|i| list || i.kind == MediaKind::Audio).filter_map(|i| i.prompt_token.clone()).filter(|t| !self.requested_prompts.contains(t)).take(PROMPT_FETCHES_PER_FRAME).collect();
for token in wanted {
self.request_prompt(Some(token));
}
}
fn overlays(&mut self, ctx: &egui::Context) {
if let Some(lb) = &mut self.lightbox {
let feed = match lb.kind {
_ if self.page == Page::Library => &self.library,
MediaKind::Image => &self.image.feed,
MediaKind::Video => &self.video.feed,
};
let order = feed.ordered_tokens();
let item = self.image.feed.find(&lb.token).or_else(|| self.video.feed.find(&lb.token)).or_else(|| self.library.find(&lb.token)).cloned();
match item {
fn overlays(&mut self, ctx: &egui::Context, frame: &eframe::Frame) {
if let Some((kind, token)) = self.lightbox.as_ref().map(|lb| (lb.kind, lb.token.clone())) {
let key = if self.page == Page::Library { FeedKey::Library } else { FeedKey::of(kind) };
let order = self.feed(key).ordered_tokens();
match self.find_item(&token).cloned() {
Some(item) => {
let prompt = item.prompt_token.as_ref().and_then(|t| self.prompt_records.get(t));
let actions = lb.show(ctx, &item, &order, prompt, &self.catalog, &mut self.cache);
let actions = self.lightbox.as_mut().map(|lb| lb.show(ctx, &item, &order, prompt, &self.catalog, &mut self.cache, &mut self.player)).unwrap_or_default();
for action in actions {
self.lightbox_action(ctx, action, &item);
}
@@ -555,8 +691,11 @@ impl ArtcraftApp {
},
PickerAction::Confirm(picks) => {
if let Some(p) = self.picker.take() {
let refs = if p.page == MediaKind::Image { &mut self.image.refs } else { &mut self.video.refs };
common::attach_from_library(refs, p.kind, p.slot, picks);
match p.page {
MediaKind::Image => common::attach_from_library(&mut self.image.refs, p.kind, p.slot, picks),
MediaKind::Video => common::attach_from_library(&mut self.video.refs, p.kind, p.slot, picks),
MediaKind::Audio => self.audio.attach_from_library(&mut self.toasts, &self.catalog, p.kind, picks),
}
}
},
}
@@ -584,9 +723,32 @@ impl ArtcraftApp {
None => {},
}
}
if self.settings_open {
if let Some(modal) = &mut self.characters_modal {
let pending: Vec<String> = self.pending_characters.iter().map(|(_, n)| n.clone()).collect();
for action in modal.show(ctx, &self.characters, &pending, &mut self.cache) {
match action {
CharactersAction::Close => self.characters_modal = None,
CharactersAction::Select { token, name } => {
self.video.mention_character(&name, &token);
self.characters_modal = None;
},
CharactersAction::Upload { ref_id, path } => self.backend.upload_file(ref_id, RefKind::Image, path),
CharactersAction::Create { image_token, name, description } => self.backend.create_character(image_token, name, description),
CharactersAction::Edit { token, name, description } => self.backend.edit_character(token, name, description),
CharactersAction::Delete { token, name } => self.backend.delete_character(token, name),
CharactersAction::Preview(url) => self.preview = Some(url),
CharactersAction::Toast(message) => self.toasts.error(message),
}
if self.characters_modal.is_none() {
break;
}
}
}
if self.settings_open && self.midjourney_login.is_none() {
let username = self.user.as_ref().map(|u| u.username.clone());
match settings_dialog(ctx, &mut self.enter_to_generate, username.as_deref(), self.credits, self.backend.data_root()) {
let accounts = self.backend.accounts();
let view = AccountsView { midjourney: accounts.midjourney_account(), fal_key_hint: accounts.fal_key_hint() };
match settings_dialog(ctx, &mut self.enter_to_generate, username.as_deref(), self.credits, &view, &mut self.fal_key_input, self.backend.data_root()) {
Some(SettingsAction::Close) => self.settings_open = false,
Some(SettingsAction::SignOut) => {
self.settings_open = false;
@@ -596,6 +758,32 @@ impl ArtcraftApp {
self.settings_open = false;
self.open_login();
},
Some(SettingsAction::Account(action)) => self.account_action(action),
None => {},
}
}
if let Some(provider) = self.provider_setup.filter(|_| self.midjourney_login.is_none()) {
match setup_dialog(ctx, provider, &mut self.fal_key_input) {
Some(SetupAction::Close) => self.provider_setup = None,
Some(SetupAction::ConnectMidjourney) => {
self.provider_setup = None;
self.account_action(AccountAction::ConnectMidjourney);
},
Some(SetupAction::SaveFalKey) => {
self.provider_setup = None;
self.account_action(AccountAction::SaveFalKey);
},
None => {},
}
}
if let Some(login) = &mut self.midjourney_login {
match login.show(ctx, frame) {
Some(LoginEvent::Captured(cookies)) => {
self.midjourney_login = None;
self.backend.save_midjourney_login(cookies);
self.toasts.success("Midjourney connected");
},
Some(LoginEvent::Closed) => self.midjourney_login = None,
None => {},
}
}
@@ -618,6 +806,26 @@ impl ArtcraftApp {
}
}
/// Connects or disconnects a provider account.
fn account_action(&mut self, action: AccountAction) {
match action {
AccountAction::ConnectMidjourney => self.midjourney_login = Some(MidjourneyLogin::new(self.backend.data_root())),
AccountAction::DisconnectMidjourney => self.backend.disconnect_midjourney(),
AccountAction::SaveFalKey => match self.backend.accounts().set_fal_key(&self.fal_key_input) {
Ok(()) => {
self.fal_key_input.clear();
self.toasts.success("FAL API key saved");
},
Err(err) => self.toasts.error(format!("Failed to save API key: {err}")),
},
AccountAction::RemoveFalKey => {
if let Err(err) = self.backend.accounts().clear_fal_key() {
self.toasts.error(format!("Failed to clear provider credential: {err}"));
}
},
}
}
fn lightbox_action(&mut self, ctx: &egui::Context, action: LightboxAction, item: &FeedItem) {
match action {
LightboxAction::Close => self.lightbox = None,
@@ -625,7 +833,7 @@ impl ArtcraftApp {
if let Some(next) = self.find_item(&token).cloned() {
self.request_prompt(next.prompt_token);
}
for feed in [&mut self.image.feed, &mut self.video.feed, &mut self.library] {
for feed in self.feeds_mut() {
if feed.find(&token).is_some() {
feed.last_viewed = Some(token.clone());
}
@@ -660,15 +868,20 @@ impl eframe::App for ArtcraftApp {
self.timers(ctx);
}
fn ui(&mut self, ui: &mut Ui, _frame: &mut eframe::Frame) {
fn ui(&mut self, ui: &mut Ui, frame: &mut eframe::Frame) {
let ctx = ui.ctx().clone();
self.cache.poll(&ctx);
if let Some(err) = self.player.poll() {
self.toasts.error(err);
}
let playing = [&self.video.refs, &self.image.refs].iter().flat_map(|r| r.audios.iter()).map(|r| r.id).find(|id| self.player.is_playing(&format!("ref:{id}")));
crate::prompt_box::deck::set_playing_audio(&ctx, playing);
self.shell(ui);
self.page_ui(ui);
for request in std::mem::take(&mut self.requests) {
self.handle_request(&ctx, request);
}
self.overlays(&ctx);
self.overlays(&ctx, frame);
self.toasts.show(&ctx);
window::window_edges(&ctx);
if let Some(shot) = &mut self.screenshot {
@@ -677,7 +890,7 @@ impl eframe::App for ArtcraftApp {
}
fn save(&mut self, storage: &mut dyn eframe::Storage) {
let saved = Saved { page: self.page, enter_to_generate: self.enter_to_generate, view_mode: self.view_mode, sidebar_open: self.sidebar_open, image: self.image.settings.clone(), video: self.video.settings.clone() };
let saved = Saved { page: self.page, enter_to_generate: self.enter_to_generate, view_mode: self.view_mode, autoplay: self.autoplay, sidebar_open: self.sidebar_open, image: self.image.settings.clone(), video: self.video.settings.clone(), audio: self.audio.settings.clone() };
eframe::set_value(storage, STORAGE_KEY, &saved);
}
@@ -733,6 +946,11 @@ impl ScreenshotRequest {
/// `artcraft_<token>.<ext>`, the extension taken from the URL.
fn file_name_for(item: &FeedItem) -> String {
let ext = item.full_url.rsplit('.').next().filter(|e| e.len() <= 4 && e.chars().all(|c| c.is_ascii_alphanumeric())).unwrap_or(if item.kind == MediaKind::Video { "mp4" } else { "png" });
let fallback = match item.kind {
MediaKind::Image => "png",
MediaKind::Video => "mp4",
MediaKind::Audio => "mp3",
};
let ext = item.full_url.rsplit('.').next().filter(|e| e.len() <= 4 && e.chars().all(|c| c.is_ascii_alphanumeric())).unwrap_or(fallback);
format!("artcraft_{}.{ext}", item.token)
}
+343
View File
@@ -0,0 +1,343 @@
//! Audio, one clip at a time (`audio-playback-controller`): reference cards play and stop on
//! click (the webapp's `DeckCard`), feed tracks pause, resume and seek (`WaveformAudioPlayer`).
//! Tracks also get waveforms, decoded off the UI thread. The output device opens on first use;
//! files download in the background.
use std::collections::HashMap;
use std::io::Cursor;
use std::sync::Arc;
use std::sync::mpsc::{Receiver, Sender, channel};
use std::time::Duration;
use log::warn;
use rodio::{Decoder, OutputStream, OutputStreamHandle, Sink, Source};
use tokio::sync::Semaphore;
/// The webapp's reference preview volume.
const PREVIEW_VOLUME: f32 = 0.2;
/// Peaks kept per waveform (resampled to however many bars are drawn).
const PEAK_COUNT: usize = 256;
/// Frames folded into one peak while decoding.
const WINDOW_FRAMES: usize = 1024;
/// Waveforms decoding at once (each downloads its whole file).
const MAX_CONCURRENT_WAVEFORMS: usize = 2;
pub struct AudioPlayer {
rt: tokio::runtime::Handle,
http: reqwest::Client,
/// Opened on first play (and kept: closing it would stop the sound).
output: Option<(OutputStream, OutputStreamHandle)>,
sink: Option<Sink>,
/// What's playing (or loading).
current: Option<Clip>,
tx: Sender<Loaded>,
rx: Receiver<Loaded>,
waveforms: HashMap<String, WaveformLookup>,
limiter: Arc<Semaphore>,
/// The tracks' volume slider, 0–1.
track_volume: f32,
}
/// A track's shape: peaks (0–1, loudest = 1) and its length in seconds.
#[derive(Debug, PartialEq)]
pub struct Waveform {
pub peaks: Vec<f32>,
pub duration: f32,
}
/// What [`AudioPlayer::waveform`] knows about a track.
#[derive(Clone)]
pub enum WaveformLookup {
Loading,
Ready(Arc<Waveform>),
Failed,
}
/// Where the current track is.
#[derive(Clone, Copy, Debug, PartialEq)]
pub struct Playback {
/// Still downloading.
pub loading: bool,
pub paused: bool,
/// Seconds in.
pub position: f32,
}
/// The clip that's playing, by the caller's key (a track's key is its URL).
struct Clip {
key: String,
track: bool,
/// Where to start once it has downloaded (a seek before it loaded).
start_at: f32,
}
enum Loaded {
Clip { key: String, bytes: Result<Arc<[u8]>, String> },
Waveform { url: String, waveform: Option<Waveform> },
}
impl AudioPlayer {
pub fn new(rt: tokio::runtime::Handle, http: reqwest::Client) -> Self {
let (tx, rx) = channel();
Self { rt, http, output: None, sink: None, current: None, tx, rx, waveforms: HashMap::new(), limiter: Arc::new(Semaphore::new(MAX_CONCURRENT_WAVEFORMS)), track_volume: 1.0 }
}
/// Starts a reference clip under `key`, or stops it if it's what's playing.
pub fn toggle(&mut self, ctx: &egui::Context, key: &str, url: &str) {
if self.is_playing(key) {
self.stop();
} else {
self.start(ctx, key, url, false, 0.0);
}
}
/// Plays or pauses a feed track.
pub fn toggle_track(&mut self, ctx: &egui::Context, url: &str) {
match (&self.sink, self.is_playing(url)) {
(Some(sink), true) if sink.is_paused() => sink.play(),
(Some(sink), true) => sink.pause(),
// Still downloading: a second click cancels.
(None, true) => self.stop(),
_ => self.start(ctx, url, url, true, 0.0),
}
}
/// Jumps a feed track to `secs` (starting it there if it isn't the current one).
pub fn seek_track(&mut self, ctx: &egui::Context, url: &str, secs: f32) {
if !self.is_playing(url) {
self.start(ctx, url, url, true, secs);
return;
}
match &self.sink {
Some(sink) => {
if let Err(err) = sink.try_seek(Duration::from_secs_f32(secs.max(0.0))) {
warn!("Audio seek failed: {err}");
}
sink.play();
},
None => {
if let Some(clip) = &mut self.current {
clip.start_at = secs;
}
},
}
}
pub fn stop(&mut self) {
if let Some(sink) = self.sink.take() {
sink.stop();
}
self.current = None;
}
/// Whether `key` is playing, paused or about to play.
pub fn is_playing(&self, key: &str) -> bool {
self.current.as_ref().is_some_and(|c| c.key == key)
}
/// Where the track at `url` is, when it's the current one.
pub fn track(&self, url: &str) -> Option<Playback> {
let clip = self.current.as_ref().filter(|c| c.key == url)?;
Some(match &self.sink {
Some(sink) => Playback { loading: false, paused: sink.is_paused(), position: sink.get_pos().as_secs_f32() },
None => Playback { loading: true, paused: false, position: clip.start_at },
})
}
pub fn track_volume(&self) -> f32 {
self.track_volume
}
pub fn set_track_volume(&mut self, volume: f32) {
self.track_volume = volume.clamp(0.0, 1.0);
if let (Some(sink), Some(clip)) = (&self.sink, &self.current) {
if clip.track {
sink.set_volume(self.track_volume);
}
}
}
/// The waveform of the track at `url`, downloading and decoding it the first time it's asked
/// for.
pub fn waveform(&mut self, ctx: &egui::Context, url: &str) -> WaveformLookup {
if let Some(lookup) = self.waveforms.get(url) {
return lookup.clone();
}
self.waveforms.insert(url.to_owned(), WaveformLookup::Loading);
let (http, limiter, tx, ctx, url) = (self.http.clone(), self.limiter.clone(), self.tx.clone(), ctx.clone(), url.to_owned());
self.rt.spawn(async move {
let waveform = match limiter.acquire_owned().await {
Ok(_permit) => match download(&http, &url).await {
Ok(bytes) => tokio::task::spawn_blocking(move || decode_waveform(bytes)).await.ok().flatten(),
Err(err) => {
warn!("Waveform download failed for {url}: {err}");
None
},
},
Err(_) => None,
};
let _ = tx.send(Loaded::Waveform { url, waveform });
ctx.request_repaint();
});
WaveformLookup::Loading
}
/// Starts downloads that finished and notices clips that ended. Call once per frame. Returns an
/// error message when a clip couldn't play.
pub fn poll(&mut self) -> Option<String> {
let mut error = None;
while let Ok(loaded) = self.rx.try_recv() {
match loaded {
Loaded::Waveform { url, waveform } => {
self.waveforms.insert(url, waveform.map_or(WaveformLookup::Failed, |w| WaveformLookup::Ready(Arc::new(w))));
},
Loaded::Clip { key, bytes } => {
// Superseded by another clip (or stopped) while downloading.
let Some(clip) = self.current.as_ref().filter(|c| c.key == key) else {
continue;
};
let (volume, start_at) = (if clip.track { self.track_volume } else { PREVIEW_VOLUME }, clip.start_at);
if let Err(err) = bytes.and_then(|b| self.play_bytes(b, volume, start_at)) {
warn!("Audio playback failed: {err}");
self.current = None;
error = Some(format!("Couldn't play the audio: {err}"));
}
},
}
}
if self.sink.as_ref().is_some_and(Sink::empty) {
self.sink = None;
self.current = None;
}
error
}
fn start(&mut self, ctx: &egui::Context, key: &str, url: &str, track: bool, start_at: f32) {
self.stop();
self.current = Some(Clip { key: key.to_owned(), track, start_at });
let (http, tx, ctx, key, url) = (self.http.clone(), self.tx.clone(), ctx.clone(), key.to_owned(), url.to_owned());
self.rt.spawn(async move {
let bytes = download(&http, &url).await;
let _ = tx.send(Loaded::Clip { key, bytes });
ctx.request_repaint();
});
}
fn play_bytes(&mut self, bytes: Arc<[u8]>, volume: f32, start_at: f32) -> Result<(), String> {
if self.output.is_none() {
self.output = Some(OutputStream::try_default().map_err(|e| e.to_string())?);
}
let Some((_, handle)) = &self.output else {
return Err("no audio output".to_owned());
};
let sink = Sink::try_new(handle).map_err(|e| e.to_string())?;
let source = Decoder::new(Cursor::new(bytes)).map_err(|e| e.to_string())?;
sink.set_volume(volume);
sink.append(source);
if start_at > 0.0 {
if let Err(err) = sink.try_seek(Duration::from_secs_f32(start_at)) {
warn!("Audio seek failed: {err}");
}
}
self.sink = Some(sink);
Ok(())
}
}
async fn download(http: &reqwest::Client, url: &str) -> Result<Arc<[u8]>, String> {
let response = http.get(url).send().await.and_then(|r| r.error_for_status()).map_err(|e| e.to_string())?;
response.bytes().await.map(|b| Arc::from(b.as_ref())).map_err(|e| e.to_string())
}
/// Decodes a whole file into its waveform: the loudest sample of each window, folded down to
/// [`PEAK_COUNT`] peaks and scaled so the loudest is 1.
fn decode_waveform(bytes: Arc<[u8]>) -> Option<Waveform> {
let decoder = match Decoder::new(Cursor::new(bytes)) {
Ok(d) => d,
Err(err) => {
warn!("Waveform decode failed: {err}");
return None;
},
};
let (channels, rate) = (usize::from(decoder.channels().max(1)), decoder.sample_rate().max(1));
let window = WINDOW_FRAMES * channels;
let (mut windows, mut loudest, mut count) = (Vec::new(), 0_u16, 0_usize);
for sample in decoder {
loudest = loudest.max(sample.unsigned_abs());
count += 1;
if count % window == 0 {
windows.push(f32::from(loudest));
loudest = 0;
}
}
if count % window != 0 {
windows.push(f32::from(loudest));
}
if count == 0 {
return None;
}
Some(Waveform { peaks: normalized(&resample(&windows, PEAK_COUNT)), duration: (count / channels) as f32 / rate as f32 })
}
/// `values` folded (by maximum) or stretched to exactly `count` entries.
pub fn resample(values: &[f32], count: usize) -> Vec<f32> {
if values.is_empty() || count == 0 {
return vec![0.0; count];
}
(0..count)
.map(|i| {
let start = i * values.len() / count;
let end = ((i + 1) * values.len() / count).max(start + 1).min(values.len());
values[start..end].iter().copied().fold(0.0, f32::max)
})
.collect()
}
fn normalized(values: &[f32]) -> Vec<f32> {
let max = values.iter().copied().fold(0.0, f32::max);
if max <= 0.0 {
return values.to_vec();
}
values.iter().map(|v| v / max).collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn resampling_keeps_the_loudest_of_each_span() {
assert_eq!(resample(&[0.1, 0.9, 0.2, 0.4], 2), [0.9, 0.4]);
assert_eq!(resample(&[0.5, 1.0], 4), [0.5, 0.5, 1.0, 1.0], "short inputs stretch");
assert_eq!(resample(&[], 3), [0.0, 0.0, 0.0]);
assert_eq!(normalized(&[0.0, 2.0, 1.0]), [0.0, 1.0, 0.5]);
}
#[test]
fn decodes_a_wav_into_peaks_and_duration() {
// One second of 8 kHz mono: silence, then a loud half.
let rate = 8000_u32;
let samples: Vec<i16> = (0..rate).map(|i| if i < rate / 2 { 0 } else { 16000 }).collect();
let mut wav = Vec::new();
wav.extend_from_slice(b"RIFF");
wav.extend_from_slice(&(36 + samples.len() as u32 * 2).to_le_bytes());
wav.extend_from_slice(b"WAVEfmt ");
wav.extend_from_slice(&16_u32.to_le_bytes());
wav.extend_from_slice(&1_u16.to_le_bytes());
wav.extend_from_slice(&1_u16.to_le_bytes());
wav.extend_from_slice(&rate.to_le_bytes());
wav.extend_from_slice(&(rate * 2).to_le_bytes());
wav.extend_from_slice(&2_u16.to_le_bytes());
wav.extend_from_slice(&16_u16.to_le_bytes());
wav.extend_from_slice(b"data");
wav.extend_from_slice(&(samples.len() as u32 * 2).to_le_bytes());
for s in &samples {
wav.extend_from_slice(&s.to_le_bytes());
}
let waveform = decode_waveform(Arc::from(wav)).expect("decodes");
assert!((waveform.duration - 1.0).abs() < 0.01, "duration {}", waveform.duration);
assert_eq!(waveform.peaks.len(), PEAK_COUNT);
assert_eq!(waveform.peaks[0], 0.0, "starts silent");
assert_eq!(*waveform.peaks.last().unwrap(), 1.0, "ends loud");
}
}
+132 -41
View File
@@ -1,14 +1,18 @@
//! Images shown in the UI: CDN thumbnails and local previews, as GPU textures.
//! Images shown in the UI: CDN thumbnails, animated video previews and local previews, as GPU
//! textures.
//!
//! Downloads run on the tokio runtime (a few at a time) and decode off the UI thread; the UI
//! uploads finished images in [`MediaCache::poll`]. Textures nobody has drawn for a while are
//! dropped once the cache grows past its budget, so long feeds don't hold every image forever.
use std::collections::HashMap;
use std::io::Cursor;
use std::sync::Arc;
use std::sync::mpsc::{Receiver, Sender, channel};
use std::time::Duration;
use egui::{ColorImage, TextureHandle, TextureOptions};
use image::AnimationDecoder;
use log::warn;
use tokio::runtime::Handle;
use tokio::sync::Semaphore;
@@ -17,10 +21,14 @@ use tokio::sync::Semaphore;
const MAX_CONCURRENT_DOWNLOADS: usize = 6;
/// Longest side of a decoded image; larger ones are downscaled before upload.
const MAX_DECODED_SIDE: u32 = 2048;
/// Animated previews: longest side and frame budget (each frame is its own texture).
const ANIMATED_SIDE: u32 = 320;
const ANIMATED_MAX_FRAMES: usize = 48;
/// Textures kept even when unused (roughly two screens of thumbnails).
const SOFT_CAPACITY: usize = 160;
/// Frames an image may go undrawn before it can be evicted.
/// Frames an image may go undrawn before it can be evicted (animations go sooner: they're heavy).
const EVICT_AFTER_FRAMES: u64 = 600;
const EVICT_ANIMATED_AFTER_FRAMES: u64 = 120;
pub struct MediaCache {
rt: Handle,
@@ -40,14 +48,30 @@ pub enum Lookup {
}
enum Entry {
Loading { last_used: u64 },
Ready { texture: TextureHandle, last_used: u64 },
Loading {
last_used: u64,
},
Ready {
texture: TextureHandle,
last_used: u64,
},
/// Frames with how long each shows (seconds), and their total.
Animated {
frames: Vec<(TextureHandle, f32)>,
total: f32,
last_used: u64,
},
Failed,
}
struct Loaded {
key: String,
image: Option<ColorImage>,
decoded: Option<Decoded>,
}
enum Decoded {
Still(ColorImage),
Frames(Vec<(ColorImage, f32)>),
}
impl MediaCache {
@@ -58,23 +82,13 @@ impl MediaCache {
/// The texture for `url`, starting its download the first time it's asked for.
pub fn get(&mut self, ctx: &egui::Context, url: &str) -> Lookup {
let frame = self.frame;
match self.entries.get_mut(url) {
Some(Entry::Ready { texture, last_used }) => {
*last_used = frame;
Lookup::Ready(texture.clone())
},
Some(Entry::Loading { last_used }) => {
*last_used = frame;
Lookup::Loading
},
Some(Entry::Failed) => Lookup::Failed,
None => {
self.entries.insert(url.to_owned(), Entry::Loading { last_used: frame });
self.spawn_download(ctx, url.to_owned());
Lookup::Loading
},
}
self.lookup(ctx, url, url, false)
}
/// The current frame of an animated image (a video's animated preview); still images come
/// back as they are.
pub fn get_animated(&mut self, ctx: &egui::Context, url: &str) -> Lookup {
self.lookup(ctx, &format!("anim:{url}"), url, true)
}
/// Decodes `bytes` off the UI thread and registers the result under `key`.
@@ -85,8 +99,8 @@ impl MediaCache {
self.entries.insert(key.to_owned(), Entry::Loading { last_used: self.frame });
let (tx, ctx, key) = (self.tx.clone(), ctx.clone(), key.to_owned());
self.rt.spawn_blocking(move || {
let image = decode(&bytes);
let _ = tx.send(Loaded { key, image });
let decoded = decode(&bytes).map(Decoded::Still);
let _ = tx.send(Loaded { key, decoded });
ctx.request_repaint();
});
}
@@ -94,39 +108,78 @@ impl MediaCache {
/// Uploads finished downloads and evicts stale textures. Call once per frame.
pub fn poll(&mut self, ctx: &egui::Context) {
self.frame += 1;
while let Ok(Loaded { key, image }) = self.rx.try_recv() {
while let Ok(Loaded { key, decoded }) = self.rx.try_recv() {
// Dropped while loading (evicted): don't resurrect it.
let Some(entry) = self.entries.get_mut(&key) else {
continue;
};
*entry = match image {
Some(image) => Entry::Ready { texture: ctx.load_texture(&key, image, TextureOptions::LINEAR), last_used: self.frame },
*entry = match decoded {
Some(Decoded::Still(image)) => Entry::Ready { texture: ctx.load_texture(&key, image, TextureOptions::LINEAR), last_used: self.frame },
Some(Decoded::Frames(frames)) => {
let total = frames.iter().map(|(_, d)| d).sum();
let frames = frames.into_iter().enumerate().map(|(i, (img, d))| (ctx.load_texture(format!("{key}#{i}"), img, TextureOptions::LINEAR), d)).collect();
Entry::Animated { frames, total, last_used: self.frame }
},
None => Entry::Failed,
};
}
if self.entries.len() > SOFT_CAPACITY {
let frame = self.frame;
self.entries.retain(|_, e| match e {
Entry::Ready { last_used, .. } | Entry::Loading { last_used } => frame.saturating_sub(*last_used) < EVICT_AFTER_FRAMES,
Entry::Failed => true,
});
let frame = self.frame;
let crowded = self.entries.len() > SOFT_CAPACITY;
self.entries.retain(|_, e| match e {
Entry::Animated { last_used, .. } => frame.saturating_sub(*last_used) < EVICT_ANIMATED_AFTER_FRAMES,
Entry::Ready { last_used, .. } | Entry::Loading { last_used } => !crowded || frame.saturating_sub(*last_used) < EVICT_AFTER_FRAMES,
Entry::Failed => true,
});
}
fn lookup(&mut self, ctx: &egui::Context, key: &str, url: &str, animated: bool) -> Lookup {
let frame = self.frame;
match self.entries.get_mut(key) {
Some(Entry::Ready { texture, last_used }) => {
*last_used = frame;
Lookup::Ready(texture.clone())
},
Some(Entry::Animated { frames, total, last_used }) => {
*last_used = frame;
let t = (ctx.input(|i| i.time) as f32).rem_euclid(total.max(0.01));
let (mut acc, mut index) = (0.0, frames.len() - 1);
for (i, (_, d)) in frames.iter().enumerate() {
acc += d;
if t < acc {
index = i;
break;
}
}
ctx.request_repaint_after(Duration::from_secs_f32((acc - t).max(0.01)));
Lookup::Ready(frames[index].0.clone())
},
Some(Entry::Loading { last_used }) => {
*last_used = frame;
Lookup::Loading
},
Some(Entry::Failed) => Lookup::Failed,
None => {
self.entries.insert(key.to_owned(), Entry::Loading { last_used: frame });
self.spawn_download(ctx, key.to_owned(), url.to_owned(), animated);
Lookup::Loading
},
}
}
fn spawn_download(&self, ctx: &egui::Context, url: String) {
fn spawn_download(&self, ctx: &egui::Context, key: String, url: String, animated: bool) {
let (http, limiter, tx, ctx) = (self.http.clone(), self.limiter.clone(), self.tx.clone(), ctx.clone());
self.rt.spawn(async move {
let image = match limiter.acquire_owned().await {
Ok(_permit) => download(&http, &url).await,
let decoded = match limiter.acquire_owned().await {
Ok(_permit) => download(&http, &url, animated).await,
Err(_) => None,
};
let _ = tx.send(Loaded { key: url, image });
let _ = tx.send(Loaded { key, decoded });
ctx.request_repaint();
});
}
}
async fn download(http: &reqwest::Client, url: &str) -> Option<ColorImage> {
async fn download(http: &reqwest::Client, url: &str, animated: bool) -> Option<Decoded> {
let response = match http.get(url).send().await.and_then(|r| r.error_for_status()) {
Ok(response) => response,
Err(err) => {
@@ -135,7 +188,7 @@ async fn download(http: &reqwest::Client, url: &str) -> Option<ColorImage> {
},
};
let bytes = response.bytes().await.ok()?;
tokio::task::spawn_blocking(move || decode(&bytes)).await.ok().flatten()
tokio::task::spawn_blocking(move || if animated { decode_frames(&bytes).map(Decoded::Frames).or_else(|| decode(&bytes).map(Decoded::Still)) } else { decode(&bytes).map(Decoded::Still) }).await.ok().flatten()
}
fn decode(bytes: &[u8]) -> Option<ColorImage> {
@@ -147,7 +200,45 @@ fn decode(bytes: &[u8]) -> Option<ColorImage> {
},
};
let image = if image.width().max(image.height()) > MAX_DECODED_SIDE { image.thumbnail(MAX_DECODED_SIDE, MAX_DECODED_SIDE) } else { image };
let rgba = image.to_rgba8();
Some(to_color_image(image.to_rgba8()))
}
/// The frames of an animated GIF or WebP (subsampled to the budget), or `None` when it isn't
/// animated.
fn decode_frames(bytes: &[u8]) -> Option<Vec<(ColorImage, f32)>> {
let frames = match image::guess_format(bytes).ok()? {
image::ImageFormat::Gif => image::codecs::gif::GifDecoder::new(Cursor::new(bytes)).ok()?.into_frames().collect_frames().ok()?,
image::ImageFormat::WebP => {
let decoder = image::codecs::webp::WebPDecoder::new(Cursor::new(bytes)).ok()?;
if !decoder.has_animation() {
return None;
}
decoder.into_frames().collect_frames().ok()?
},
_ => return None,
};
if frames.len() < 2 {
return None;
}
// Keep every n-th frame, folding the skipped frames' time into the kept one.
let step = frames.len().div_ceil(ANIMATED_MAX_FRAMES);
let mut out = Vec::with_capacity(frames.len() / step + 1);
for chunk in frames.chunks(step) {
let secs: f32 = chunk
.iter()
.map(|f| {
let (n, d) = f.delay().numer_denom_ms();
n as f32 / d.max(1) as f32 / 1000.0
})
.sum();
let image = image::DynamicImage::ImageRgba8(chunk[0].buffer().clone()).thumbnail(ANIMATED_SIDE, ANIMATED_SIDE);
// Some encoders write 0 ms delays; browsers treat those as ~100 ms.
out.push((to_color_image(image.to_rgba8()), if secs < 0.02 { 0.1 * chunk.len() as f32 } else { secs }));
}
Some(out)
}
fn to_color_image(rgba: image::RgbaImage) -> ColorImage {
let size = [rgba.width() as usize, rgba.height() as usize];
Some(ColorImage::from_rgba_unmultiplied(size, rgba.as_raw()))
ColorImage::from_rgba_unmultiplied(size, rgba.as_raw())
}
+263 -11
View File
@@ -1,8 +1,11 @@
//! Everything that talks to the ArtCraft API. Calls run on a tokio runtime and report back as
//! [`Event`]s, which the UI drains once per frame; nothing here blocks the UI thread.
pub mod audio;
pub mod media_cache;
pub mod providers;
pub mod session;
pub mod video;
pub mod wire;
use std::future::Future;
@@ -27,6 +30,7 @@ use artcraft_client::error::storyteller_error::StorytellerError;
use artcraft_client::utils::api_host::ApiHost;
use artcraft_client::utils::basic_json_post_request::basic_json_post_request;
use artcraft_client::utils::login_challenge_client::{LoginChallengeClient, LoginChallengeClientError};
use cookie_store_wrapper::cookie_store::CookieStore;
use enums::common::payments_namespace::PaymentsNamespace;
use log::{error, warn};
use serde::de::DeserializeOwned;
@@ -35,8 +39,13 @@ use tokens::tokens::media_files::MediaFileToken;
use crate::feed::types::MediaKind;
use crate::models::{self, ModelInfo};
use providers::Provider;
use providers::accounts::ProviderAccounts;
use providers::jobs::{JobStore, ProviderJob};
use providers::submit::SubmitError;
use providers::worker::Worker;
use session::Credentials;
use wire::{BatchMedia, MediaFile, PromptResponse, SessionInfo, SessionJobs, SessionUser, UploadResponse, UserMediaList};
use wire::{AudioModels, BatchMedia, CharactersPage, MediaFile, PromptResponse, SessionInfo, SessionJobs, SessionUser, UploadResponse, UserMediaList};
/// Items per library page (`useGalleryData`'s `PAGE_SIZE`).
pub const LIBRARY_PAGE_SIZE: u32 = 40;
@@ -45,6 +54,10 @@ const MODEL_LIST_ATTEMPTS: u32 = 3;
const MODEL_LIST_BACKOFF: Duration = Duration::from_millis(250);
/// The share link base (`SHARE_URL_BASE`).
pub const SHARE_URL_BASE: &str = "https://getartcraft.com/media/";
/// fal's cost estimate (the provider-aware legacy endpoint; OmniGen only prices ArtCraft).
const PROVIDER_COST_PATH: &str = "/v1/generate/cost_estimate/image";
/// Characters are made for this model (the Tauri app's `CharactersModal`).
const CHARACTER_MODEL: &str = "seedance_2p0";
/// Sent on every request (`artcraft_client_identity::ARTCRAFT_DESKTOP_USER_AGENT`).
const USER_AGENT: &str = "storyteller-client/1.0";
@@ -53,6 +66,7 @@ const USER_AGENT: &str = "storyteller-client/1.0";
pub enum Modality {
Image,
Video,
Audio,
}
impl Modality {
@@ -60,6 +74,7 @@ impl Modality {
match self {
Modality::Image => "image",
Modality::Video => "video",
Modality::Audio => "audio",
}
}
}
@@ -69,6 +84,7 @@ impl From<MediaKind> for Modality {
match kind {
MediaKind::Image => Modality::Image,
MediaKind::Video => Modality::Video,
MediaKind::Audio => Modality::Audio,
}
}
}
@@ -98,6 +114,8 @@ pub enum Event {
EnqueueFailed {
modality: Modality,
message: String,
/// The HTTP status, when the server answered.
status: Option<u16>,
},
Uploaded {
ref_id: u64,
@@ -119,6 +137,27 @@ pub enum Event {
items: Vec<MediaFile>,
},
Prompt(wire::Prompt),
/// The user's characters, newest first.
Characters(Vec<wire::Character>),
/// A character is being made by job `job_token`.
CharacterCreating {
job_token: String,
name: String,
},
CharacterChanged,
/// Generating with `Provider` needs its account connected first.
ProviderSetupNeeded(Provider),
/// A Midjourney or fal job's results are in the library, as `batch_token`.
ProviderJobFinished {
job_id: String,
batch_token: String,
},
ProviderJobFailed {
job_id: String,
reason: String,
},
/// A provider account was connected or disconnected.
ProviderAccountsChanged,
Deleted(String),
Saved(PathBuf),
Toast {
@@ -134,6 +173,8 @@ pub struct EnqueueMeta {
pub model_id: String,
pub batch_count: u32,
pub ref_image: Option<String>,
/// Who runs it (other providers' jobs are tracked locally).
pub provider: Provider,
}
/// A finished upload: the media token, plus what the server knows about the file.
@@ -155,6 +196,8 @@ pub struct Backend {
creds: Arc<RwLock<Option<Credentials>>>,
http: reqwest::Client,
device_login_cancel: Arc<AtomicBool>,
accounts: ProviderAccounts,
jobs: JobStore,
}
impl Backend {
@@ -165,7 +208,10 @@ impl Backend {
let host = session::api_host(&root);
let creds = session::load(&root);
let http = reqwest::Client::builder().user_agent(USER_AGENT).gzip(true).timeout(Duration::from_secs(60)).build().unwrap_or_default();
Self { rt, ctx, tx, rx, host, root, creds: Arc::new(RwLock::new(creds)), http, device_login_cancel: Arc::new(AtomicBool::new(false)) }
let (accounts, jobs) = (ProviderAccounts::load(&root), JobStore::load(&root));
let creds = Arc::new(RwLock::new(creds));
rt.spawn(Worker { host: host.clone(), creds: creds.clone(), http: http.clone(), accounts: accounts.clone(), jobs: jobs.clone(), tx: tx.clone(), ctx: ctx.clone() }.run());
Self { rt, ctx, tx, rx, host, root, creds, http, device_login_cancel: Arc::new(AtomicBool::new(false)), accounts, jobs }
}
pub fn runtime(&self) -> tokio::runtime::Handle {
@@ -322,7 +368,8 @@ impl Backend {
/// Fetches the OmniGen model listing for `modality`.
pub fn load_models(&self, modality: Modality) {
let host = self.host.clone();
let api = self.api();
let host = api.host.clone();
self.spawn(move |tx| async move {
let mut last_err = String::new();
for attempt in 0..MODEL_LIST_ATTEMPTS {
@@ -330,15 +377,16 @@ impl Backend {
tokio::time::sleep(MODEL_LIST_BACKOFF).await;
}
let result = match modality {
Modality::Image => omni_gen_list_image_models(OmniGenListImageModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::image_model).collect::<Vec<_>>()),
Modality::Video => omni_gen_list_video_models(OmniGenListVideoModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::video_model).collect::<Vec<_>>()),
Modality::Image => omni_gen_list_image_models(OmniGenListImageModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::image_model).collect::<Vec<_>>()).map_err(|e| api_message(&e)),
Modality::Video => omni_gen_list_video_models(OmniGenListVideoModelsArgs { api_host: &host, maybe_creds: None, provider: None }).await.map(|r| r.models.iter().map(models::video_model).collect::<Vec<_>>()).map_err(|e| api_message(&e)),
Modality::Audio => api.get_json::<AudioModels>("/v1/omni_gen/models/audio").await.map(|r| r.models.iter().map(models::audio_model).collect::<Vec<_>>()),
};
match result {
Ok(models) => {
let _ = tx.send(Event::Models(modality, Ok(models)));
return;
},
Err(err) => last_err = api_message(&err),
Err(err) => last_err = err,
}
}
error!("Listing {modality:?} models failed: {last_err}");
@@ -367,7 +415,7 @@ impl Backend {
let api = self.api();
self.spawn(move |tx| async move {
let Some(creds) = api.creds.as_ref() else {
let _ = tx.send(Event::EnqueueFailed { modality, message: "Please sign in to generate.".to_owned() });
let _ = tx.send(Event::EnqueueFailed { modality, message: "Please sign in to generate.".to_owned(), status: None });
return;
};
fields.entry("idempotency_token").or_insert_with(|| Value::String(uuid::Uuid::new_v4().to_string()));
@@ -376,19 +424,95 @@ impl Backend {
Ok(response) => {
let tokens = job_tokens(&response);
if tokens.is_empty() {
let _ = tx.send(Event::EnqueueFailed { modality, message: "The server didn't start a job.".to_owned() });
let _ = tx.send(Event::EnqueueFailed { modality, message: "The server didn't start a job.".to_owned(), status: None });
} else {
let _ = tx.send(Event::Enqueued { modality, job_tokens: tokens, meta });
}
api.send_credits(&tx).await;
},
Err(err) => {
let _ = tx.send(Event::EnqueueFailed { modality, message: api_message(&err) });
let _ = tx.send(Event::EnqueueFailed { modality, message: api_message(&err), status: api_status(&err) });
},
}
});
}
// --- Other providers ---------------------------------------------------------------------
/// Starts an image generation at Midjourney or fal; `image_urls` are the references' links.
pub fn generate_with_provider(&self, provider: Provider, fields: Map<String, Value>, image_urls: Vec<String>, meta: EnqueueMeta) {
let (api, accounts, jobs) = (self.api(), self.accounts.clone(), self.jobs.clone());
self.spawn(move |tx| async move {
match providers::submit::submit(&api.host, api.creds.as_ref(), &accounts, provider, &fields, image_urls).await {
Ok(job) => {
let id = job.id.clone();
jobs.put(job);
let _ = tx.send(Event::Enqueued { modality: Modality::Image, job_tokens: vec![id], meta });
},
Err(SubmitError::NeedsAccount(provider)) => {
let _ = tx.send(Event::ProviderSetupNeeded(provider));
},
Err(SubmitError::Message(message)) => {
let _ = tx.send(Event::EnqueueFailed { modality: Modality::Image, message, status: None });
},
}
});
}
/// What an image generation costs at `provider`: nothing extra for direct Midjourney (it uses
/// the user's subscription), fal's price from the server.
pub fn estimate_provider_cost(&self, provider: Provider, key: String, fields: Map<String, Value>) {
match provider {
Provider::Artcraft => self.estimate_cost(Modality::Image, key, fields),
Provider::Midjourney => {
let _ = self.tx.send(Event::Cost { key, credits: Some(0) });
},
Provider::Fal => {
let api = self.api();
self.spawn(move |tx| async move {
let credits = match basic_json_post_request::<_, Value>(&api.host, PROVIDER_COST_PATH, None, fal_cost_body(&fields)).await {
Ok(response) => response.get("cost_in_credits").and_then(Value::as_u64),
Err(err) => {
warn!("FAL cost estimate failed: {}", api_message(&err));
None
},
};
let _ = tx.send(Event::Cost { key, credits });
});
},
}
}
pub fn accounts(&self) -> &ProviderAccounts {
&self.accounts
}
/// Midjourney and fal jobs still running (they get pending cards at startup).
pub fn provider_jobs(&self) -> Vec<ProviderJob> {
self.jobs.all()
}
/// Keeps the Midjourney login captured by the sign-in window.
pub fn save_midjourney_login(&self, cookies: CookieStore) {
let accounts = self.accounts.clone();
self.spawn(move |tx| async move {
if let Err(err) = accounts.midjourney.save_login(cookies).await {
let _ = tx.send(Event::Toast { error: true, message: format!("Couldn't save the Midjourney login: {err}") });
}
let _ = tx.send(Event::ProviderAccountsChanged);
});
}
pub fn disconnect_midjourney(&self) {
let accounts = self.accounts.clone();
self.spawn(move |tx| async move {
if let Err(err) = accounts.midjourney.clear_credentials().await {
let _ = tx.send(Event::Toast { error: true, message: format!("Couldn't disconnect Midjourney: {err}") });
}
let _ = tx.send(Event::ProviderAccountsChanged);
});
}
// --- Uploads -----------------------------------------------------------------------------
/// Uploads a reference file from disk; images, MP4 videos and audio.
@@ -485,6 +609,92 @@ impl Backend {
});
}
/// Loads every page of the user's characters.
pub fn load_characters(&self) {
let api = self.api();
if api.creds.is_none() {
return;
}
self.spawn(move |tx| async move {
let mut all = Vec::new();
let mut cursor: Option<i64> = None;
loop {
let path = match cursor {
Some(c) => format!("/v1/characters/session?cursor={c}"),
None => "/v1/characters/session".to_owned(),
};
match api.request_json::<CharactersPage>(reqwest::Method::GET, &path, None).await {
Ok(page) => {
all.extend(page.characters);
match page.next_cursor {
Some(next) if Some(next) != cursor => cursor = Some(next),
_ => break,
}
},
Err(err) => {
warn!("Loading characters failed: {err}");
break;
},
}
}
let _ = tx.send(Event::Characters(all));
});
}
/// Starts making a character from an uploaded reference image (`POST /v1/character/create`).
pub fn create_character(&self, image_token: String, name: String, description: Option<String>) {
let api = self.api();
self.spawn(move |tx| async move {
let body = serde_json::json!({ "image_media_token": image_token, "model": CHARACTER_MODEL, "uuid_idempotency_token": uuid::Uuid::new_v4().to_string(), "character_name": name, "character_description": description });
match api.request_json::<Value>(reqwest::Method::POST, "/v1/character/create", Some(body)).await {
Ok(r) => match r.get("inference_job_token").and_then(Value::as_str) {
Some(job) => {
let _ = tx.send(Event::CharacterCreating { job_token: job.to_owned(), name });
},
None => {
let _ = tx.send(Event::Toast { error: true, message: "Failed to create character".to_owned() });
},
},
Err(err) => {
let _ = tx.send(Event::Toast { error: true, message: if err.is_empty() { "Failed to create character".to_owned() } else { err } });
},
}
});
}
/// Renames or re-describes a character (`POST /v1/character/edit`).
pub fn edit_character(&self, token: String, name: String, description: String) {
let api = self.api();
self.spawn(move |tx| async move {
let body = serde_json::json!({ "token": token, "updated_name": name, "updated_description": description });
match api.request_json::<Value>(reqwest::Method::POST, "/v1/character/edit", Some(body)).await {
Ok(_) => {
let _ = tx.send(Event::Toast { error: false, message: "Character updated".to_owned() });
let _ = tx.send(Event::CharacterChanged);
},
Err(err) => {
let _ = tx.send(Event::Toast { error: true, message: format!("Failed to update character: {err}") });
},
}
});
}
/// `DELETE /v1/character/{token}`.
pub fn delete_character(&self, token: String, name: String) {
let api = self.api();
self.spawn(move |tx| async move {
match api.request_json::<Value>(reqwest::Method::DELETE, &format!("/v1/character/{token}"), None).await {
Ok(_) => {
let _ = tx.send(Event::Toast { error: false, message: format!("Character \"{name}\" deleted") });
let _ = tx.send(Event::CharacterChanged);
},
Err(err) => {
let _ = tx.send(Event::Toast { error: true, message: format!("Failed to delete character: {err}") });
},
}
});
}
pub fn delete_media(&self, token: String) {
let api = self.api();
self.spawn(move |tx| async move {
@@ -589,8 +799,16 @@ struct Api {
impl Api {
/// A GET with the desktop app's identity headers and cookies, decoded leniently.
async fn get_json<T: DeserializeOwned>(&self, path: &str) -> Result<T, String> {
self.request_json(reqwest::Method::GET, path, None).await
}
/// Any JSON call with the desktop app's identity headers and cookies, decoded leniently.
async fn request_json<T: DeserializeOwned>(&self, method: reqwest::Method, path: &str, body: Option<Value>) -> Result<T, String> {
let url = format!("{}{path}", self.host.to_api_hostname_and_scheme());
let mut request = self.http.get(&url).header("Accept", "application/json").header("Origin", self.host.request_origin());
let mut request = self.http.request(method, &url).header("Accept", "application/json").header("Origin", self.host.request_origin());
if let Some(body) = body {
request = request.json(&body);
}
if let Some(cookie) = self.creds.as_ref().and_then(StorytellerCredentialSet::maybe_as_cookie_header) {
request = request.header("Cookie", cookie);
}
@@ -644,7 +862,7 @@ impl Api {
}
let info = self.get_json::<FileResponse>(&format!("/v1/media_files/file/{token}")).await.ok();
let file = info.map(|r| r.media_file);
UploadedMedia { duration_secs: file.as_ref().and_then(|f| f.maybe_duration_millis).map_or(0.0, |ms| ms as f32 / 1000.0), thumbnail: file.as_ref().and_then(|f| f.media_links.thumbnail(256)), full_url: file.as_ref().map(|f| f.media_links.cdn_url.clone()).filter(|u| !u.is_empty()), token }
UploadedMedia { duration_secs: file.as_ref().and_then(|f| f.maybe_duration_millis).map_or(0.0, |ms| ms as f32 / 1000.0), thumbnail: file.as_ref().and_then(|f| f.thumbnail(256)), full_url: file.as_ref().map(|f| f.media_links.cdn_url.clone()).filter(|u| !u.is_empty()), token }
}
}
@@ -662,6 +880,15 @@ fn session_from_set_cookie(header: &str) -> Option<String> {
cookie::Cookie::parse(header).ok().filter(|c| c.name() == "session").map(|c| c.value().to_owned())
}
/// The fal cost request (`estimate_image_cost_command`): the page's fields, its references
/// counted as the generation mode.
fn fal_cost_body(fields: &Map<String, Value>) -> Value {
let images = fields.get("image_media_tokens").and_then(Value::as_array).map_or(0, Vec::len);
let mode = if images == 0 { serde_json::json!({ "type": "text_to_image" }) } else { serde_json::json!({ "type": "image_edit", "count": images }) };
let field = |name: &str| fields.get(name).cloned().unwrap_or(Value::Null);
serde_json::json!({ "model": field("model"), "provider": "fal", "generation_mode": mode, "aspect_ratio": field("aspect_ratio"), "resolution": field("resolution"), "quality": field("quality"), "image_batch_count": field("image_batch_count") })
}
/// All job tokens a generate call created (`inference_job_token` plus `all_job_tokens`).
fn job_tokens(response: &Value) -> Vec<String> {
let mut tokens: Vec<String> = response.get("inference_job_token").and_then(Value::as_str).map(str::to_owned).into_iter().collect();
@@ -687,6 +914,24 @@ pub fn api_message(err: &StorytellerError) -> String {
}
}
/// The HTTP status behind a client error, when there was a response.
fn api_status(err: &StorytellerError) -> Option<u16> {
let StorytellerError::Api(api) = err else {
return None;
};
Some(match api {
ApiError::InvalidRequest(_) => 400,
ApiError::Unauthorized(_) => 401,
ApiError::PaymentRequired(_) => 402,
ApiError::Forbidden(_) => 403,
ApiError::NotFound(_) => 404,
ApiError::TooManyRequests(_) => 429,
ApiError::InternalServerError { .. } => 500,
ApiError::UncategorizedBadResponseWithStatusAndBody { status_code, .. } => status_code.as_u16(),
_ => return None,
})
}
fn api_error_message(err: &ApiError) -> String {
let body = match err {
ApiError::InvalidRequest(b) | ApiError::Unauthorized(b) | ApiError::PaymentRequired(b) | ApiError::Forbidden(b) | ApiError::NotFound(b) | ApiError::TooManyRequests(b) => Some(b.as_str()),
@@ -718,6 +963,13 @@ mod tests {
assert_eq!(job_tokens(&serde_json::json!({ "inference_job_token": "only" })), ["only"]);
}
#[test]
fn fal_costs_count_references_as_the_mode() {
let fields = serde_json::json!({ "model": "flux_1_dev", "prompt": "x", "aspect_ratio": "square", "image_batch_count": 2, "image_media_tokens": ["mf_estimate_0"] });
let body = fal_cost_body(fields.as_object().unwrap());
assert_eq!(body, serde_json::json!({ "model": "flux_1_dev", "provider": "fal", "generation_mode": { "type": "image_edit", "count": 1 }, "aspect_ratio": "square", "resolution": null, "quality": null, "image_batch_count": 2 }));
}
#[test]
fn reads_the_servers_error_message() {
assert_eq!(body_message(r#"{"success":false,"error_message":"Not enough credits"}"#).as_deref(), Some("Not enough credits"));
@@ -0,0 +1,93 @@
//! The provider accounts, in the same files as the Tauri app (`~/Artcraft/credentials`), so an
//! account connected in either app works in both: the Midjourney login
//! (`midjourney_cookies.toml`) and the fal API key (`fal.api_key.txt`).
use std::path::{Path, PathBuf};
use std::sync::Arc;
use log::warn;
use midjourney_client::credentials::credential_manager::MidjourneyCredentialManager;
const MIDJOURNEY_FILENAME: &str = "midjourney_cookies.toml";
/// The Tauri app's pre-TOML Midjourney state, removed on disconnect so it can't come back.
const MIDJOURNEY_LEGACY_FILENAME: &str = "midjourney_state.json";
const FAL_KEY_FILENAME: &str = "fal.api_key.txt";
#[derive(Clone)]
pub struct ProviderAccounts {
pub midjourney: MidjourneyCredentialManager,
fal_key_path: Arc<PathBuf>,
}
impl ProviderAccounts {
pub fn load(data_root: &Path) -> Self {
let dir = data_root.join("credentials");
if let Err(err) = std::fs::create_dir_all(&dir) {
warn!("Couldn't create {}: {err}", dir.display());
}
let (path, legacy) = (dir.join(MIDJOURNEY_FILENAME), vec![dir.join(MIDJOURNEY_LEGACY_FILENAME)]);
let midjourney = MidjourneyCredentialManager::load(path.clone(), legacy.clone()).unwrap_or_else(|err| {
warn!("Couldn't load the Midjourney login: {err}");
MidjourneyCredentialManager::new(path, legacy, None)
});
Self { midjourney, fal_key_path: Arc::new(dir.join(FAL_KEY_FILENAME)) }
}
/// The connected Midjourney account's email ("" when it's connected but unknown), or `None`.
pub fn midjourney_account(&self) -> Option<String> {
match (self.midjourney.maybe_copy_cookie_store(), self.midjourney.maybe_account_info()) {
(Ok(None), Ok(None)) | (Err(_), _) | (_, Err(_)) => None,
(_, Ok(info)) => Some(info.and_then(|i| i.maybe_email).unwrap_or_default()),
}
}
pub fn fal_key(&self) -> Option<String> {
std::fs::read_to_string(&*self.fal_key_path).ok().map(|k| k.trim().to_owned()).filter(|k| !k.is_empty())
}
/// The key as Settings shows it (`first6********`).
pub fn fal_key_hint(&self) -> Option<String> {
self.fal_key().map(|k| mask_key(&k))
}
pub fn set_fal_key(&self, key: &str) -> std::io::Result<()> {
std::fs::write(&*self.fal_key_path, key.trim())
}
pub fn clear_fal_key(&self) -> std::io::Result<()> {
match std::fs::remove_file(&*self.fal_key_path) {
Err(err) if err.kind() != std::io::ErrorKind::NotFound => Err(err),
_ => Ok(()),
}
}
}
/// `first6********`, or just the stars for short keys (`provider_list_command`).
fn mask_key(key: &str) -> String {
if key.chars().count() > 6 {
format!("{}********", key.chars().take(6).collect::<String>())
} else {
"********".to_owned()
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn fal_keys_are_trimmed_masked_and_cleared() {
let dir = std::env::temp_dir().join(format!("artcraft-native-accounts-{}", uuid::Uuid::new_v4()));
let accounts = ProviderAccounts::load(&dir);
assert_eq!(accounts.fal_key(), None);
accounts.set_fal_key(" abcdef123456:secret \n").unwrap();
assert_eq!(accounts.fal_key().as_deref(), Some("abcdef123456:secret"));
assert_eq!(accounts.fal_key_hint().as_deref(), Some("abcdef********"));
accounts.clear_fal_key().unwrap();
accounts.clear_fal_key().unwrap();
assert_eq!(accounts.fal_key(), None);
assert_eq!(mask_key("short"), "********");
assert_eq!(accounts.midjourney_account(), None);
let _ = std::fs::remove_dir_all(dir);
}
}
@@ -0,0 +1,122 @@
//! Generations running at Midjourney or fal, saved (`state/native_provider_jobs.json`) so they
//! finish even after a restart. Each also checkpoints its upload into the user's library: the
//! batch it goes into, one idempotency token per result, and what's uploaded so far.
use std::io::Write;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use log::warn;
use serde::{Deserialize, Serialize};
use super::Provider;
const FILENAME: &str = "native_provider_jobs.json";
/// One generation at another provider.
#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)]
pub struct ProviderJob {
/// Ours (`local_<uuid>`): the feed's pending card is keyed by it.
pub id: String,
pub provider: Provider,
pub model_id: String,
pub prompt: String,
/// Unix seconds.
pub created_at: i64,
pub batch_count: u32,
/// The provider's job: Midjourney's job id, or fal's request id.
pub provider_job_id: String,
/// fal's queue URLs.
pub maybe_status_url: Option<String>,
pub maybe_response_url: Option<String>,
/// The ArtCraft prompt the results are attributed to.
pub maybe_prompt_token: Option<String>,
/// The library batch the results go into.
pub batch_token: String,
/// Upload idempotency tokens, one per result (made as needed).
pub upload_tokens: Vec<String>,
/// Media tokens uploaded so far, in result order.
pub uploaded: Vec<String>,
}
impl ProviderJob {
/// The idempotency token for result `index`, making (and keeping) new ones as needed.
pub fn upload_token(&mut self, index: usize) -> String {
while self.upload_tokens.len() <= index {
self.upload_tokens.push(uuid::Uuid::new_v4().to_string());
}
self.upload_tokens[index].clone()
}
}
/// The saved jobs, shared by the submitter, the worker and the UI.
#[derive(Clone)]
pub struct JobStore {
path: Arc<PathBuf>,
jobs: Arc<Mutex<Vec<ProviderJob>>>,
}
impl JobStore {
pub fn load(data_root: &Path) -> Self {
let path = data_root.join("state").join(FILENAME);
let jobs = std::fs::read_to_string(&path).ok().and_then(|text| serde_json::from_str(&text).map_err(|err| warn!("Ignoring unreadable {}: {err}", path.display())).ok()).unwrap_or_default();
Self { path: Arc::new(path), jobs: Arc::new(Mutex::new(jobs)) }
}
pub fn all(&self) -> Vec<ProviderJob> {
self.jobs.lock().map(|j| j.clone()).unwrap_or_default()
}
/// Adds or replaces (by id) a job, then saves.
pub fn put(&self, job: ProviderJob) {
self.mutate(|jobs| match jobs.iter_mut().find(|j| j.id == job.id) {
Some(existing) => *existing = job,
None => jobs.push(job),
});
}
pub fn remove(&self, id: &str) {
self.mutate(|jobs| jobs.retain(|j| j.id != id));
}
fn mutate(&self, change: impl FnOnce(&mut Vec<ProviderJob>)) {
let Ok(mut jobs) = self.jobs.lock() else {
return;
};
change(&mut jobs);
if let Err(err) = save(&self.path, &jobs) {
warn!("Couldn't save {}: {err}", self.path.display());
}
}
}
/// Writes atomically, so a crash can't truncate the list.
fn save(path: &Path, jobs: &[ProviderJob]) -> anyhow::Result<()> {
let dir = path.parent().ok_or_else(|| anyhow::anyhow!("no state directory"))?;
std::fs::create_dir_all(dir)?;
let mut file = tempfile::NamedTempFile::new_in(dir)?;
file.write_all(serde_json::to_string_pretty(jobs)?.as_bytes())?;
file.persist(path)?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn jobs_survive_a_restart_and_keep_their_upload_tokens() {
let dir = std::env::temp_dir().join(format!("artcraft-native-jobs-{}", uuid::Uuid::new_v4()));
let store = JobStore::load(&dir);
let mut job = ProviderJob { id: "local_1".into(), provider: Provider::Midjourney, model_id: "midjourney_7".into(), prompt: "a fox".into(), created_at: 1, batch_count: 4, provider_job_id: "mj".into(), maybe_status_url: None, maybe_response_url: None, maybe_prompt_token: None, batch_token: "batch_1".into(), upload_tokens: Vec::new(), uploaded: Vec::new() };
let first = job.upload_token(2);
assert_eq!(job.upload_tokens.len(), 3);
assert_eq!(job.upload_token(2), first, "a retried upload reuses its token");
store.put(job.clone());
let reloaded = JobStore::load(&dir);
assert_eq!(reloaded.all(), vec![job]);
reloaded.remove("local_1");
assert!(JobStore::load(&dir).all().is_empty());
let _ = std::fs::remove_dir_all(dir);
}
}
@@ -0,0 +1,110 @@
//! Generating through another provider (`ClassyModelSelector`'s "Select Provider"): Midjourney
//! with the user's own account, or fal with their API key, instead of ArtCraft. Images only;
//! video always runs on ArtCraft. Jobs are tracked here (they aren't ArtCraft jobs) and their
//! results uploaded into the user's library, like the Tauri app.
pub mod accounts;
pub mod jobs;
pub mod submit;
pub mod worker;
use serde::{Deserialize, Serialize};
use crate::models::ModelInfo;
/// Midjourney's models; direct jobs always make four images and take no references.
const MIDJOURNEY_MODELS: [&str; 4] = ["midjourney", "midjourney_7", "midjourney_7_niji", "midjourney_8"];
const MIDJOURNEY_BATCH: u16 = 4;
/// Who runs a generation.
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, Hash, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum Provider {
#[default]
Artcraft,
Midjourney,
Fal,
}
impl Provider {
/// The picker's name (`getProviderDisplayName`).
pub fn label(self) -> &'static str {
match self {
Provider::Artcraft => "ArtCraft",
Provider::Midjourney => "Midjourney",
Provider::Fal => "FAL",
}
}
/// The logo's key in the creator icons.
pub fn icon(self) -> &'static str {
match self {
Provider::Artcraft => "artcraft",
Provider::Midjourney => "midjourney",
Provider::Fal => "fal",
}
}
/// The providers `model_id` can run on, the default first (`ImageModels.ts`,
/// `buildModelsFromListing.ts`): fal for four ArtCraft models, Midjourney directly for
/// Midjourney's.
pub fn for_model(model_id: &str) -> &'static [Provider] {
match model_id {
"flux_1_dev" | "flux_1_schnell" | "nano_banana_2" | "nano_banana_pro" => &[Provider::Artcraft, Provider::Fal],
"midjourney" => &[Provider::Midjourney, Provider::Artcraft],
"midjourney_7" | "midjourney_7_niji" | "midjourney_8" => &[Provider::Artcraft, Provider::Midjourney],
_ => &[Provider::Artcraft],
}
}
/// `picked` if `model_id` runs there, else the model's default.
pub fn resolve(model_id: &str, picked: Option<Provider>) -> Provider {
let allowed = Provider::for_model(model_id);
picked.filter(|p| allowed.contains(p)).unwrap_or(allowed[0])
}
/// What the model can do on this provider (`imageModelForProvider`): direct Midjourney makes
/// four images, without references, resolution or quality choices.
pub fn adapt(self, model: &ModelInfo) -> ModelInfo {
let mut m = model.clone();
if self == Provider::Midjourney && is_midjourney(&model.id) {
m.batch_sizes = vec![MIDJOURNEY_BATCH];
m.batch_default = MIDJOURNEY_BATCH;
m.image_refs_max = 0;
m.resolutions.clear();
m.resolution_default = None;
m.qualities.clear();
m.quality_default = None;
}
m
}
}
pub fn is_midjourney(model_id: &str) -> bool {
MIDJOURNEY_MODELS.contains(&model_id)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn models_offer_their_providers_default_first() {
assert_eq!(Provider::for_model("nano_banana_pro"), [Provider::Artcraft, Provider::Fal]);
assert_eq!(Provider::for_model("midjourney"), [Provider::Midjourney, Provider::Artcraft]);
assert_eq!(Provider::for_model("midjourney_8"), [Provider::Artcraft, Provider::Midjourney]);
assert_eq!(Provider::for_model("seedream_4"), [Provider::Artcraft]);
assert_eq!(Provider::resolve("seedream_4", Some(Provider::Fal)), Provider::Artcraft, "a pick the model can't use falls back");
assert_eq!(Provider::resolve("flux_1_dev", Some(Provider::Fal)), Provider::Fal);
}
#[test]
fn direct_midjourney_makes_four_images_without_options() {
let model = ModelInfo { id: "midjourney_7".into(), batch_sizes: vec![1, 2, 4], batch_default: 1, image_refs_max: 1, resolutions: vec!["one_k".into()], aspect_ratios: vec!["square".into()], ..Default::default() };
let direct = Provider::Midjourney.adapt(&model);
assert_eq!((direct.batch_sizes.as_slice(), direct.batch_default, direct.image_refs_max), ([4].as_slice(), 4, 0));
assert!(direct.resolutions.is_empty());
assert_eq!(direct.aspect_ratios, model.aspect_ratios, "aspect ratios stay");
assert_eq!(Provider::Artcraft.adapt(&model).batch_sizes, model.batch_sizes);
}
}
@@ -0,0 +1,184 @@
//! Starting an image generation at Midjourney or fal (the Tauri app's `handle_midjourney` and
//! `handle_fal`): the ArtCraft prompt record, then the provider's request through the router.
use artcraft_client::credentials::storyteller_credential_set::StorytellerCredentialSet;
use artcraft_client::endpoints::prompts::create_prompt::create_prompt;
use artcraft_client::utils::api_host::ApiHost;
use artcraft_router::api::image_list_ref::ImageListRef;
use artcraft_router::api::router_image_model::RouterImageModel;
use artcraft_router::api::router_provider::RouterProvider;
use artcraft_router::client::generation_mode_mismatch_strategy::GenerationModeMismatchStrategy;
use artcraft_router::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
use artcraft_router::client::router_client::RouterClient;
use artcraft_router::client::router_fal_client::RouterFalClient;
use artcraft_router::client::router_midjourney_client::RouterMidjourneyClient;
use artcraft_router::errors::artcraft_router_error::ArtcraftRouterError;
use artcraft_router::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
use artcraft_router::generate::generate_image::generate_image_response::GenerateImageResponse;
use artcraft_router::generate::generate_image::image_generation_draft_or_request::ImageGenerationDraftOrRequest;
use artcraft_router::generate::generate_image::router_image_request_to_artcraft_prompt::router_image_request_to_artcraft_prompt;
use log::warn;
use serde::de::DeserializeOwned;
use serde_json::{Map, Value};
use tokens::tokens::batch_generations::BatchGenerationToken;
use super::accounts::ProviderAccounts;
use super::jobs::ProviderJob;
use super::{Provider, is_midjourney};
use crate::backend::api_message;
/// Direct Midjourney jobs always make this many images.
const MIDJOURNEY_BATCH: u16 = 4;
/// Why a generation couldn't start.
#[derive(Debug, PartialEq)]
pub enum SubmitError {
/// The provider's account isn't connected (or its login expired).
NeedsAccount(Provider),
Message(String),
}
impl From<String> for SubmitError {
fn from(message: String) -> Self {
SubmitError::Message(message)
}
}
impl From<&str> for SubmitError {
fn from(message: &str) -> Self {
SubmitError::Message(message.to_owned())
}
}
/// Starts `fields` (the image page's OmniGen request) at `provider`; `image_urls` are the
/// references' CDN links.
pub async fn submit(host: &ApiHost, creds: Option<&StorytellerCredentialSet>, accounts: &ProviderAccounts, provider: Provider, fields: &Map<String, Value>, image_urls: Vec<String>) -> Result<ProviderJob, SubmitError> {
match provider {
Provider::Midjourney => submit_midjourney(host, creds, accounts, fields, image_urls).await,
Provider::Fal => submit_fal(host, creds, accounts, fields, image_urls).await,
Provider::Artcraft => Err("ArtCraft generations go through OmniGen".into()),
}
}
async fn submit_midjourney(host: &ApiHost, creds: Option<&StorytellerCredentialSet>, accounts: &ProviderAccounts, fields: &Map<String, Value>, image_urls: Vec<String>) -> Result<ProviderJob, SubmitError> {
let model_id = model_id(fields);
if !is_midjourney(model_id) {
return Err("This model cannot use a Midjourney account".into());
}
if !image_urls.is_empty() {
return Err("Direct Midjourney does not support image references yet".into());
}
if fields.get("image_batch_count").and_then(Value::as_u64).is_some_and(|n| n != u64::from(MIDJOURNEY_BATCH)) {
return Err("Direct Midjourney generates four images per job".into());
}
let mut builder = request_builder(RouterProvider::Midjourney, fields, Vec::new())?;
builder.resolution = None;
builder.quality = None;
builder.image_batch_count = Some(MIDJOURNEY_BATCH);
let prompt_request = router_image_request_to_artcraft_prompt(&builder);
let Ok(ImageGenerationDraftOrRequest::Request(generation)) = builder.build2() else {
return Err("Unexpected Midjourney request draft".into());
};
let session = accounts.midjourney.session().await.map_err(|err| err.to_string())?.ok_or(SubmitError::NeedsAccount(Provider::Midjourney))?;
// The prompt comes first, so the results can be attributed to it after a restart.
let creds = creds.ok_or("Please sign in to generate.")?;
let prompt = create_prompt(host, Some(creds), prompt_request).await.map_err(|err| api_message(&err))?;
let client = RouterClient::Midjourney(RouterMidjourneyClient::new(session.cookie_header, session.user_id, session.browser));
let GenerateImageResponse::Midjourney(response) = generation.send_request(&client).await.map_err(router_message(Provider::Midjourney))? else {
return Err("Unexpected Midjourney response".into());
};
Ok(new_job(Provider::Midjourney, fields, u32::from(MIDJOURNEY_BATCH), response.job_id, None, None, Some(prompt.prompt_token.to_string())))
}
async fn submit_fal(host: &ApiHost, creds: Option<&StorytellerCredentialSet>, accounts: &ProviderAccounts, fields: &Map<String, Value>, image_urls: Vec<String>) -> Result<ProviderJob, SubmitError> {
let key = accounts.fal_key().ok_or(SubmitError::NeedsAccount(Provider::Fal))?;
let builder = request_builder(RouterProvider::Fal, fields, image_urls)?;
// The prompt record is a nicety here: the generation goes ahead without it.
let maybe_prompt_token = match creds {
Some(creds) => create_prompt(host, Some(creds), router_image_request_to_artcraft_prompt(&builder)).await.map(|p| p.prompt_token.to_string()).map_err(|err| warn!("Couldn't create the prompt record: {}", api_message(&err))).ok(),
None => None,
};
let generation = match builder.build2() {
Ok(ImageGenerationDraftOrRequest::Request(generation)) => generation,
Ok(ImageGenerationDraftOrRequest::Draft(_)) => return Err("Fal should not be sending draft requests".into()),
Err(err) => return Err(err.to_string().into()),
};
let client = RouterClient::Fal(RouterFalClient::new_polling_only_from_raw_key(&key));
let GenerateImageResponse::Fal(response) = generation.send_request(&client).await.map_err(router_message(Provider::Fal))? else {
return Err("Unexpected FAL response".into());
};
let request_id = response.request_id.or(response.gateway_request_id).ok_or("FAL didn't return a request id")?;
let count = fields.get("image_batch_count").and_then(Value::as_u64).map_or(1, |n| n.max(1) as u32);
Ok(new_job(Provider::Fal, fields, count, request_id, response.maybe_status_url, response.maybe_response_url, maybe_prompt_token))
}
/// The router's request for the page's fields. The setting values are the router's own names
/// (`wide_sixteen_by_nine`, `one_k`, `high`).
fn request_builder(provider: RouterProvider, fields: &Map<String, Value>, image_urls: Vec<String>) -> Result<GenerateImageRequestBuilder, String> {
let model = match (parse::<RouterImageModel>(fields.get("model")), model_id(fields)) {
(Some(model), _) => model,
// The generic Midjourney model is the newest version.
(None, "midjourney") => RouterImageModel::Midjourney8,
(None, id) => return Err(format!("{id} isn't available on this provider")),
};
let mismatch = if matches!(provider, RouterProvider::Midjourney) { GenerationModeMismatchStrategy::AbortGeneration } else { GenerationModeMismatchStrategy::GenerateAnyway };
Ok(GenerateImageRequestBuilder { model, provider, prompt: fields.get("prompt").and_then(Value::as_str).map(str::to_owned), image_inputs: (!image_urls.is_empty()).then_some(ImageListRef::Urls(image_urls)), resolution: parse(fields.get("resolution")), aspect_ratio: parse(fields.get("aspect_ratio")), quality: parse(fields.get("quality")), image_batch_count: fields.get("image_batch_count").and_then(Value::as_u64).map(|n| n as u16), horizontal_angle: None, vertical_angle: None, zoom: None, request_mismatch_mitigation_strategy: RequestMismatchMitigationStrategy::PayMoreUpgrade, generation_mode_mismatch_strategy: Some(mismatch), idempotency_token: None })
}
fn new_job(provider: Provider, fields: &Map<String, Value>, batch_count: u32, provider_job_id: String, maybe_status_url: Option<String>, maybe_response_url: Option<String>, maybe_prompt_token: Option<String>) -> ProviderJob {
ProviderJob { id: format!("local_{}", uuid::Uuid::new_v4()), provider, model_id: model_id(fields).to_owned(), prompt: fields.get("prompt").and_then(Value::as_str).unwrap_or_default().to_owned(), created_at: chrono::Utc::now().timestamp(), batch_count, provider_job_id, maybe_status_url, maybe_response_url, maybe_prompt_token, batch_token: BatchGenerationToken::generate().to_string(), upload_tokens: Vec::new(), uploaded: Vec::new() }
}
/// What the user sees for a router error: billing problems point at the provider's website
/// (`GenericProviderBillingBlock`).
fn router_message(provider: Provider) -> impl Fn(ArtcraftRouterError) -> SubmitError {
move |err| match err {
ArtcraftRouterError::ProviderBillingError(_) => SubmitError::Message(format!("Please set up {} on their website to use it with ArtCraft.", provider.label())),
other => SubmitError::Message(other.to_string()),
}
}
fn model_id(fields: &Map<String, Value>) -> &str {
fields.get("model").and_then(Value::as_str).unwrap_or_default()
}
fn parse<T: DeserializeOwned>(value: Option<&Value>) -> Option<T> {
value.cloned().and_then(|v| serde_json::from_value(v).ok())
}
#[cfg(test)]
mod tests {
use serde_json::json;
use super::*;
#[test]
fn page_fields_become_the_routers_request() {
let fields = json!({ "model": "nano_banana_pro", "prompt": "a fox", "aspect_ratio": "wide_sixteen_by_nine", "resolution": "two_k", "quality": "high", "image_batch_count": 2 });
let b = request_builder(RouterProvider::Fal, fields.as_object().unwrap(), vec!["https://cdn/x.png".into()]).unwrap();
assert_eq!(b.model, RouterImageModel::NanoBananaPro);
assert_eq!((b.prompt.as_deref(), b.image_batch_count), (Some("a fox"), Some(2)));
assert!(b.aspect_ratio.is_some() && b.resolution.is_some() && b.quality.is_some() && b.image_inputs.is_some());
let generic = request_builder(RouterProvider::Midjourney, json!({ "model": "midjourney" }).as_object().unwrap(), Vec::new()).unwrap();
assert_eq!(generic.model, RouterImageModel::Midjourney8);
assert!(request_builder(RouterProvider::Fal, json!({ "model": "seedance_2p0" }).as_object().unwrap(), Vec::new()).is_err());
}
#[tokio::test]
async fn direct_midjourney_checks_its_limits_before_anything_else() {
let dir = std::env::temp_dir().join(format!("artcraft-native-submit-{}", uuid::Uuid::new_v4()));
let accounts = ProviderAccounts::load(&dir);
let host = ApiHost::Storyteller;
let run = |fields: Value, urls: Vec<String>| {
let (host, accounts) = (&host, &accounts);
async move { submit(host, None, accounts, Provider::Midjourney, fields.as_object().unwrap(), urls).await }
};
assert_eq!(run(json!({ "model": "flux_1_dev" }), Vec::new()).await.unwrap_err(), SubmitError::Message("This model cannot use a Midjourney account".into()));
assert_eq!(run(json!({ "model": "midjourney_7" }), vec!["u".into()]).await.unwrap_err(), SubmitError::Message("Direct Midjourney does not support image references yet".into()));
assert_eq!(run(json!({ "model": "midjourney_7", "image_batch_count": 2 }), Vec::new()).await.unwrap_err(), SubmitError::Message("Direct Midjourney generates four images per job".into()));
assert_eq!(run(json!({ "model": "midjourney_7", "prompt": "a fox", "image_batch_count": 4 }), Vec::new()).await.unwrap_err(), SubmitError::NeedsAccount(Provider::Midjourney), "no login yet");
let fal = submit(&host, None, &accounts, Provider::Fal, json!({ "model": "flux_1_dev" }).as_object().unwrap(), Vec::new()).await;
assert_eq!(fal.unwrap_err(), SubmitError::NeedsAccount(Provider::Fal));
let _ = std::fs::remove_dir_all(dir);
}
}
@@ -0,0 +1,335 @@
//! Finishing provider jobs in the background (the Tauri app's `midjourney_long_polling_thread`
//! and `poll_fal_tasks`): fal's queue is polled; Midjourney's feed is read every 15 s and its
//! websocket reports completions sooner. Finished results are downloaded and uploaded into the
//! user's library as one batch, resuming where they left off after a failure or restart.
use std::collections::HashSet;
use std::sync::mpsc::Sender;
use std::sync::{Arc, RwLock};
use std::time::{Duration, Instant};
use artcraft_client::credentials::storyteller_credential_set::StorytellerCredentialSet;
use artcraft_client::endpoints::media_files::upload_image_media_file_from_file::{UploadImageFromFileArgs, upload_image_media_file_from_file_with_idempotency_token};
use artcraft_client::utils::api_host::ApiHost;
use enums::common::generation_provider::GenerationProvider;
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::polling::poll_job_response::poll_job_response::{PollJobResponseArgs, poll_job_response};
use fal_client::polling::poll_job_response::success_case_extractors::PollResponseExtractedContents;
use fal_client::polling::poll_job_status::poll_job_status::{FalJobStatus, PollJobStatusArgs, poll_job_status};
use log::{info, warn};
use midjourney_client::client::websocket::midjourney_websocket::MidjourneyWebSocket;
use midjourney_client::client::websocket::midjourney_ws_event::MidjourneyWsEvent;
use midjourney_client::client::websocket::open_midjourney_websocket::{OpenMidjourneyWebSocketRequest, open_midjourney_websocket};
use midjourney_client::credentials::credential_manager::MidjourneySession;
use midjourney_client::endpoints::imagine::{ImagineArgs, ImagineRequest, imagine};
use midjourney_client::recipes::get_user_info::{GetUserInfoArgs, get_user_info};
use midjourney_client::utils::image_downloader_client::ImageDownloaderClient;
use tokens::tokens::batch_generations::BatchGenerationToken;
use tokens::tokens::prompts::PromptToken;
use tokio::sync::broadcast::Receiver;
use tokio::sync::broadcast::error::TryRecvError;
use tokio::time::timeout;
use super::Provider;
use super::accounts::ProviderAccounts;
use super::jobs::{JobStore, ProviderJob};
use crate::backend::Event;
use crate::backend::session::Credentials;
/// How often jobs are checked (fal's poll interval).
const TICK: Duration = Duration::from_secs(2);
const FEED_INTERVAL: Duration = Duration::from_secs(15);
const SOCKET_RETRY_INTERVAL: Duration = Duration::from_secs(60);
const REQUEST_TIMEOUT: Duration = Duration::from_secs(20);
/// Each download and upload of a result.
const TRANSFER_TIMEOUT: Duration = Duration::from_secs(60);
/// A Midjourney job not seen finished by then is given up on.
const MIDJOURNEY_GIVE_UP_SECS: i64 = 60 * 60;
/// Direct Midjourney jobs make four images.
const MIDJOURNEY_RESULTS: usize = 4;
pub struct Worker {
pub host: ApiHost,
pub creds: Arc<RwLock<Option<Credentials>>>,
pub http: reqwest::Client,
pub accounts: ProviderAccounts,
pub jobs: JobStore,
pub tx: Sender<Event>,
pub ctx: egui::Context,
}
/// Where a job's results come from.
enum Results<'a> {
Urls(Vec<String>),
Midjourney(&'a ImageDownloaderClient),
}
impl Results<'_> {
fn count(&self) -> usize {
match self {
Results::Urls(urls) => urls.len(),
Results::Midjourney(_) => MIDJOURNEY_RESULTS,
}
}
}
/// What the Midjourney side remembers between ticks.
#[derive(Default)]
struct MidjourneyWatch {
socket: Option<Socket>,
/// Jobs seen finished (by the websocket or the feed).
finished: HashSet<String>,
next_feed: Option<Instant>,
next_socket_attempt: Option<Instant>,
}
struct Socket {
credential_id: String,
socket: MidjourneyWebSocket,
events: Receiver<Arc<MidjourneyWsEvent>>,
subscribed: HashSet<String>,
}
impl Worker {
pub async fn run(self) {
let mut watch = MidjourneyWatch::default();
loop {
self.tick(&mut watch).await;
tokio::time::sleep(TICK).await;
}
}
async fn tick(&self, watch: &mut MidjourneyWatch) {
let jobs = self.jobs.all();
let (midjourney, fal): (Vec<ProviderJob>, Vec<ProviderJob>) = jobs.into_iter().partition(|j| j.provider == Provider::Midjourney);
if midjourney.is_empty() {
watch.close();
}
// Results go into the library: nothing can finish until someone's signed in.
let Some(creds) = self.creds.read().ok().and_then(|c| c.as_ref().map(Credentials::to_set)) else {
return;
};
if !fal.is_empty() {
if let Some(key) = self.accounts.fal_key().map(|k| FalApiKey::from_str(&k)) {
for job in fal {
self.poll_fal(job, &key, &creds).await;
}
}
}
if !midjourney.is_empty() {
self.poll_midjourney(watch, midjourney, &creds).await;
}
}
async fn poll_fal(&self, job: ProviderJob, key: &FalApiKey, creds: &StorytellerCredentialSet) {
let Some(status_url) = job.maybe_status_url.clone() else {
return self.fail(&job, "No queue status URL to check");
};
let status = match timeout(REQUEST_TIMEOUT, poll_job_status(PollJobStatusArgs { status_url: &status_url, api_key: key })).await {
Ok(Ok(status)) => status,
other => return warn!("Polling FAL job {} failed: {other:?}", job.provider_job_id),
};
if status.status != FalJobStatus::Completed {
return;
}
let Some(response_url) = job.maybe_response_url.clone().or(status.response_url) else {
return self.fail(&job, "Job completed but no response URL available");
};
let response = match timeout(REQUEST_TIMEOUT, poll_job_response(PollJobResponseArgs { response_url: &response_url, api_key: key })).await {
Ok(Ok(response)) => response,
other => return warn!("Fetching FAL results for {} failed (will retry): {other:?}", job.provider_job_id),
};
let urls = image_urls(response.extracted_contents.as_ref());
if urls.is_empty() {
return self.fail(&job, "FAL returned no images");
}
if let Err(err) = self.finish(job, Results::Urls(urls), creds).await {
warn!("Uploading FAL results failed (will retry): {err}");
}
}
async fn poll_midjourney(&self, watch: &mut MidjourneyWatch, jobs: Vec<ProviderJob>, creds: &StorytellerCredentialSet) {
let session = match self.accounts.midjourney.session().await {
Ok(Some(session)) => session,
Ok(None) => return watch.close(),
Err(err) => return warn!("Midjourney login check failed: {err}"),
};
let now = chrono::Utc::now().timestamp();
for job in jobs.iter().filter(|j| now - j.created_at > MIDJOURNEY_GIVE_UP_SECS && !watch.finished.contains(&j.provider_job_id)) {
self.fail(job, "Midjourney didn't finish this job");
}
let pending: HashSet<String> = jobs.iter().filter(|j| now - j.created_at <= MIDJOURNEY_GIVE_UP_SECS || watch.finished.contains(&j.provider_job_id)).map(|j| j.provider_job_id.clone()).collect();
watch.finished.retain(|id| pending.contains(id));
watch.refresh(&session, &pending).await;
let downloader = match ImageDownloaderClient::create(Some(session.browser.clone())) {
Ok(downloader) => downloader,
Err(err) => return warn!("Midjourney downloader unavailable: {err:?}"),
};
let done: Vec<ProviderJob> = jobs.into_iter().filter(|j| watch.finished.contains(&j.provider_job_id)).collect();
for job in done {
let id = job.provider_job_id.clone();
match self.finish(job, Results::Midjourney(&downloader), creds).await {
Ok(()) => {
watch.finished.remove(&id);
},
Err(err) => warn!("Uploading Midjourney results for {id} failed (will retry): {err}"),
}
}
}
/// Uploads the results not uploaded yet, then reports the job done.
async fn finish(&self, mut job: ProviderJob, results: Results<'_>, creds: &StorytellerCredentialSet) -> anyhow::Result<()> {
let provider = match job.provider {
Provider::Midjourney => GenerationProvider::Midjourney,
Provider::Fal => GenerationProvider::Fal,
Provider::Artcraft => GenerationProvider::Artcraft,
};
let prompt = job.maybe_prompt_token.as_deref().map(PromptToken::new_from_str);
let batch = BatchGenerationToken::new_from_str(&job.batch_token);
for index in job.uploaded.len()..results.count() {
let (bytes, ext) = timeout(TRANSFER_TIMEOUT, self.download(&results, &job, index)).await??;
let mut file = tempfile::Builder::new().prefix("artcraft-provider-").suffix(&format!(".{ext}")).tempfile()?;
std::io::Write::write_all(&mut file, &bytes)?;
// The token is saved before the upload, so a retry can't upload the same result twice.
let token = job.upload_token(index);
self.jobs.put(job.clone());
let args = UploadImageFromFileArgs { api_host: &self.host, maybe_creds: Some(creds), path: file.path(), is_intermediate_system_file: false, maybe_prompt_token: prompt.as_ref(), maybe_generation_provider: Some(provider), maybe_batch_token: Some(&batch) };
let uploaded = timeout(TRANSFER_TIMEOUT, upload_image_media_file_from_file_with_idempotency_token(args, &token)).await??;
job.uploaded.push(uploaded.media_file_token.to_string());
self.jobs.put(job.clone());
}
info!("{:?} job {} finished with {} results", job.provider, job.provider_job_id, job.uploaded.len());
self.jobs.remove(&job.id);
self.send(Event::ProviderJobFinished { job_id: job.id, batch_token: job.batch_token });
Ok(())
}
async fn download(&self, results: &Results<'_>, job: &ProviderJob, index: usize) -> anyhow::Result<(Vec<u8>, &'static str)> {
match results {
Results::Urls(urls) => {
let url = &urls[index];
let bytes = self.http.get(url).send().await?.error_for_status()?.bytes().await?;
Ok((bytes.to_vec(), image_extension(url)))
},
Results::Midjourney(downloader) => Ok((downloader.download_image(&job.provider_job_id, index as u8).await?, "png")),
}
}
fn fail(&self, job: &ProviderJob, reason: &str) {
warn!("{:?} job {} failed: {reason}", job.provider, job.provider_job_id);
self.jobs.remove(&job.id);
self.send(Event::ProviderJobFailed { job_id: job.id.clone(), reason: reason.to_owned() });
}
fn send(&self, event: Event) {
let _ = self.tx.send(event);
self.ctx.request_repaint();
}
}
impl MidjourneyWatch {
/// Notes which pending jobs have finished: websocket events as they come, and the feed every
/// so often (it also covers frames the socket missed, and jobs from before a restart).
async fn refresh(&mut self, session: &MidjourneySession, pending: &HashSet<String>) {
if self.socket.as_ref().is_some_and(|s| s.credential_id != session.credential_id || !s.socket.is_connected()) {
self.close();
}
self.drain(pending);
let now = Instant::now();
if self.next_feed.is_none_or(|t| now >= t) {
self.next_feed = Some(now + FEED_INTERVAL);
match timeout(REQUEST_TIMEOUT, imagine(ImagineArgs { request: ImagineRequest { user_id: &session.user_id, page_size: None }, cookie_header: &session.cookie_header, hostname: None, browser: Some(session.browser.clone()) })).await {
Ok(Ok(feed)) => self.finished.extend(feed.items.into_iter().filter_map(|item| item.id).filter(|id| pending.contains(id))),
other => warn!("Reading the Midjourney feed failed: {other:?}"),
}
}
if self.socket.is_none() && self.next_socket_attempt.is_none_or(|t| now >= t) {
self.next_socket_attempt = Some(now + SOCKET_RETRY_INTERVAL);
match timeout(REQUEST_TIMEOUT, open_socket(session)).await {
Ok(Ok(socket)) => self.socket = Some(socket),
_ => log::debug!("Midjourney websocket unavailable; the feed still reports completions"),
}
}
if let Some(socket) = self.socket.as_mut() {
socket.subscribed.retain(|id| pending.contains(id));
for id in pending {
if !socket.subscribed.contains(id) && socket.socket.subscribe_to_job(id).is_ok() {
socket.subscribed.insert(id.clone());
}
}
}
}
fn drain(&mut self, pending: &HashSet<String>) {
let Some(socket) = self.socket.as_mut() else {
return;
};
loop {
match socket.events.try_recv() {
Ok(event) => {
if let MidjourneyWsEvent::Completed { job_id, .. } = event.as_ref() {
if pending.contains(job_id) {
self.finished.insert(job_id.clone());
}
}
},
// Missed frames: read the feed now.
Err(TryRecvError::Lagged(_)) => self.next_feed = None,
Err(TryRecvError::Empty | TryRecvError::Closed) => break,
}
}
}
fn close(&mut self) {
if let Some(socket) = self.socket.take() {
socket.socket.close();
}
}
}
async fn open_socket(session: &MidjourneySession) -> anyhow::Result<Socket> {
let info = get_user_info(GetUserInfoArgs { cookie_header: &session.cookie_header, hostname: None, browser: Some(session.browser.clone()) }).await?;
let token = info.websocket_token.ok_or_else(|| anyhow::anyhow!("No Midjourney websocket token"))?;
let socket = open_midjourney_websocket(OpenMidjourneyWebSocketRequest { websocket_token: &token, user_id: session.user_id.clone(), hostname: None, browser: Some(session.browser.clone()) }).await?;
let events = socket.events();
Ok(Socket { credential_id: session.credential_id.clone(), socket, events, subscribed: HashSet::new() })
}
/// A fal job's image links: the batch, else the single image.
fn image_urls(extracted: Option<&PollResponseExtractedContents>) -> Vec<String> {
let Some(extracted) = extracted else {
return Vec::new();
};
let batch: Vec<String> = extracted.images.iter().flatten().filter_map(|i| i.url.clone()).collect();
if !batch.is_empty() {
return batch;
}
extracted.image.as_ref().and_then(|i| i.url.clone()).into_iter().collect()
}
/// The file extension a result URL ends in (PNG when it doesn't say).
fn image_extension(url: &str) -> &'static str {
let path = url.split(['?', '#']).next().unwrap_or_default().to_ascii_lowercase();
["jpg", "jpeg", "webp", "png"].into_iter().find(|ext| path.ends_with(&format!(".{ext}"))).unwrap_or("png")
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn results_keep_their_file_type() {
assert_eq!(image_extension("https://fal.media/files/a/b.jpeg?x=1"), "jpeg");
assert_eq!(image_extension("https://fal.media/files/a/b.webp"), "webp");
assert_eq!(image_extension("https://fal.media/files/a/b"), "png");
}
#[test]
fn fal_results_prefer_the_batch() {
let contents = fal_client::polling::poll_job_response::success_case_extractors::extract_contents_from_response(&serde_json::json!({ "images": [{ "url": "https://fal/1.png" }, { "url": "https://fal/2.png" }] }));
assert_eq!(image_urls(contents.as_ref()), ["https://fal/1.png", "https://fal/2.png"]);
let single = fal_client::polling::poll_job_response::success_case_extractors::extract_contents_from_response(&serde_json::json!({ "image": { "url": "https://fal/one.png" } }));
assert_eq!(image_urls(single.as_ref()), ["https://fal/one.png"]);
assert!(image_urls(None).is_empty());
}
}
+422
View File
@@ -0,0 +1,422 @@
//! In-app video playback through FFmpeg (when it's installed): `ffprobe` reads the clip's size,
//! rate and length, one `ffmpeg` streams RGBA frames into a texture and another streams PCM into
//! rodio. Pausing and seeking restart both at the new position. Without FFmpeg the lightbox opens
//! videos in the system player instead.
use std::collections::VecDeque;
use std::io::Read;
use std::process::{Child, Command, Stdio};
use std::sync::mpsc::{Receiver, SyncSender, TryRecvError, channel, sync_channel};
use std::sync::OnceLock;
use std::time::{Duration, Instant};
use egui::{Align2, Color32, ColorImage, FontId, Rect, Sense, TextureHandle, TextureOptions, Ui, pos2, vec2};
use log::warn;
use rodio::{OutputStream, OutputStreamHandle, Sink, Source};
use crate::theme;
use crate::ui::icons::{self, Icon};
/// Frames are scaled down to at most this width (the lightbox never shows them larger).
const MAX_WIDTH: u32 = 1280;
const AUDIO_RATE: u32 = 44_100;
/// Decoded frames buffered ahead of the clock (back-pressure on `ffmpeg`).
const FRAME_QUEUE: usize = 4;
const CONTROLS_HEIGHT: f32 = 40.0;
/// Whether `ffmpeg` and `ffprobe` are on the PATH (checked once).
pub fn ffmpeg_available() -> bool {
static AVAILABLE: OnceLock<bool> = OnceLock::new();
*AVAILABLE.get_or_init(|| ["ffmpeg", "ffprobe"].iter().all(|tool| command(tool).arg("-version").stdout(Stdio::null()).stderr(Stdio::null()).status().is_ok_and(|s| s.success())))
}
#[derive(Clone, Copy, Debug)]
struct VideoInfo {
width: u32,
height: u32,
fps: f64,
duration: f64,
}
pub struct VideoPlayer {
url: String,
probe: Option<Receiver<Result<VideoInfo, String>>>,
info: Option<VideoInfo>,
stream: Option<Stream>,
texture: Option<TextureHandle>,
/// Media time of the frame on screen.
position: f64,
muted: bool,
error: Option<String>,
output: Option<(OutputStream, OutputStreamHandle)>,
}
/// One run of the decoders from a start position.
struct Stream {
start: f64,
started: Instant,
frames: Receiver<(ColorImage, f64)>,
next: Option<(ColorImage, f64)>,
ended: bool,
children: Vec<Child>,
sink: Option<Sink>,
}
impl Drop for Stream {
fn drop(&mut self) {
for child in &mut self.children {
let _ = child.kill();
let _ = child.wait();
}
}
}
impl VideoPlayer {
/// Starts probing `url`; playback begins as soon as its size is known.
pub fn new(url: &str) -> Self {
let (tx, rx) = channel();
let probe_url = url.to_owned();
std::thread::spawn(move || {
let _ = tx.send(probe(&probe_url));
});
Self { url: url.to_owned(), probe: Some(rx), info: None, stream: None, texture: None, position: 0.0, muted: false, error: None, output: None }
}
pub fn url(&self) -> &str {
&self.url
}
/// Draws the video fitted inside `rect` with its controls; `poster` (the still frame) shows
/// until the first decoded frame arrives.
pub fn ui(&mut self, ui: &mut Ui, rect: Rect, poster: Option<&TextureHandle>) {
self.poll_probe();
let ctx = ui.ctx().clone();
self.advance(&ctx);
ui.painter().rect_filled(rect, 0.0, Color32::BLACK);
let video_area = Rect::from_min_max(rect.min, pos2(rect.right(), rect.bottom() - CONTROLS_HEIGHT));
match (&self.texture, &self.error) {
(_, Some(err)) => {
ui.painter().text(video_area.center(), Align2::CENTER_CENTER, format!("Couldn't play this video: {err}"), FontId::new(13.0, egui::FontFamily::Proportional), theme::BAD);
},
(Some(texture), None) => {
let fit = crate::overlays::lightbox::fit_inside(texture.size_vec2(), video_area.shrink(4.0).size());
egui::Image::new(texture).paint_at(ui, Rect::from_center_size(video_area.center(), fit));
},
(None, None) => {
if let Some(poster) = poster {
let fit = crate::overlays::lightbox::fit_inside(poster.size_vec2(), video_area.shrink(4.0).size());
egui::Image::new(poster).tint(Color32::from_gray(150)).paint_at(ui, Rect::from_center_size(video_area.center(), fit));
}
icons::paint_spinner(ui, Rect::from_center_size(video_area.center(), vec2(32.0, 32.0)), Color32::WHITE);
},
}
// Clicking the picture toggles play/pause.
let click = ui.interact(video_area, ui.id().with(("video-surface", &self.url)), Sense::click());
if click.clicked() {
self.toggle_pause();
}
self.controls(ui, Rect::from_min_max(pos2(rect.left(), rect.bottom() - CONTROLS_HEIGHT), rect.max));
if self.stream.is_some() {
ctx.request_repaint_after(Duration::from_millis(10));
}
}
fn controls(&mut self, ui: &mut Ui, bar: Rect) {
let p = ui.painter().clone();
p.rect_filled(bar, 0.0, theme::fade(theme::PANEL, 0.95));
let playing = self.stream.is_some();
let button = Rect::from_center_size(pos2(bar.left() + 22.0, bar.center().y), vec2(28.0, 28.0));
let resp = ui.interact(button, ui.id().with(("video-play", &self.url)), Sense::click()).on_hover_text(if playing { "Pause" } else { "Play" }).on_hover_cursor(egui::CursorIcon::PointingHand);
if playing {
for dx in [-3.5, 3.5] {
p.rect_filled(Rect::from_center_size(button.center() + vec2(dx, 0.0), vec2(3.0, 12.0)), 0.0, theme::INK);
}
} else {
icons::paint(&p, button.shrink(7.0), Icon::Play, theme::INK);
}
if resp.clicked() {
self.toggle_pause();
}
let duration = self.info.map_or(0.0, |i| i.duration);
let time = format!("{} / {}", clock(self.position), clock(duration));
let label = p.text(pos2(button.right() + 8.0, bar.center().y), Align2::LEFT_CENTER, time, FontId::new(11.5, egui::FontFamily::Monospace), theme::MUTED);
let mute = Rect::from_center_size(pos2(bar.right() - 22.0, bar.center().y), vec2(28.0, 28.0));
let mute_resp = ui.interact(mute, ui.id().with(("video-mute", &self.url)), Sense::click()).on_hover_text(if self.muted { "Unmute" } else { "Mute" }).on_hover_cursor(egui::CursorIcon::PointingHand);
icons::paint(&p, mute.shrink(7.0), if self.muted { Icon::VolumeOff } else { Icon::Volume }, theme::INK);
if mute_resp.clicked() {
self.muted = !self.muted;
if let Some(sink) = self.stream.as_ref().and_then(|s| s.sink.as_ref()) {
sink.set_volume(if self.muted { 0.0 } else { 1.0 });
}
}
// The scrubber: click or drag to seek.
let track = Rect::from_min_max(pos2(label.right() + 14.0, bar.center().y - 8.0), pos2(mute.left() - 12.0, bar.center().y + 8.0));
let scrub = ui.interact(track, ui.id().with(("video-scrub", &self.url)), Sense::click_and_drag()).on_hover_cursor(egui::CursorIcon::PointingHand);
let line = Rect::from_center_size(track.center(), vec2(track.width(), 4.0));
p.rect_filled(line, egui::CornerRadius::same(2), theme::WASH_HOVER);
let fraction = if duration > 0.0 { (self.position / duration).clamp(0.0, 1.0) as f32 } else { 0.0 };
p.rect_filled(Rect::from_min_size(line.min, vec2(line.width() * fraction, line.height())), egui::CornerRadius::same(2), theme::ACCENT_400);
if (scrub.clicked() || scrub.drag_stopped()) && duration > 0.0 {
if let Some(pos) = scrub.interact_pointer_pos() {
let t = (((pos.x - line.left()) / line.width()).clamp(0.0, 1.0) as f64) * duration;
self.seek(t);
}
}
}
fn poll_probe(&mut self) {
let Some(rx) = &self.probe else {
return;
};
match rx.try_recv() {
Ok(Ok(info)) => {
self.info = Some(info);
self.probe = None;
self.play_from(0.0);
},
Ok(Err(err)) => {
self.error = Some(err);
self.probe = None;
},
Err(_) => {},
}
}
/// Shows the newest frame that's due; loops at the end.
fn advance(&mut self, ctx: &egui::Context) {
let Some(stream) = &mut self.stream else {
return;
};
let now = stream.start + stream.started.elapsed().as_secs_f64();
let mut shown = None;
loop {
if stream.next.is_none() {
match stream.frames.try_recv() {
Ok(frame) => stream.next = Some(frame),
Err(TryRecvError::Disconnected) => {
stream.ended = true;
break;
},
Err(TryRecvError::Empty) => break,
}
}
match &stream.next {
Some((_, pts)) if *pts <= now => shown = stream.next.take(),
_ => break,
}
}
if let Some((image, pts)) = shown {
self.position = pts;
match &mut self.texture {
Some(t) => t.set(image, TextureOptions::LINEAR),
None => self.texture = Some(ctx.load_texture(format!("video:{}", self.url), image, TextureOptions::LINEAR)),
}
}
if stream.ended && stream.next.is_none() {
self.play_from(0.0);
}
}
fn toggle_pause(&mut self) {
if self.stream.take().is_none() && self.info.is_some() {
self.play_from(self.position);
}
}
fn seek(&mut self, t: f64) {
self.position = t;
self.play_from(t);
}
fn play_from(&mut self, start: f64) {
self.stream = None;
let Some(info) = self.info else {
return;
};
match self.spawn(info, start) {
Ok(stream) => self.stream = Some(stream),
Err(err) => self.error = Some(err),
}
}
fn spawn(&mut self, info: VideoInfo, start: f64) -> Result<Stream, String> {
let (w, h) = output_size(info);
let mut video = command("ffmpeg").args(["-nostdin", "-loglevel", "error", "-ss", &format!("{start:.3}"), "-i", &self.url, "-an", "-vf", &format!("scale={w}:{h}"), "-f", "rawvideo", "-pix_fmt", "rgba", "-"]).stdout(Stdio::piped()).stderr(Stdio::null()).spawn().map_err(|e| format!("ffmpeg: {e}"))?;
let stdout = video.stdout.take().ok_or("ffmpeg has no output")?;
let (frame_tx, frame_rx) = sync_channel(FRAME_QUEUE);
let fps = info.fps.max(1.0);
std::thread::spawn(move || read_frames(stdout, w, h, start, fps, frame_tx));
let mut children = vec![video];
let sink = self.spawn_audio(start, &mut children);
Ok(Stream { start, started: Instant::now(), frames: frame_rx, next: None, ended: false, children, sink })
}
/// The soundtrack as PCM into rodio (silently skipped when the clip has none).
fn spawn_audio(&mut self, start: f64, children: &mut Vec<Child>) -> Option<Sink> {
let mut audio = command("ffmpeg").args(["-nostdin", "-loglevel", "error", "-ss", &format!("{start:.3}"), "-i", &self.url, "-vn", "-ac", "2", "-ar", &AUDIO_RATE.to_string(), "-f", "s16le", "-"]).stdout(Stdio::piped()).stderr(Stdio::null()).spawn().ok()?;
let mut stdout = audio.stdout.take()?;
children.push(audio);
if self.output.is_none() {
self.output = OutputStream::try_default().map_err(|e| warn!("No audio output: {e}")).ok();
}
let (_, handle) = self.output.as_ref()?;
let sink = Sink::try_new(handle).ok()?;
let (tx, rx) = channel::<Vec<i16>>();
std::thread::spawn(move || {
let mut buf = vec![0u8; 8192];
while let Ok(n) = stdout.read(&mut buf) {
if n == 0 || tx.send(buf[..n - n % 2].chunks_exact(2).map(|b| i16::from_le_bytes([b[0], b[1]])).collect()).is_err() {
break;
}
}
});
sink.set_volume(if self.muted { 0.0 } else { 1.0 });
sink.append(PcmStream { rx, buf: VecDeque::new() });
Some(sink)
}
}
/// Reads fixed-size RGBA frames from `ffmpeg` and stamps them with their media time.
fn read_frames(mut stdout: impl Read, w: u32, h: u32, start: f64, fps: f64, tx: SyncSender<(ColorImage, f64)>) {
let mut buf = vec![0u8; (w * h * 4) as usize];
let mut n = 0u64;
while stdout.read_exact(&mut buf).is_ok() {
let image = ColorImage::from_rgba_unmultiplied([w as usize, h as usize], &buf);
if tx.send((image, start + n as f64 / fps)).is_err() {
break;
}
n += 1;
}
}
/// Live PCM from a channel; silence while it waits, the end when the sender's gone.
struct PcmStream {
rx: Receiver<Vec<i16>>,
buf: VecDeque<i16>,
}
impl Iterator for PcmStream {
type Item = i16;
fn next(&mut self) -> Option<i16> {
if self.buf.is_empty() {
match self.rx.try_recv() {
Ok(chunk) => self.buf.extend(chunk),
Err(TryRecvError::Empty) => return Some(0),
Err(TryRecvError::Disconnected) => return None,
}
}
self.buf.pop_front().or(Some(0))
}
}
impl Source for PcmStream {
fn current_frame_len(&self) -> Option<usize> {
None
}
fn channels(&self) -> u16 {
2
}
fn sample_rate(&self) -> u32 {
AUDIO_RATE
}
fn total_duration(&self) -> Option<Duration> {
None
}
}
/// Reads the clip's size, rate and length. A small probe keeps this ~2 s on CDN files instead of
/// ~14 s (ffprobe otherwise scans much of the file over HTTP).
fn probe(url: &str) -> Result<VideoInfo, String> {
#[derive(serde::Deserialize, Default)]
#[serde(default)]
struct Probe {
streams: Vec<ProbeStream>,
format: ProbeFormat,
}
#[derive(serde::Deserialize, Default)]
#[serde(default)]
struct ProbeStream {
width: u32,
height: u32,
avg_frame_rate: String,
}
#[derive(serde::Deserialize, Default)]
#[serde(default)]
struct ProbeFormat {
duration: String,
}
let out = command("ffprobe").args(["-v", "error", "-probesize", "65536", "-analyzeduration", "0", "-select_streams", "v:0", "-show_entries", "stream=width,height,avg_frame_rate:format=duration", "-of", "json", url]).output().map_err(|e| format!("ffprobe: {e}"))?;
let probe: Probe = serde_json::from_slice(&out.stdout).map_err(|e| format!("ffprobe: {e}"))?;
let stream = probe.streams.first().ok_or("no video stream")?;
let fps = match stream.avg_frame_rate.split_once('/') {
Some((n, d)) => n.parse::<f64>().unwrap_or(24.0) / d.parse::<f64>().unwrap_or(1.0).max(1.0),
None => stream.avg_frame_rate.parse().unwrap_or(24.0),
};
Ok(VideoInfo { width: stream.width.max(2), height: stream.height.max(2), fps: if fps.is_finite() && fps > 0.0 { fps } else { 24.0 }, duration: probe.format.duration.parse().unwrap_or(0.0) })
}
/// The decoded size: at most `MAX_WIDTH` wide, both sides even (as `scale` requires).
fn output_size(info: VideoInfo) -> (u32, u32) {
let w = info.width.min(MAX_WIDTH) & !1;
let h = ((info.height as f64 * w as f64 / info.width as f64).round() as u32).max(2) & !1;
(w, h)
}
/// `m:ss`.
fn clock(secs: f64) -> String {
let s = secs.max(0.0) as u64;
format!("{}:{:02}", s / 60, s % 60)
}
/// A command that doesn't flash a console window on Windows.
fn command(program: &str) -> Command {
#[allow(unused_mut)]
let mut cmd = Command::new(program);
#[cfg(windows)]
{
use std::os::windows::process::CommandExt;
const CREATE_NO_WINDOW: u32 = 0x0800_0000;
cmd.creation_flags(CREATE_NO_WINDOW);
}
cmd
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn output_sizes_are_even_and_capped() {
let info = |width, height| VideoInfo { width, height, fps: 24.0, duration: 5.0 };
assert_eq!(output_size(info(1920, 1080)), (1280, 720));
assert_eq!(output_size(info(721, 1281)), (720, 1278));
assert_eq!(clock(65.9), "1:05");
}
/// Needs FFmpeg on the PATH: `cargo test -p artcraft_native -- --ignored`.
#[test]
#[ignore]
fn probes_and_decodes_a_generated_clip() {
let dir = std::env::temp_dir().join("artcraft-native-video-test");
std::fs::create_dir_all(&dir).unwrap();
let clip = dir.join("clip.mp4");
let status = command("ffmpeg").args(["-y", "-loglevel", "error", "-f", "lavfi", "-i", "testsrc=duration=2:size=640x360:rate=24", "-f", "lavfi", "-i", "sine=duration=2", "-shortest", "-pix_fmt", "yuv420p"]).arg(&clip).status().unwrap();
assert!(status.success());
let url = clip.to_string_lossy().into_owned();
let info = probe(&url).unwrap();
assert_eq!((info.width, info.height), (640, 360));
assert!((info.fps - 24.0).abs() < 0.01 && (info.duration - 2.0).abs() < 0.2);
let mut player = VideoPlayer { url: url.clone(), probe: None, info: Some(info), stream: None, texture: None, position: 0.0, muted: true, error: None, output: None };
let stream = player.spawn(info, 1.0).unwrap();
let (frame, pts) = stream.frames.recv_timeout(Duration::from_secs(10)).unwrap();
assert_eq!(frame.size, [640, 360]);
assert!((pts - 1.0).abs() < 1e-9, "frames are stamped from the start position");
let count = 1 + stream.frames.iter().count();
assert!((20..=28).contains(&count), "about a second of frames at 24 fps, got {count}");
}
}
+87
View File
@@ -42,6 +42,17 @@ impl MediaLinks {
}
}
impl MediaLinks {
/// A video's animated preview, `width` pixels wide where the CDN can resize.
pub fn animated_preview(&self, width: u32) -> Option<String> {
let previews = self.maybe_video_previews.as_ref()?;
if !previews.animated_thumbnail_template.is_empty() {
return Some(previews.animated_thumbnail_template.replace("{WIDTH}", &width.to_string()));
}
(!previews.animated.is_empty()).then(|| previews.animated.clone())
}
}
/// `GET /v1/media_files/list/user/{username}`.
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
@@ -70,10 +81,56 @@ pub struct MediaFile {
pub maybe_model_type: Option<String>,
pub maybe_origin_model_type: Option<String>,
pub maybe_duration_millis: Option<u64>,
pub maybe_title: Option<String>,
pub media_links: MediaLinks,
pub created_at: String,
}
impl MediaFile {
/// The file's still thumbnail; audio files have none (their CDN link is the audio itself).
pub fn thumbnail(&self, width: u32) -> Option<String> {
if self.media_class.as_deref() == Some("audio") {
return None;
}
self.media_links.thumbnail(width)
}
}
/// `GET /v1/omni_gen/models/audio`. The client has no binding for it (and the API's type only
/// serializes), so it's read here; capability flags are left out when false.
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
pub struct AudioModels {
pub success: bool,
pub models: Vec<AudioModel>,
}
#[derive(Clone, Debug, Default, Deserialize)]
#[serde(default)]
pub struct AudioModel {
pub model: String,
pub model_creator: Option<String>,
pub full_name: Option<String>,
pub extra_info_short: Option<String>,
pub text_prompt_supported: Option<bool>,
pub style_prompt_supported: bool,
pub audio_references_supported: bool,
pub audio_references_max: Option<u16>,
pub image_references_supported: bool,
pub image_references_max: Option<u16>,
pub keep_lyrics_supported: bool,
pub instrumental_toggle_supported: bool,
pub loopable_toggle_supported: bool,
pub bpm_supported: bool,
pub musical_key_supported: bool,
pub sample_rate_hz_options: Vec<u32>,
pub sample_rate_hz_default: Option<u32>,
pub speed_supported: bool,
pub volume_supported: bool,
pub pitch_supported: bool,
pub is_disabled: bool,
}
/// `GET /v1/media_files/batch_gen_redux/{token}`.
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
@@ -194,6 +251,36 @@ pub struct ContextImage {
pub media_links: MediaLinks,
}
/// A saved character (`GET /v1/characters/session`).
#[derive(Clone, Debug, Default, Deserialize)]
#[serde(default)]
pub struct Character {
pub token: String,
pub name: String,
pub maybe_description: Option<String>,
pub maybe_avatar: Option<MediaLinks>,
pub maybe_full_image: Option<MediaLinks>,
pub models: Vec<String>,
}
impl Character {
pub fn avatar_url(&self) -> Option<String> {
self.maybe_avatar.as_ref().or(self.maybe_full_image.as_ref()).and_then(|l| l.thumbnail(256))
}
pub fn full_url(&self) -> Option<String> {
self.maybe_full_image.as_ref().or(self.maybe_avatar.as_ref()).map(|l| l.cdn_url.clone()).filter(|u| !u.is_empty())
}
}
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
pub struct CharactersPage {
pub success: bool,
pub characters: Vec<Character>,
pub next_cursor: Option<i64>,
}
/// `GET /v1/session` (only the parts the app shows).
#[derive(Debug, Default, Deserialize)]
#[serde(default)]
+128 -57
View File
@@ -1,16 +1,17 @@
//! The generation feed as a masonry grid (`GenerationGridView`) or a list (`GenerationListView`):
//! running jobs first-class alongside failures and finished media, newest first, loading more as
//! the user scrolls.
//! the user scrolls. Audio plays in place, with its waveform.
use std::collections::{HashMap, HashSet};
use egui::{Align2, Color32, CornerRadius, FontFamily, FontId, Id, Mesh, Rect, Sense, Stroke, Ui, pos2, vec2};
use crate::backend::audio::AudioPlayer;
use crate::backend::media_cache::{Lookup, MediaCache};
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob, pending_status, time_ago};
use crate::theme;
use crate::ui::icons::{self, Icon};
use crate::ui::{creator_icons, widgets};
use crate::ui::{creator_icons, waveform, widgets};
/// `react-masonry-css` gutter and the cap on tall portraits (≈ 5:7).
const GAP: f32 = 8.0;
@@ -18,6 +19,8 @@ const MAX_RATIO: f32 = 1.4;
/// Start loading the next page this far before the end.
const LOAD_MORE_MARGIN: f32 = 400.0;
const LIST_THUMB: f32 = 100.0;
/// The list's compact audio player is at most `max-w-md` wide.
const LIST_PLAYER_MAX: f32 = 448.0;
#[derive(Clone, Copy, Debug, Default, PartialEq, Eq, serde::Serialize, serde::Deserialize)]
pub enum ViewMode {
@@ -61,6 +64,8 @@ pub struct FeedView<'a> {
/// Prompt texts by prompt token (list view and pending cards).
pub prompts: &'a HashMap<String, String>,
pub make_video: bool,
/// Play videos' animated previews instead of their still frames.
pub autoplay: bool,
pub bottom_padding: f32,
}
@@ -84,7 +89,7 @@ impl Entry<'_> {
#[derive(Clone, Default)]
pub struct RatioCache(HashMap<String, f32>);
pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &mut RatioCache, names: &dyn ModelNames) -> Vec<FeedAction> {
pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &mut RatioCache, audio: &mut AudioPlayer, names: &dyn ModelNames) -> Vec<FeedAction> {
let mut entries: Vec<Entry<'_>> = view.pending.iter().map(Entry::Pending).chain(view.failed.iter().map(Entry::Failed)).chain(view.items.iter().map(Entry::Item)).collect();
entries.sort_by_key(|e| std::cmp::Reverse(e.time()));
let mut actions = Vec::new();
@@ -111,10 +116,10 @@ pub fn show(ui: &mut Ui, view: &FeedView<'_>, cache: &mut MediaCache, ratios: &m
match (entry, view.mode) {
(Entry::Pending(job), ViewMode::Grid) => pending_card(ui, rect, job, now, names),
(Entry::Failed(job), ViewMode::Grid) => failed_card(ui, view.id, rect, job, cache, names, &mut actions),
(Entry::Item(item), ViewMode::Grid) => item_card(ui, view, rect, item, cache, ratios, names, &mut actions),
(Entry::Item(item), ViewMode::Grid) => item_card(ui, view, rect, item, cache, ratios, audio, names, &mut actions),
(Entry::Pending(job), ViewMode::List) => pending_row(ui, rect, job, now, names),
(Entry::Failed(job), ViewMode::List) => failed_row(ui, view.id, rect, job, names, &mut actions),
(Entry::Item(item), ViewMode::List) => item_row(ui, view, rect, item, cache, names, now, &mut actions),
(Entry::Item(item), ViewMode::List) => item_row(ui, view, rect, item, cache, audio, names, now, &mut actions),
}
}
@@ -226,25 +231,15 @@ fn failed_card(ui: &mut Ui, id: Id, rect: Rect, job: &FailedJob, cache: &mut Med
}
#[allow(clippy::too_many_arguments)]
fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache, names: &dyn ModelNames, actions: &mut Vec<FeedAction>) {
fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache, audio: &mut AudioPlayer, names: &dyn ModelNames, actions: &mut Vec<FeedAction>) {
let resp = ui.interact(rect, view.id.with(("item", &item.token)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
let hovered = resp.hovered();
ui.painter().rect_filled(rect, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
match item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
Some(Lookup::Ready(t)) => {
let size = t.size_vec2();
if size.x > 0.0 && !ratios.0.contains_key(&item.token) {
ratios.0.insert(item.token.clone(), size.y / size.x);
}
egui::Image::new(&t).uv(cover_uv_for(size, rect.size())).corner_radius(theme::RADIUS).paint_at(ui, rect);
},
Some(Lookup::Loading) => shimmer(ui, rect),
_ => icons::paint(ui.painter(), Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), if item.kind == MediaKind::Video { Icon::Video } else { Icon::Image }, theme::FAINT),
}
if item.kind == MediaKind::Video {
let badge = Rect::from_center_size(rect.center(), vec2(40.0, 40.0));
ui.painter().circle_filled(badge.center(), 20.0, Color32::from_black_alpha(110));
icons::paint(ui.painter(), badge.shrink(12.0).translate(vec2(1.5, 0.0)), Icon::Play, theme::fade(Color32::WHITE, 0.9));
// The pointer may be on an action button (or the audio player) rather than the card itself.
let hovered = resp.hovered() || ui.rect_contains_pointer(rect);
let is_audio = item.kind == MediaKind::Audio;
if is_audio {
audio_face(ui, view, rect, item, audio);
} else {
visual_face(ui, view, rect, item, cache, ratios);
}
let selected = view.selected.contains(&item.token);
let ring = if selected {
@@ -276,25 +271,17 @@ fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cach
p.galley(pos2(badge.left() + 20.0, badge.center().y - galley.size().y / 2.0), galley, Color32::WHITE);
}
let mut action_clicked = false;
// The pointer may be on an action button rather than the card itself.
if (hovered || ui.rect_contains_pointer(rect)) && !view.selecting {
let footer = bottom_gradient(ui, rect, 56.0, 178);
let chip_y = footer.bottom() - 18.0;
let mut x = footer.left() + 8.0;
x = chip(ui, pos2(x, chip_y), Some(if item.kind == MediaKind::Video { Icon::Video } else { Icon::Image }), None, item.kind.label());
if hovered && !view.selecting {
// Audio keeps its player at the bottom, so its chips and actions go on top.
let band = edge_fade(ui, rect, 56.0, 178, is_audio);
let chip_y = if is_audio { band.top() + 18.0 } else { band.bottom() - 18.0 };
let mut x = band.left() + 8.0;
x = chip(ui, pos2(x, chip_y), Some(kind_icon(item.kind)), None, item.kind.label());
if let Some(model) = item.model_id.as_deref() {
chip(ui, pos2(x + 6.0, chip_y), None, Some(&names.creator(model)), &truncate(&names.display_name(model), 16));
}
let mut buttons: Vec<(Icon, &str, FeedAction)> = Vec::new();
if item.prompt_token.is_some() {
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
}
if view.make_video && item.kind == MediaKind::Image {
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
}
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
let pill = Rect::from_min_size(pos2(footer.right() - 8.0 - 8.0 - 28.0 * buttons.len() as f32, chip_y - 18.0), vec2(8.0 + 28.0 * buttons.len() as f32, 36.0));
let buttons = item_actions(view, item, None);
let pill = Rect::from_min_size(pos2(band.right() - 8.0 - 8.0 - 28.0 * buttons.len() as f32, chip_y - 18.0), vec2(8.0 + 28.0 * buttons.len() as f32, 36.0));
ui.painter().rect_filled(pill, 0.0, Color32::from_black_alpha(153));
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(pill.shrink(4.0)).layout(egui::Layout::left_to_right(egui::Align::Center)));
child.spacing_mut().item_spacing.x = 0.0;
@@ -310,6 +297,87 @@ fn item_card(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cach
}
}
/// An image or video card's face: the thumbnail (or animated preview) and a video's play badge.
fn visual_face(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, ratios: &mut RatioCache) {
ui.painter().rect_filled(rect, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
// Videos play their animated preview when previews are on (the still shows while it loads).
let animated = item.animated.as_deref().filter(|_| view.autoplay && item.kind == MediaKind::Video).map(|u| cache.get_animated(ui.ctx(), u));
let lookup = match animated {
Some(Lookup::Ready(t)) => Some(Lookup::Ready(t)),
_ => item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)),
};
match lookup {
Some(Lookup::Ready(t)) => {
let size = t.size_vec2();
if size.x > 0.0 && !ratios.0.contains_key(&item.token) {
ratios.0.insert(item.token.clone(), size.y / size.x);
}
egui::Image::new(&t).uv(cover_uv_for(size, rect.size())).corner_radius(theme::RADIUS).paint_at(ui, rect);
},
Some(Lookup::Loading) => shimmer(ui, rect),
_ => icons::paint(ui.painter(), Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), kind_icon(item.kind), theme::FAINT),
}
if item.kind == MediaKind::Video && !(view.autoplay && item.animated.is_some()) {
let badge = Rect::from_center_size(rect.center(), vec2(40.0, 40.0));
ui.painter().circle_filled(badge.center(), 20.0, Color32::from_black_alpha(110));
icons::paint(ui.painter(), badge.shrink(12.0).translate(vec2(1.5, 0.0)), Icon::Play, theme::fade(Color32::WHITE, 0.9));
}
}
/// An audio card's face (`GalleryCard`'s audio branch, a square tile): the title, a music note
/// and the waveform player along the bottom.
fn audio_face(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, audio: &mut AudioPlayer) {
let mut mesh = Mesh::default();
let (top, bottom) = (theme::fade(theme::ACCENT, 0.14), theme::fade(theme::CONTROLS, 0.6));
mesh.colored_vertex(rect.left_top(), top);
mesh.colored_vertex(rect.right_top(), top);
mesh.colored_vertex(rect.right_bottom(), bottom);
mesh.colored_vertex(rect.left_bottom(), bottom);
mesh.add_triangle(0, 1, 2);
mesh.add_triangle(0, 2, 3);
ui.painter().add(mesh);
// Clear of the selection checkbox when selecting.
let inset = if view.selecting { 36.0 } else { 12.0 };
let title = item.prompt_token.as_deref().and_then(|t| view.prompts.get(t)).cloned().unwrap_or_else(|| fallback_title(item));
let galley = clamp_lines(ui, &title, 13.0, theme::fade(Color32::WHITE, 0.85), rect.width() - inset - 12.0, 2);
ui.painter().galley(pos2(rect.left() + inset, rect.top() + 12.0), galley, Color32::WHITE);
waveform::music_tile(ui, Rect::from_center_size(rect.center() - vec2(0.0, 12.0), vec2(48.0, 48.0)), true);
let bar = Rect::from_min_max(pos2(rect.left() + 12.0, rect.bottom() - 56.0), pos2(rect.right() - 12.0, rect.bottom() - 12.0));
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar));
waveform::player(&mut child, view.id.with(("player", &item.token)), audio, &item.full_url, item.duration_secs, false);
}
/// The hover actions for an item: copy prompt (list rows, when the text is known), recreate,
/// make video, share and download. Audio only shares and downloads (`Audio v1`).
fn item_actions<'a>(view: &FeedView<'_>, item: &FeedItem, prompt: Option<&String>) -> Vec<(Icon, &'a str, FeedAction)> {
let mut buttons = Vec::new();
if let Some(text) = prompt {
buttons.push((Icon::Copy, "Copy prompt", FeedAction::CopyPrompt(text.clone())));
}
if item.prompt_token.is_some() && item.kind != MediaKind::Audio {
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
}
if view.make_video && item.kind == MediaKind::Image {
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
}
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
buttons
}
/// What an item is called without its prompt: its title, else "Audio Generation" and the like.
fn fallback_title(item: &FeedItem) -> String {
item.title.clone().unwrap_or_else(|| format!("{} Generation", item.kind.label()))
}
fn kind_icon(kind: MediaKind) -> Icon {
match kind {
MediaKind::Image => Icon::Image,
MediaKind::Video => Icon::Video,
MediaKind::Audio => Icon::Music,
}
}
fn pending_row(ui: &mut Ui, rect: Rect, job: &PendingJob, now: i64, names: &dyn ModelNames) {
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
ui.painter().rect_filled(thumb, theme::RADIUS, Color32::from_white_alpha(8));
@@ -359,7 +427,7 @@ fn failed_row(ui: &mut Ui, id: Id, rect: Rect, job: &FailedJob, names: &dyn Mode
}
#[allow(clippy::too_many_arguments)]
fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, names: &dyn ModelNames, now: i64, actions: &mut Vec<FeedAction>) {
fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache: &mut MediaCache, audio: &mut AudioPlayer, names: &dyn ModelNames, now: i64, actions: &mut Vec<FeedAction>) {
let resp = ui.interact(rect, view.id.with(("row", &item.token)), Sense::click());
let hovered = resp.hovered();
if hovered {
@@ -367,7 +435,10 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
}
let thumb = Rect::from_min_size(rect.min + vec2(0.0, 12.0), vec2(LIST_THUMB, LIST_THUMB));
ui.painter().rect_filled(thumb, theme::RADIUS, theme::fade(theme::CONTROLS, 0.4));
if let Some(Lookup::Ready(t)) = item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
let is_audio = item.kind == MediaKind::Audio;
if is_audio {
waveform::music_tile(ui, thumb, false);
} else if let Some(Lookup::Ready(t)) = item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
egui::Image::new(&t).uv(crate::prompt_box::deck::cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, thumb);
}
if view.selected.contains(&item.token) {
@@ -378,7 +449,8 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
let prompt = item.prompt_token.as_deref().and_then(|t| view.prompts.get(t));
match prompt {
Some(text) => {
let g = clamp_lines(ui, text, 14.0, theme::INK, w, 3);
// Audio keeps room for its player under the text.
let g = clamp_lines(ui, text, 14.0, theme::INK, w, if is_audio { 2 } else { 3 });
ui.painter().galley(pos2(x, thumb.top()), g, theme::INK);
},
None if item.prompt_token.is_some() => {
@@ -387,9 +459,13 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
}
},
None => {
ui.painter().text(pos2(x, thumb.top() + 8.0), Align2::LEFT_CENTER, format!("{} Generation", item.kind.label()), FontId::new(14.0, FontFamily::Proportional), theme::MUTED);
ui.painter().text(pos2(x, thumb.top() + 8.0), Align2::LEFT_CENTER, fallback_title(item), FontId::new(14.0, FontFamily::Proportional), theme::MUTED);
},
}
if is_audio {
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(Rect::from_min_size(pos2(x, thumb.top() + 44.0), vec2(w.min(LIST_PLAYER_MAX), 32.0))));
waveform::player(&mut child, view.id.with(("row-player", &item.token)), audio, &item.full_url, item.duration_secs, true);
}
let meta_y = thumb.bottom() - 10.0;
let mut mx = x;
if let Some(model) = item.model_id.as_deref() {
@@ -400,18 +476,7 @@ fn item_row(ui: &mut Ui, view: &FeedView<'_>, rect: Rect, item: &FeedItem, cache
ui.painter().text(pos2(rect.right(), thumb.top() + 8.0), Align2::RIGHT_CENTER, time_ago(item.created_at, now), FontId::new(11.0, FontFamily::Proportional), theme::FAINT);
let mut action_clicked = false;
if hovered && !view.selecting {
let mut buttons: Vec<(Icon, &str, FeedAction)> = Vec::new();
if let Some(text) = prompt {
buttons.push((Icon::Copy, "Copy prompt", FeedAction::CopyPrompt(text.clone())));
}
if item.prompt_token.is_some() {
buttons.push((Icon::RotateCw, "Recreate", FeedAction::Recreate(item.token.clone())));
}
if view.make_video && item.kind == MediaKind::Image {
buttons.push((Icon::Video, "Make Video", FeedAction::MakeVideo(item.token.clone())));
}
buttons.push((Icon::Link, "Share", FeedAction::Share(item.token.clone())));
buttons.push((Icon::Download, "Download", FeedAction::Download(item.token.clone())));
let buttons = item_actions(view, item, prompt);
let bar = Rect::from_min_size(pos2(rect.right() - 28.0 * buttons.len() as f32, thumb.bottom() - 28.0), vec2(28.0 * buttons.len() as f32, 28.0));
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar).layout(egui::Layout::left_to_right(egui::Align::Center)));
child.spacing_mut().item_spacing.x = 0.0;
@@ -464,9 +529,15 @@ fn model_chip(ui: &Ui, left_center: egui::Pos2, model_id: &str, names: &dyn Mode
/// Darkens the bottom of `rect` (`bg-gradient-to-t from-black/70`), returning the faded band.
fn bottom_gradient(ui: &Ui, rect: Rect, height: f32, alpha: u8) -> Rect {
let band = Rect::from_min_max(pos2(rect.left(), (rect.bottom() - height).max(rect.top())), rect.max);
edge_fade(ui, rect, height, alpha, false)
}
/// Darkens `rect` toward its top or bottom edge over `height`, returning the faded band.
fn edge_fade(ui: &Ui, rect: Rect, height: f32, alpha: u8, top_edge: bool) -> Rect {
let band = if top_edge { Rect::from_min_max(rect.min, pos2(rect.right(), (rect.top() + height).min(rect.bottom()))) } else { Rect::from_min_max(pos2(rect.left(), (rect.bottom() - height).max(rect.top())), rect.max) };
let mut mesh = Mesh::default();
let (top, bottom) = (Color32::TRANSPARENT, Color32::from_black_alpha(alpha));
let (clear, dark) = (Color32::TRANSPARENT, Color32::from_black_alpha(alpha));
let (top, bottom) = if top_edge { (dark, clear) } else { (clear, dark) };
mesh.colored_vertex(band.left_top(), top);
mesh.colored_vertex(band.right_top(), top);
mesh.colored_vertex(band.right_bottom(), bottom);
+76 -13
View File
@@ -3,15 +3,19 @@
use std::collections::{HashMap, HashSet};
use crate::backend::wire::{Job, JobPhase, MediaFile, unix_secs};
use crate::backend::EnqueueMeta;
use crate::backend::providers::Provider;
use crate::backend::providers::jobs::ProviderJob;
use crate::backend::wire::{Job, JobPhase, MediaFile, unix_secs};
use crate::feed::types::{FailedJob, FeedItem, MediaKind, PendingJob};
use crate::models::Catalog;
use crate::models::{self, Catalog};
/// Failures older than this are history, not news.
const FAILURE_WINDOW_SECS: i64 = 24 * 60 * 60;
/// Thumbnails are requested at this width (`getMediaThumbnail(@512)`).
pub const THUMBNAIL_WIDTH: u32 = 512;
/// Animated previews are requested smaller: every frame becomes a texture.
const ANIMATED_WIDTH: u32 = 320;
/// Something the store wants the app to do or say after an update.
#[derive(Debug, PartialEq)]
@@ -54,12 +58,12 @@ impl FeedStore {
/// Shows the jobs a generate call just started, before the next poll catches up.
pub fn add_enqueued(&mut self, kind: MediaKind, job_tokens: &[String], meta: &EnqueueMeta, catalog: &Catalog) {
let now = chrono::Utc::now().timestamp();
let expected = catalog.find(&meta.model_id).map_or(30.0, |m| m.expected_secs);
let expected = catalog.find(&meta.model_id).map_or_else(|| models::expected_secs(&meta.model_id, kind), |m| m.expected_secs);
for token in job_tokens {
self.meta.insert(token.clone(), meta.clone());
self.seen_running.insert(token.clone());
if !self.pending.iter().any(|p| &p.job_token == token) {
self.pending.insert(0, PendingJob { job_token: token.clone(), kind, prompt: meta.prompt.clone(), model_id: Some(meta.model_id.clone()), created_at: now, server_progress: None, expected_secs: expected, batch_count: meta.batch_count });
self.pending.insert(0, PendingJob { job_token: token.clone(), kind, prompt: meta.prompt.clone(), model_id: Some(meta.model_id.clone()), created_at: now, server_progress: None, expected_secs: expected, batch_count: meta.batch_count, local: meta.provider != Provider::Artcraft });
}
}
}
@@ -79,8 +83,8 @@ impl FeedStore {
match job.status.phase() {
JobPhase::Running => {
self.seen_running.insert(job.job_token.clone());
let expected = model_id.as_deref().and_then(|id| catalog.find(id)).map_or(if kind == MediaKind::Video { 900.0 } else { 30.0 }, |m| m.expected_secs);
pending.push(PendingJob { job_token: job.job_token.clone(), kind, prompt, model_id, created_at: unix_secs(&job.created_at), server_progress: Some(job.status.progress_percentage), expected_secs: expected, batch_count: meta.map_or(1, |m| m.batch_count) });
let expected = model_id.as_deref().and_then(|id| catalog.find(id)).map_or_else(|| models::expected_secs(model_id.as_deref().unwrap_or_default(), kind), |m| m.expected_secs);
pending.push(PendingJob { job_token: job.job_token.clone(), kind, prompt, model_id, created_at: unix_secs(&job.created_at), server_progress: Some(job.status.progress_percentage), expected_secs: expected, batch_count: meta.map_or(1, |m| m.batch_count), local: false });
},
JobPhase::Failed => {
let updated = unix_secs(&job.updated_at);
@@ -88,7 +92,7 @@ impl FeedStore {
notices.push(FeedNotice::Failed(job.status.maybe_failure_message.clone().unwrap_or_else(|| job.status.failure_reason())));
}
if now - updated <= FAILURE_WINDOW_SECS {
failed.push(FailedJob { job_token: job.job_token.clone(), prompt, model_id, updated_at: updated, reason: job.status.failure_reason(), message: job.status.maybe_failure_message.clone().filter(|_| job.status.maybe_failure_category_updated.as_deref() != Some("unknown")), ref_image: meta.and_then(|m| m.ref_image.clone()) });
failed.push(FailedJob { job_token: job.job_token.clone(), prompt, model_id, updated_at: updated, reason: job.status.failure_reason(), message: job.status.maybe_failure_message.clone().filter(|_| job.status.maybe_failure_category_updated.as_deref() != Some("unknown")), ref_image: meta.and_then(|m| m.ref_image.clone()), local: false });
}
},
JobPhase::Succeeded => {
@@ -103,20 +107,23 @@ impl FeedStore {
};
// The finished card takes the pending card's place: date it by the job.
let created_at = unix_secs(&job.created_at);
self.push_item(FeedItem { token: result.entity_token.clone(), kind, thumbnail: result.media_links.thumbnail(THUMBNAIL_WIDTH), full_url: result.media_links.cdn_url.clone(), created_at, model_id: model_id.clone(), prompt_token: job.request.maybe_prompt_token.clone(), batch_token: result.maybe_batch_token.clone(), duration_secs: None });
let thumbnail = result.media_links.thumbnail(THUMBNAIL_WIDTH).filter(|_| kind != MediaKind::Audio);
self.push_item(FeedItem { token: result.entity_token.clone(), kind, title: None, thumbnail, full_url: result.media_links.cdn_url.clone(), created_at, model_id: model_id.clone(), prompt_token: job.request.maybe_prompt_token.clone(), batch_token: result.maybe_batch_token.clone(), duration_secs: None, animated: result.media_links.animated_preview(ANIMATED_WIDTH) });
if let Some(batch) = &result.maybe_batch_token {
notices.push(FeedNotice::LoadBatch { job_token: job.job_token.clone(), batch_token: batch.clone() });
}
},
}
}
// Jobs started a moment ago may not be in the list yet: keep them.
// Jobs started a moment ago may not be in the list yet, and other providers' never are: keep
// them.
for p in self.pending.drain(..) {
let listed = jobs.iter().any(|j| j.job_token == p.job_token);
if !listed && now - p.created_at < 60 && !pending.iter().any(|q: &PendingJob| q.job_token == p.job_token) {
if (p.local || (!listed && now - p.created_at < 60)) && !pending.iter().any(|q: &PendingJob| q.job_token == p.job_token) {
pending.push(p);
}
}
failed.extend(self.failed.drain(..).filter(|f| f.local));
// Everything finished before the first poll comes from the library instead.
self.first_poll_done = true;
pending.sort_by_key(|p| std::cmp::Reverse(p.created_at));
@@ -137,6 +144,35 @@ impl FeedStore {
}
}
/// Pending cards for the Midjourney and fal jobs still running (after a restart).
pub fn restore_local(&mut self, jobs: &[ProviderJob], catalog: &Catalog) {
for job in jobs {
if !self.pending.iter().any(|p| p.job_token == job.id) {
let expected = catalog.find(&job.model_id).map_or_else(|| models::expected_secs(&job.model_id, MediaKind::Image), |m| m.expected_secs);
self.pending.push(PendingJob { job_token: job.id.clone(), kind: MediaKind::Image, prompt: job.prompt.clone(), model_id: Some(job.model_id.clone()), created_at: job.created_at, server_progress: None, expected_secs: expected, batch_count: job.batch_count, local: true });
}
}
self.pending.sort_by_key(|p| std::cmp::Reverse(p.created_at));
}
/// A Midjourney or fal job's results are in: its card goes (the batch brings the images).
/// Returns whether it was this feed's.
pub fn finish_local(&mut self, job_id: &str) -> bool {
let before = self.pending.len();
self.pending.retain(|p| p.job_token != job_id);
self.pending.len() != before
}
/// A Midjourney or fal job failed: its card shows the reason until dismissed.
pub fn fail_local(&mut self, job_id: &str, reason: &str) -> bool {
let Some(i) = self.pending.iter().position(|p| p.job_token == job_id) else {
return false;
};
let p = self.pending.remove(i);
self.failed.insert(0, FailedJob { job_token: p.job_token, prompt: p.prompt, model_id: p.model_id, updated_at: chrono::Utc::now().timestamp(), reason: "Generation failed".to_owned(), message: Some(reason.to_owned()), ref_image: self.meta.get(job_id).and_then(|m| m.ref_image.clone()), local: true });
true
}
pub fn library_started(&self) -> bool {
self.library_started
}
@@ -207,19 +243,23 @@ pub fn feed_item(kind: MediaKind, file: &MediaFile) -> FeedItem {
kind: match file.media_class.as_deref() {
Some("video") => MediaKind::Video,
Some("image") => MediaKind::Image,
Some("audio") => MediaKind::Audio,
_ => kind,
},
thumbnail: file.media_links.thumbnail(THUMBNAIL_WIDTH),
title: file.maybe_title.clone().filter(|t| !t.is_empty()),
thumbnail: file.thumbnail(THUMBNAIL_WIDTH),
full_url: file.media_links.cdn_url.clone(),
created_at: unix_secs(&file.created_at),
model_id: file.maybe_model_type.clone().or_else(|| file.maybe_origin_model_type.clone()),
prompt_token: file.maybe_prompt_token.clone(),
batch_token: file.maybe_batch_token.clone(),
duration_secs: file.maybe_duration_millis.map(|ms| ms as f32 / 1000.0),
animated: file.media_links.animated_preview(ANIMATED_WIDTH),
}
}
/// Whether a job made an image or a video: by its model, else by its category.
/// Whether a job made an image, a video or audio: by its model, else by its category
/// (`useGenerationJobs` checks video before image before audio).
fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
if let Some(model) = job.request.maybe_model_type.as_deref() {
if catalog.image.iter().any(|m| m.id == model) {
@@ -228,12 +268,17 @@ fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
if catalog.video.iter().any(|m| m.id == model) {
return Some(MediaKind::Video);
}
if catalog.audio.iter().any(|m| m.id == model) {
return Some(MediaKind::Audio);
}
}
let category = job.request.inference_category.as_str();
if category.contains("video") {
Some(MediaKind::Video)
} else if category.contains("image") {
Some(MediaKind::Image)
} else if category.contains("audio") {
Some(MediaKind::Audio)
} else {
None
}
@@ -242,6 +287,7 @@ fn job_kind(job: &Job, catalog: &Catalog) -> Option<MediaKind> {
#[cfg(test)]
mod tests {
use super::*;
use crate::backend::providers::Provider;
use crate::models::ModelInfo;
fn catalog() -> Catalog {
@@ -314,10 +360,27 @@ mod tests {
assert_eq!(store.pending.len(), 1);
}
#[test]
fn provider_jobs_outlive_polls_until_they_finish_or_fail() {
let (c, mut store) = (catalog(), FeedStore::default());
let meta = EnqueueMeta { prompt: "p".into(), model_id: "midjourney_7".into(), batch_count: 4, ref_image: None, provider: Provider::Midjourney };
store.add_enqueued(MediaKind::Image, &["local_a".into(), "local_b".into()], &meta, &c);
store.pending.iter_mut().for_each(|p| p.created_at -= 3600);
store.apply_jobs(MediaKind::Image, &[], &c);
assert_eq!(store.pending.len(), 2, "the server never lists them");
assert!(store.finish_local("local_a"));
assert!(store.fail_local("local_b", "Midjourney didn't finish this job"));
store.apply_jobs(MediaKind::Image, &[], &c);
assert!(store.pending.is_empty());
assert_eq!(store.failed[0].message.as_deref(), Some("Midjourney didn't finish this job"), "the failure stays until dismissed");
store.dismiss("local_b");
assert!(store.failed.is_empty());
}
#[test]
fn just_enqueued_jobs_survive_a_poll_that_predates_them() {
let (c, mut store) = (catalog(), FeedStore::default());
let meta = EnqueueMeta { prompt: "p".into(), model_id: "nano_banana_pro".into(), batch_count: 2, ref_image: None };
let meta = EnqueueMeta { prompt: "p".into(), model_id: "nano_banana_pro".into(), batch_count: 2, ref_image: None, provider: Provider::Artcraft };
store.add_enqueued(MediaKind::Image, &["fresh".into()], &meta, &c);
store.apply_jobs(MediaKind::Image, &[], &c);
assert_eq!(store.pending.len(), 1);
+13 -2
View File
@@ -4,6 +4,7 @@
pub enum MediaKind {
Image,
Video,
Audio,
}
impl MediaKind {
@@ -11,6 +12,7 @@ impl MediaKind {
match self {
MediaKind::Image => "Image",
MediaKind::Video => "Video",
MediaKind::Audio => "Audio",
}
}
@@ -18,17 +20,20 @@ impl MediaKind {
match self {
MediaKind::Image => "images",
MediaKind::Video => "videos",
MediaKind::Audio => "audio clips",
}
}
}
/// A finished image or video in the user's library.
/// A finished image, video or audio clip in the user's library.
#[derive(Clone, Debug)]
pub struct FeedItem {
/// The media file token.
pub token: String,
pub kind: MediaKind,
/// A thumbnail URL (512 px wide where the CDN can resize).
/// The file's title (`maybe_title`), when it has one.
pub title: Option<String>,
/// A thumbnail URL (512 px wide where the CDN can resize); audio has none.
pub thumbnail: Option<String>,
/// The original file.
pub full_url: String,
@@ -39,6 +44,8 @@ pub struct FeedItem {
pub prompt_token: Option<String>,
pub batch_token: Option<String>,
pub duration_secs: Option<f32>,
/// A video's animated preview (shown when previews play).
pub animated: Option<String>,
}
/// A generation the server is still working on.
@@ -55,6 +62,8 @@ pub struct PendingJob {
/// How long this model usually takes, for the estimated progress bar.
pub expected_secs: f32,
pub batch_count: u32,
/// Running at Midjourney or fal (tracked here, not in the server's job list).
pub local: bool,
}
/// A generation that failed or was cancelled.
@@ -68,6 +77,8 @@ pub struct FailedJob {
pub message: Option<String>,
/// The first reference image, shown faintly behind the error.
pub ref_image: Option<String>,
/// A Midjourney or fal job (tracked here, not in the server's job list).
pub local: bool,
}
/// The progress and time-left labels for a pending job (`derivePendingStatus`).
+105 -7
View File
@@ -8,10 +8,15 @@ use crate::feed::types::MediaKind;
/// Default models (`defaultModelForPage.ts`).
pub const DEFAULT_IMAGE_MODEL: &str = "nano_banana_pro";
pub const DEFAULT_VIDEO_MODEL: &str = "seedance_2p0";
pub const DEFAULT_AUDIO_MODEL: &str = "suno_music";
/// The fallback prompt limit (`maxPromptLength ?? 1000`).
pub const DEFAULT_PROMPT_MAX: usize = 1000;
/// The model pickers group by family once they list this many models.
const GROUP_THRESHOLD: usize = 8;
/// Audio models that work from exactly one audio track (`AUDIO_MODELS_REQUIRING_AUDIO_REF`).
const AUDIO_MODELS_REQUIRING_AUDIO_REF: [&str; 2] = ["suno_remix", "suno_sample"];
/// The musical keys (`SoundsSettingsPopover`; there are no E keys): value, label, short label.
pub const MUSICAL_KEYS: [(&str, &str, &str); 13] = [("auto", "Auto", "Auto"), ("c_major", "C Major", "C"), ("c_minor", "C Minor", "Cm"), ("d_major", "D Major", "D"), ("d_minor", "D Minor", "Dm"), ("f_major", "F Major", "F"), ("f_minor", "F Minor", "Fm"), ("g_major", "G Major", "G"), ("g_minor", "G Minor", "Gm"), ("a_major", "A Major", "A"), ("a_minor", "A Minor", "Am"), ("b_major", "B Major", "B"), ("b_minor", "B Minor", "Bm")];
/// One generation model and what it accepts.
#[derive(Clone, Debug, Default)]
@@ -45,6 +50,8 @@ pub struct ModelInfo {
pub video_refs_max_secs: Option<u16>,
pub audio_refs_max: usize,
pub audio_refs_max_secs: Option<u16>,
/// `@Character` mentions (0 = not supported).
pub character_refs_max: usize,
pub duration_options: Vec<u16>,
pub duration_min: Option<u16>,
pub duration_max: Option<u16>,
@@ -54,10 +61,42 @@ pub struct ModelInfo {
pub bitrate_default: Option<String>,
pub output_formats: Vec<String>,
pub output_format_default: Option<String>,
/// Audio only.
pub audio: AudioCaps,
/// How long a generation usually takes (drives the estimated progress bar).
pub expected_secs: f32,
}
/// What an audio model takes besides the prompt and references.
#[derive(Clone, Debug, Default, PartialEq)]
pub struct AudioCaps {
pub style_prompt: bool,
pub instrumental: bool,
pub keep_lyrics: bool,
pub loopable: bool,
pub bpm: bool,
pub musical_key: bool,
pub sample_rates: Vec<u32>,
pub sample_rate_default: Option<u32>,
pub speed: bool,
pub volume: bool,
pub pitch: bool,
/// Remixes and samples work from exactly one audio track.
pub requires_audio_ref: bool,
}
impl AudioCaps {
/// Whether the Tuning popover shows.
pub fn has_tuning(&self) -> bool {
!self.sample_rates.is_empty() || self.speed || self.volume || self.pitch
}
/// `picked` if the model offers it, else its default, else its first rate.
pub fn sample_rate(&self, picked: Option<u32>) -> Option<u32> {
picked.filter(|hz| self.sample_rates.contains(hz)).or(self.sample_rate_default).or_else(|| self.sample_rates.first().copied())
}
}
impl ModelInfo {
/// Batch sizes the count picker offers (`predefined` or `1..=max`).
pub fn batch_options(&self) -> Vec<u16> {
@@ -162,6 +201,7 @@ pub fn video_model(d: &artcraft_client::endpoints::omni_gen::models::video::omni
m.video_refs_max_secs = d.video_references_max_total_duration_seconds;
m.audio_refs_max = max(d.audio_references_supported, d.audio_references_max);
m.audio_refs_max_secs = d.audio_references_max_total_duration_seconds;
m.character_refs_max = max(d.character_references_supported, d.character_references_max);
m.aspect_ratios = strings(&d.aspect_ratio_options);
m.aspect_default = d.aspect_ratio_default.clone().map(String::from);
m.resolutions = strings(&d.resolution_options);
@@ -182,6 +222,22 @@ pub fn video_model(d: &artcraft_client::endpoints::omni_gen::models::video::omni
m
}
/// Builds a model from the server's audio listing.
pub fn audio_model(d: &crate::backend::wire::AudioModel) -> ModelInfo {
let mut m = base(&d.model, MediaKind::Audio, d.full_name.as_deref(), d.model_creator.clone());
if let Some(short) = d.extra_info_short.as_deref().filter(|s| !s.is_empty()) {
m.description = short.to_owned();
}
m.disabled = d.is_disabled;
m.prompt_supported = d.text_prompt_supported.unwrap_or(true);
m.prompt_max = None;
let max = |supported: bool, max: Option<u16>| if supported { max.map_or(1, usize::from) } else { 0 };
m.audio_refs_max = max(d.audio_references_supported, d.audio_references_max);
m.image_refs_max = max(d.image_references_supported, d.image_references_max);
m.audio = AudioCaps { style_prompt: d.style_prompt_supported, instrumental: d.instrumental_toggle_supported, keep_lyrics: d.keep_lyrics_supported, loopable: d.loopable_toggle_supported, bpm: d.bpm_supported, musical_key: d.musical_key_supported, sample_rates: d.sample_rate_hz_options.clone(), sample_rate_default: d.sample_rate_hz_default, speed: d.speed_supported, volume: d.volume_supported, pitch: d.pitch_supported, requires_audio_ref: AUDIO_MODELS_REQUIRING_AUDIO_REF.contains(&d.model.as_str()) };
m
}
fn strings<T: Clone + Into<String>>(options: &Option<Vec<T>>) -> Vec<String> {
options.as_deref().unwrap_or_default().iter().cloned().map(Into::into).collect()
}
@@ -209,7 +265,7 @@ fn base(id: &str, kind: MediaKind, full_name: Option<&str>, creator: Option<Stri
/// Families whose picker names the app curates (the static model lists' `selectorName`s).
fn has_curated_name(id: &str) -> bool {
const PREFIXES: [&str; 11] = ["flux_", "nano_banana", "gpt_image", "seedream", "seedance", "kling", "veo_", "sora", "happy_horse", "grok_imagine", "qwen"];
const PREFIXES: [&str; 13] = ["flux_", "nano_banana", "gpt_image", "seedream", "seedance", "kling", "veo_", "sora", "happy_horse", "grok_imagine", "qwen", "suno", "seed_audio"];
PREFIXES.iter().any(|p| id.starts_with(p))
}
@@ -289,12 +345,17 @@ fn description(id: &str) -> &'static str {
"grok_video" | "grok_imagine_video" => "Fastest video model",
"sora_2" => "Smart video model",
"veo_3" => "Slow, high-quality model",
"suno_music" => "Full songs from a text prompt",
"suno_remix" => "Remix an existing track",
"suno_sounds" => "Sound effects with beat control",
"suno_sample" => "Build a song from a sample",
"seed_audio_1p0" => "Sound generation with fine tuning",
_ => "",
}
}
/// Typical generation time (the static list's `progressBarTime`).
fn expected_secs(id: &str, kind: MediaKind) -> f32 {
/// Typical generation time (the static list's `progressBarTime`, else the feed's per-kind default).
pub fn expected_secs(id: &str, kind: MediaKind) -> f32 {
match id {
"grok_image" | "grok_imagine_image" | "flux_pro_1p1" | "flux_1_dev" | "flux_1_schnell" => 10.0,
"grok_video" | "grok_imagine_video" => 50.0,
@@ -304,19 +365,20 @@ fn expected_secs(id: &str, kind: MediaKind) -> f32 {
"gpt_image_1" | "gpt_image_1p5" | "gpt_image_2p5_flare" | "seedream_4" | "seedream_4p5" | "seedream_5_lite" => 60.0,
"gpt_image_2" | "gpt_image_2p5_sunburst" => 120.0,
_ if kind == MediaKind::Video => 900.0,
_ if kind == MediaKind::Audio => 120.0,
_ => 30.0,
}
}
/// The maker, by id prefix, when the server doesn't say.
pub fn creator_for(id: &str) -> &'static str {
const PREFIXES: [(&str, &str); 16] = [("flux", "black_forest_labs"), ("nano_banana", "google"), ("gemini", "google"), ("veo", "google"), ("imagen", "google"), ("gpt_image", "openai"), ("sora", "openai"), ("seedream", "bytedance"), ("seedance", "bytedance"), ("seededit", "bytedance"), ("kling", "kling"), ("grok", "grok"), ("midjourney", "midjourney"), ("qwen", "alibaba"), ("happy_horse", "alibaba"), ("minimax", "minimax")];
const PREFIXES: [(&str, &str); 18] = [("suno", "suno"), ("seed_audio", "bytedance"), ("flux", "black_forest_labs"), ("nano_banana", "google"), ("gemini", "google"), ("veo", "google"), ("imagen", "google"), ("gpt_image", "openai"), ("sora", "openai"), ("seedream", "bytedance"), ("seedance", "bytedance"), ("seededit", "bytedance"), ("kling", "kling"), ("grok", "grok"), ("midjourney", "midjourney"), ("qwen", "alibaba"), ("happy_horse", "alibaba"), ("minimax", "minimax")];
PREFIXES.iter().find(|(p, _)| id.starts_with(p)).map_or("artcraft", |(_, c)| c)
}
/// The picker family (`getModelFamilyName`).
fn family(id: &str) -> &'static str {
const FAMILIES: [(&str, &str); 22] = [("seedance", "Seedance"), ("kling", "Kling"), ("veo", "Veo"), ("sora", "Sora"), ("happy_horse", "Happy Horse"), ("wan", "Wan"), ("vidu", "Vidu"), ("preview", "Preview"), ("nano_banana", "Nano Banana"), ("gemini_25_flash", "Nano Banana"), ("seedream", "Seedream"), ("seededit", "SeedEdit"), ("gpt_image", "GPT Image"), ("flux", "Flux"), ("midjourney", "Midjourney"), ("grok", "Grok"), ("qwen", "Qwen"), ("recraft", "Recraft"), ("imagen", "Imagen"), ("minimax", "MiniMax"), ("marble", "World Labs"), ("suno", "Suno")];
const FAMILIES: [(&str, &str); 23] = [("seed_audio", "Seed Audio"), ("seedance", "Seedance"), ("kling", "Kling"), ("veo", "Veo"), ("sora", "Sora"), ("happy_horse", "Happy Horse"), ("wan", "Wan"), ("vidu", "Vidu"), ("preview", "Preview"), ("nano_banana", "Nano Banana"), ("gemini_25_flash", "Nano Banana"), ("seedream", "Seedream"), ("seededit", "SeedEdit"), ("gpt_image", "GPT Image"), ("flux", "Flux"), ("midjourney", "Midjourney"), ("grok", "Grok"), ("qwen", "Qwen"), ("recraft", "Recraft"), ("imagen", "Imagen"), ("minimax", "MiniMax"), ("marble", "World Labs"), ("suno", "Suno")];
FAMILIES.iter().find(|(p, _)| id.starts_with(p)).map_or("Other", |(_, f)| f)
}
@@ -386,13 +448,16 @@ fn video_page_models(models: &[ModelInfo]) -> Vec<ModelInfo> {
list
}
/// Both catalogs, for looking up names in the feed, and each create page's picker list.
/// Every catalog, for looking up names in the feed, and each create page's picker list.
#[derive(Default)]
pub struct Catalog {
pub image: Vec<ModelInfo>,
pub video: Vec<ModelInfo>,
pub audio: Vec<ModelInfo>,
pub image_page: Vec<ModelInfo>,
pub video_page: Vec<ModelInfo>,
/// The audio page's list: enabled, in the server's order.
pub audio_page: Vec<ModelInfo>,
}
impl Catalog {
@@ -406,8 +471,13 @@ impl Catalog {
self.video = models;
}
pub fn set_audio(&mut self, models: Vec<ModelInfo>) {
self.audio_page = models.iter().filter(|m| !m.disabled).cloned().collect();
self.audio = models;
}
pub fn find(&self, id: &str) -> Option<&ModelInfo> {
self.image.iter().chain(&self.video).find(|m| m.id == id)
self.image.iter().chain(&self.video).chain(&self.audio).find(|m| m.id == id)
}
}
@@ -502,6 +572,15 @@ pub fn quality_label(value: &str) -> String {
.to_owned()
}
/// Sample rate labels (`formatSampleRateHz`): 44100 → "44.1 kHz", 24000 → "24 kHz".
pub fn sample_rate_label(hz: u32) -> String {
if hz.is_multiple_of(1000) {
format!("{} kHz", hz / 1000)
} else {
format!("{:.1} kHz", hz as f32 / 1000.0)
}
}
/// Bitrate labels.
pub fn bitrate_label(value: &str) -> String {
match value {
@@ -530,6 +609,8 @@ mod tests {
assert_eq!(display_name("kling_3p0_pro"), "Kling 3.0 Pro");
assert_eq!(display_name("nano_banana"), "Nano Banana");
assert_eq!(display_name("seedance_2p0_bpu"), "Seedance 2.0 Plus Ultra");
assert_eq!(display_name("seed_audio_1p0"), "Seed Audio 1.0");
assert_eq!(display_name("suno_music"), "Suno Music");
}
#[test]
@@ -544,6 +625,8 @@ mod tests {
#[test]
fn falls_back_to_creator_by_prefix() {
assert_eq!(creator_for("seed_audio_1p0"), "bytedance");
assert_eq!(creator_for("suno_remix"), "suno");
assert_eq!(creator_for("seedream_4p5"), "bytedance");
assert_eq!(creator_for("veo_3"), "google");
assert_eq!(creator_for("something_new"), "artcraft");
@@ -573,6 +656,21 @@ mod tests {
assert_eq!(ModelInfo::resolve(None, &[], None), None);
}
#[test]
fn audio_listing_maps_capabilities() {
let d: crate::backend::wire::AudioModel = serde_json::from_value(serde_json::json!({ "model": "seed_audio_1p0", "model_creator": "artcraft", "audio_references_supported": true, "audio_references_max": 3, "image_references_supported": true, "sample_rate_hz_options": [8000, 24000, 44100], "sample_rate_hz_default": 24000, "speed_supported": true })).unwrap();
let m = audio_model(&d);
assert_eq!((m.name.as_str(), m.creator.as_str(), m.description.as_str()), ("Seed Audio 1.0", "bytedance", "Sound generation with fine tuning"));
assert_eq!((m.audio_refs_max, m.image_refs_max, m.prompt_max), (3, 1, None));
assert!(m.audio.has_tuning() && !m.audio.requires_audio_ref && !m.audio.style_prompt);
assert_eq!(m.audio.sample_rate(Some(44100)), Some(44100));
assert_eq!(m.audio.sample_rate(Some(11025)), Some(24000), "unsupported picks fall back to the default");
let remix = audio_model(&serde_json::from_value(serde_json::json!({ "model": "suno_remix", "audio_references_supported": true })).unwrap());
assert!(remix.audio.requires_audio_ref && !remix.audio.has_tuning());
assert_eq!((remix.audio_refs_max, remix.audio.sample_rate(None)), (1, None));
assert_eq!((sample_rate_label(44100).as_str(), sample_rate_label(24000).as_str()), ("44.1 kHz", "24 kHz"));
}
#[test]
fn batch_counts_snap_to_valid_values() {
let mut m = model("grok_imagine_image");
+12 -3
View File
@@ -2,6 +2,7 @@
use egui::{Color32, Id, Sense, Stroke, Ui, pos2, vec2};
use crate::overlays::providers::{AccountAction, AccountsView, accounts_section};
use crate::shell::{WEB_APP_URL, group_thousands};
use crate::theme;
use crate::ui::icons::Icon;
@@ -105,10 +106,12 @@ pub enum SettingsAction {
Close,
SignOut,
SignIn,
Account(AccountAction),
}
/// The settings dialog (Settings → Misc and Account in the webapp).
pub fn settings_dialog(ctx: &egui::Context, enter_to_generate: &mut bool, username: Option<&str>, credits: Option<u64>, data_root: &std::path::Path) -> Option<SettingsAction> {
/// The settings dialog (Settings → Misc and Accounts in the webapp). `fal_key` is the key being
/// typed.
pub fn settings_dialog(ctx: &egui::Context, enter_to_generate: &mut bool, username: Option<&str>, credits: Option<u64>, accounts: &AccountsView, fal_key: &mut String, data_root: &std::path::Path) -> Option<SettingsAction> {
let mut action = None;
let close = widgets::modal(ctx, Id::new("settings"), 520.0, |ui| {
ui.horizontal(|ui| {
@@ -160,6 +163,12 @@ pub fn settings_dialog(ctx: &egui::Context, enter_to_generate: &mut bool, userna
},
}
ui.add_space(18.0);
widgets::hud(ui, "Provider accounts", theme::FAINT);
ui.add_space(6.0);
if let Some(account) = accounts_section(ui, accounts, fal_key) {
action = Some(SettingsAction::Account(account));
}
ui.add_space(18.0);
widgets::hud(ui, "About", theme::FAINT);
ui.add_space(6.0);
ui.label(egui::RichText::new(format!("ArtCraft native {}", env!("CARGO_PKG_VERSION"))).size(13.0).color(theme::MUTED));
@@ -176,7 +185,7 @@ pub fn settings_dialog(ctx: &egui::Context, enter_to_generate: &mut bool, userna
}
/// A single-line input on the sunken surface with a hairline (and a blue line while focused).
fn text_field(ui: &mut Ui, value: &mut String, password: bool, hint: &str) -> egui::Response {
pub(crate) fn text_field(ui: &mut Ui, value: &mut String, password: bool, hint: &str) -> egui::Response {
let id = ui.next_auto_id();
let focused = ui.memory(|m| m.has_focus(id));
egui::Frame::new().fill(theme::SUNKEN).stroke(Stroke::new(1.0, if focused { theme::ACCENT } else { theme::LINE })).corner_radius(theme::RADIUS).inner_margin(egui::Margin::symmetric(10, 8)).show(ui, |ui| ui.add(egui::TextEdit::singleline(value).id(id).password(password).frame(egui::Frame::NONE).desired_width(f32::INFINITY).hint_text(egui::RichText::new(hint).color(theme::FAINT)))).inner
+402
View File
@@ -0,0 +1,402 @@
//! The characters dialog (`CharactersModal`): pick a saved character to mention it in the prompt,
//! make a new one from a reference image, rename or re-describe one, or delete it.
use std::path::PathBuf;
use std::sync::Arc;
use egui::{Align2, Color32, FontId, Id, Rect, Sense, Stroke, Ui, pos2, vec2};
use crate::backend::UploadedMedia;
use crate::backend::media_cache::{Lookup, MediaCache};
use crate::backend::wire::Character;
use crate::prompt_box::deck::cover_uv;
use crate::prompt_box::refs::{RefKind, RefMedia, RefStatus};
use crate::theme;
use crate::ui::icons::{self, Icon};
use crate::ui::widgets;
const COLUMNS: usize = 4;
const GAP: f32 = 12.0;
#[derive(Clone, PartialEq)]
enum View {
List,
Edit(String),
Create,
}
pub struct CharactersModal {
view: View,
confirm_delete: Option<(String, String)>,
name: String,
description: String,
image: Option<RefMedia>,
}
/// Something only the app can do for the dialog.
#[derive(Debug)]
pub enum CharactersAction {
Close,
/// Mention this character in the prompt.
Select {
token: String,
name: String,
},
Upload {
ref_id: u64,
path: PathBuf,
},
Create {
image_token: String,
name: String,
description: Option<String>,
},
Edit {
token: String,
name: String,
description: String,
},
Delete {
token: String,
name: String,
},
Preview(String),
Toast(String),
}
impl CharactersModal {
pub fn new() -> Self {
Self { view: View::List, confirm_delete: None, name: String::new(), description: String::new(), image: None }
}
/// The reference image being uploaded for a new character, if `ref_id` is it.
pub fn apply_upload(&mut self, ref_id: u64, result: Result<UploadedMedia, String>) -> Option<Result<(), String>> {
let image = self.image.as_mut().filter(|i| i.id == ref_id)?;
Some(match result {
Ok(media) => {
image.status = RefStatus::Ready;
image.token = Some(media.token);
image.full_url = media.full_url;
Ok(())
},
Err(err) => {
self.image = None;
Err(err)
},
})
}
pub fn show(&mut self, ctx: &egui::Context, characters: &[Character], pending: &[String], cache: &mut MediaCache) -> Vec<CharactersAction> {
let mut actions = Vec::new();
let width = (ctx.content_rect().width() - 80.0).min(760.0);
let close = widgets::modal(ctx, Id::new("characters"), width, |ui| match self.view.clone() {
View::List => self.list(ui, characters, pending, cache, &mut actions),
View::Edit(token) => match characters.iter().find(|c| c.token == token) {
Some(c) => self.edit(ui, c, cache, &mut actions),
None => self.view = View::List,
},
View::Create => self.create(ui, cache, &mut actions),
});
if close && self.confirm_delete.is_none() {
actions.push(CharactersAction::Close);
}
if let Some((token, name)) = self.confirm_delete.clone() {
let mut choice = None;
let dismissed = widgets::modal(ctx, Id::new("character-delete"), 420.0, |ui| {
ui.label(widgets::heading("Delete character?", 20.0));
ui.add_space(8.0);
ui.label(egui::RichText::new(format!("\u{201c}{name}\u{201d} will be removed from your characters.")).color(theme::MUTED));
ui.add_space(16.0);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if widgets::button(ui, None, "Delete", widgets::Kind::Danger, 0.0, theme::CONTROL_H).clicked() {
choice = Some(true);
}
if widgets::button(ui, None, "Cancel", widgets::Kind::Secondary, 0.0, theme::CONTROL_H).clicked() {
choice = Some(false);
}
});
});
if dismissed || choice.is_some() {
self.confirm_delete = None;
}
if choice == Some(true) {
actions.push(CharactersAction::Delete { token, name });
}
}
actions
}
fn header(&mut self, ui: &mut Ui, title: &str, back: bool, actions: &mut Vec<CharactersAction>) {
ui.horizontal(|ui| {
if back && widgets::ghost(ui, Icon::ChevronLeft, None, "Back", false).clicked() {
self.view = View::List;
}
ui.label(widgets::heading(title, 22.0));
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if widgets::ghost(ui, Icon::X, None, "Close", false).clicked() {
actions.push(CharactersAction::Close);
}
});
});
ui.add_space(12.0);
}
fn list(&mut self, ui: &mut Ui, characters: &[Character], pending: &[String], cache: &mut MediaCache, actions: &mut Vec<CharactersAction>) {
self.header(ui, "Characters", false, actions);
let tile = (ui.available_width() - GAP * (COLUMNS - 1) as f32) / COLUMNS as f32;
let max_h = ui.ctx().content_rect().height() * 0.6;
egui::ScrollArea::vertical().max_height(max_h).auto_shrink([false, true]).show(ui, |ui| {
let total = 1 + pending.len() + characters.len();
let rows = total.div_ceil(COLUMNS);
let origin = ui.cursor().min;
ui.allocate_exact_size(vec2(ui.available_width(), rows as f32 * (tile + 30.0 + GAP)), Sense::hover());
let cell = |i: usize| Rect::from_min_size(origin + vec2((i % COLUMNS) as f32 * (tile + GAP), (i / COLUMNS) as f32 * (tile + 30.0 + GAP)), vec2(tile, tile));
// "Create New" first.
let create = cell(0);
let resp = ui.interact(create, Id::new("character-create"), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, "Create New"));
ui.painter().rect_filled(create, theme::RADIUS, theme::fade(theme::CONTROLS, if resp.hovered() { 0.7 } else { 0.4 }));
icons::dashed_rect(ui.painter(), create.shrink(1.0), Stroke::new(1.5, theme::LINE_DASHED), 5.0, 4.0);
icons::paint(ui.painter(), Rect::from_center_size(create.center() - vec2(0.0, 10.0), vec2(22.0, 22.0)), Icon::Plus, theme::INK);
ui.painter().text(create.center() + vec2(0.0, 16.0), Align2::CENTER_CENTER, "Create New", FontId::new(13.0, theme::medium()), theme::INK);
if resp.clicked() {
self.name.clear();
self.description.clear();
self.image = None;
self.view = View::Create;
}
for (i, name) in pending.iter().enumerate() {
let r = cell(1 + i);
ui.painter().rect_filled(r, theme::RADIUS, theme::WASH);
icons::paint_spinner(ui, Rect::from_center_size(r.center(), vec2(24.0, 24.0)), theme::MUTED);
caption(ui, r, name, "Creating...");
}
for (i, c) in characters.iter().enumerate() {
let r = cell(1 + pending.len() + i);
let resp = ui.interact(r, Id::new(("character", &c.token)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, &c.name));
thumbnail(ui, r, c.avatar_url().as_deref(), cache);
if resp.hovered() {
ui.painter().rect_stroke(r, theme::RADIUS, Stroke::new(2.0, theme::fade(theme::ACCENT_400, 0.7)), egui::StrokeKind::Inside);
}
caption(ui, r, &c.name, "");
let mut clicked_tool = false;
if ui.rect_contains_pointer(r) {
let tools = Rect::from_min_size(pos2(r.right() - 64.0, r.top() + 6.0), vec2(58.0, 28.0));
ui.painter().rect_filled(tools, theme::RADIUS, Color32::from_black_alpha(160));
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(tools.shrink(1.0)).layout(egui::Layout::left_to_right(egui::Align::Center)));
child.spacing_mut().item_spacing.x = 0.0;
if widgets::overlay_icon(&mut child, Icon::Pencil, "Edit character").clicked() {
self.name = c.name.clone();
self.description = c.maybe_description.clone().unwrap_or_default();
self.view = View::Edit(c.token.clone());
clicked_tool = true;
}
if widgets::overlay_icon(&mut child, Icon::Trash, "Delete character").clicked() {
self.confirm_delete = Some((c.token.clone(), c.name.clone()));
clicked_tool = true;
}
}
if resp.clicked() && !clicked_tool {
actions.push(CharactersAction::Select { token: c.token.clone(), name: c.name.clone() });
}
}
});
}
fn edit(&mut self, ui: &mut Ui, c: &Character, cache: &mut MediaCache, actions: &mut Vec<CharactersAction>) {
self.header(ui, "Edit Character", true, actions);
ui.horizontal_top(|ui| {
ui.vertical(|ui| {
let (r, _) = ui.allocate_exact_size(vec2(240.0, 240.0), Sense::hover());
thumbnail(ui, r, c.full_url().or_else(|| c.avatar_url()).as_deref(), cache);
if let Some(url) = c.full_url() {
if ui.add(egui::Button::new(egui::RichText::new("View full size").size(12.0).color(theme::ACCENT_INK)).frame(false)).clicked() {
actions.push(CharactersAction::Preview(url));
}
}
});
ui.add_space(16.0);
ui.vertical(|ui| {
fields(ui, &mut self.name, &mut self.description);
ui.add_space(16.0);
ui.horizontal(|ui| {
if widgets::button(ui, None, "Save", widgets::Kind::Primary, 0.0, theme::CONTROL_H).clicked() {
if self.name.trim().is_empty() {
actions.push(CharactersAction::Toast("Name cannot be empty".to_owned()));
} else {
actions.push(CharactersAction::Edit { token: c.token.clone(), name: self.name.trim().to_owned(), description: self.description.trim().to_owned() });
self.view = View::List;
}
}
if widgets::button(ui, None, "Cancel", widgets::Kind::Secondary, 0.0, theme::CONTROL_H).clicked() {
self.view = View::List;
}
});
});
});
}
fn create(&mut self, ui: &mut Ui, cache: &mut MediaCache, actions: &mut Vec<CharactersAction>) {
self.header(ui, "New Character", true, actions);
ui.horizontal_top(|ui| {
ui.vertical(|ui| {
widgets::hud(ui, "Reference", theme::FAINT);
ui.add_space(6.0);
let (r, resp) = ui.allocate_exact_size(vec2(240.0, 240.0), Sense::click());
match &self.image {
Some(img) => {
thumbnail(ui, r, img.preview.as_deref(), cache);
if img.status == RefStatus::Uploading {
ui.painter().rect_filled(r, theme::RADIUS, Color32::from_black_alpha(110));
icons::paint_spinner(ui, Rect::from_center_size(r.center(), vec2(28.0, 28.0)), Color32::WHITE);
}
},
None => {
ui.painter().rect_filled(r, theme::RADIUS, theme::fade(theme::CONTROLS, if resp.hovered() { 0.7 } else { 0.4 }));
icons::dashed_rect(ui.painter(), r.shrink(1.0), Stroke::new(1.5, theme::LINE_DASHED), 5.0, 4.0);
icons::paint(ui.painter(), Rect::from_center_size(r.center() - vec2(0.0, 12.0), vec2(24.0, 24.0)), Icon::Upload, theme::INK);
ui.painter().text(r.center() + vec2(0.0, 18.0), Align2::CENTER_CENTER, "Upload reference image", FontId::new(13.0, theme::medium()), theme::INK);
},
}
if resp.on_hover_cursor(egui::CursorIcon::PointingHand).clicked() {
if let Some(path) = crate::pages::common::pick_files(&[RefKind::Image], false).into_iter().next() {
match std::fs::read(&path) {
Ok(bytes) => {
let mut img = RefMedia::uploading(RefKind::Image, None);
let key = format!("local://{}", img.id);
cache.insert_bytes(ui.ctx(), &key, Arc::from(bytes));
img.preview = Some(key);
actions.push(CharactersAction::Upload { ref_id: img.id, path });
self.image = Some(img);
},
Err(err) => actions.push(CharactersAction::Toast(format!("Failed to upload {}: {err}", crate::pages::common::file_name(&path)))),
}
}
}
});
ui.add_space(16.0);
ui.vertical(|ui| {
fields(ui, &mut self.name, &mut self.description);
ui.add_space(16.0);
let uploading = self.image.as_ref().is_some_and(|i| i.status == RefStatus::Uploading);
ui.horizontal(|ui| {
if ui.add_enabled_ui(!uploading, |ui| widgets::button(ui, Some(Icon::Sparkles), "Create", widgets::Kind::Primary, 0.0, theme::CONTROL_H)).inner.clicked() {
match (self.name.trim(), self.image.as_ref().and_then(|i| i.token.clone())) {
("", _) => actions.push(CharactersAction::Toast("Please enter a character name".to_owned())),
(_, None) => actions.push(CharactersAction::Toast("Please upload a reference image".to_owned())),
(name, Some(image_token)) => {
let description = Some(self.description.trim().to_owned()).filter(|d| !d.is_empty());
actions.push(CharactersAction::Create { image_token, name: name.to_owned(), description });
self.view = View::List;
},
}
}
if widgets::button(ui, None, "Cancel", widgets::Kind::Secondary, 0.0, theme::CONTROL_H).clicked() {
self.view = View::List;
}
});
});
});
}
}
/// The name and description inputs shared by create and edit.
fn fields(ui: &mut Ui, name: &mut String, description: &mut String) {
widgets::hud(ui, "Character name", theme::FAINT);
ui.add_space(4.0);
input_frame(ui, |ui| ui.add(egui::TextEdit::singleline(name).frame(egui::Frame::NONE).desired_width(f32::INFINITY).hint_text(egui::RichText::new("Character name").color(theme::FAINT))));
ui.add_space(10.0);
widgets::hud(ui, "Description", theme::FAINT);
ui.add_space(4.0);
input_frame(ui, |ui| ui.add(egui::TextEdit::multiline(description).frame(egui::Frame::NONE).desired_width(f32::INFINITY).desired_rows(4).hint_text(egui::RichText::new("Description...").color(theme::FAINT))));
}
fn input_frame(ui: &mut Ui, add: impl FnOnce(&mut Ui) -> egui::Response) {
egui::Frame::new().fill(theme::SUNKEN).stroke(theme::hairline()).corner_radius(theme::RADIUS).inner_margin(egui::Margin::symmetric(10, 8)).show(ui, |ui| add(ui));
}
fn thumbnail(ui: &Ui, rect: Rect, url: Option<&str>, cache: &mut MediaCache) {
ui.painter().rect_filled(rect, theme::RADIUS, theme::WASH);
match url.map(|u| cache.get(ui.ctx(), u)) {
Some(Lookup::Ready(t)) => {
egui::Image::new(&t).uv(cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, rect);
},
Some(Lookup::Loading) => icons::paint_spinner(ui, Rect::from_center_size(rect.center(), vec2(20.0, 20.0)), theme::FAINT),
_ => icons::paint(ui.painter(), Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), Icon::User, theme::FAINT),
}
}
/// The name under a tile (and a muted note after it).
fn caption(ui: &Ui, tile: Rect, name: &str, note: &str) {
let y = tile.bottom() + 15.0;
let galley = ui.painter().layout_no_wrap(name.to_owned(), FontId::new(13.0, theme::medium()), theme::INK);
let w = galley.size().x.min(tile.width());
ui.painter().with_clip_rect(Rect::from_min_size(pos2(tile.left(), y - 10.0), vec2(tile.width(), 20.0))).galley(pos2(tile.left(), y - galley.size().y / 2.0), galley, theme::INK);
if !note.is_empty() {
ui.painter().text(pos2(tile.left() + w + 6.0, y), Align2::LEFT_CENTER, note, FontId::new(11.5, egui::FontFamily::Proportional), theme::MUTED);
}
}
#[cfg(test)]
mod tests {
use super::*;
use egui_kittest::Harness;
use egui_kittest::kittest::Queryable;
struct State {
_rt: tokio::runtime::Runtime,
cache: MediaCache,
modal: CharactersModal,
characters: Vec<Character>,
actions: Vec<String>,
fonts_ready: bool,
}
fn harness() -> Harness<'static, State> {
let rt = tokio::runtime::Builder::new_current_thread().enable_all().build().expect("runtime");
let cache = MediaCache::new(rt.handle().clone(), reqwest::Client::new());
let characters = vec![Character { token: "c1".into(), name: "Mira".into(), ..Default::default() }];
let state = State { _rt: rt, cache, modal: CharactersModal::new(), characters, actions: Vec::new(), fonts_ready: false };
let mut harness = Harness::builder().with_size(egui::vec2(1000.0, 800.0)).build_ui_state(
|ui, s: &mut State| {
// The harness draws one frame while building: fonts are installed then, used from the next.
if !s.fonts_ready {
crate::theme::apply(ui.ctx());
s.fonts_ready = true;
return;
}
let actions = s.modal.show(ui.ctx(), &s.characters, &[], &mut s.cache);
s.actions.extend(actions.iter().map(|a| format!("{a:?}")));
},
state,
);
harness.run();
harness
}
#[test]
fn picking_a_character_mentions_it() {
let mut harness = harness();
harness.get_by_label("Mira").click();
harness.run();
assert!(harness.state().actions.iter().any(|a| a.contains("Select") && a.contains("Mira")), "{:?}", harness.state().actions);
}
#[test]
fn creating_needs_a_name_and_a_reference() {
let mut harness = harness();
harness.get_by_label("Create New").click();
harness.run();
harness.get_by_label("Create").click();
harness.run();
assert_eq!(harness.state().actions, [r#"Toast("Please enter a character name")"#]);
harness.get_by_role(egui::accesskit::Role::TextInput).focus();
harness.run();
harness.get_by_role(egui::accesskit::Role::TextInput).type_text("Mira");
harness.run();
harness.get_by_label("Create").click();
harness.run();
assert_eq!(harness.state().actions.last().map(String::as_str), Some(r#"Toast("Please upload a reference image")"#));
}
}
@@ -153,7 +153,7 @@ impl LibraryPicker {
}
fn picks(&self) -> Vec<RefMedia> {
self.selected.iter().filter_map(|t| self.items.iter().find(|i| i.token == *t)).map(|f| RefMedia::from_library(self.kind, f.token.clone(), f.media_links.thumbnail(256), Some(f.media_links.cdn_url.clone()), f.maybe_duration_millis.map_or(0.0, |ms| ms as f32 / 1000.0))).collect()
self.selected.iter().filter_map(|t| self.items.iter().find(|i| i.token == *t)).map(|f| RefMedia::from_library(self.kind, f.token.clone(), f.thumbnail(256), Some(f.media_links.cdn_url.clone()), f.maybe_duration_millis.map_or(0.0, |ms| ms as f32 / 1000.0))).collect()
}
}
@@ -162,7 +162,7 @@ fn tile_ui(ui: &mut egui::Ui, rect: Rect, item: &MediaFile, kind: RefKind, attac
let resp = ui.interact(rect, Id::new(("picker-tile", &item.token)), if attached { Sense::hover() } else { Sense::click() });
let p = ui.painter();
p.rect_filled(rect, 0.0, theme::fade(theme::CONTROLS, 0.6));
match item.media_links.thumbnail(256).map(|u| cache.get(ui.ctx(), &u)) {
match item.thumbnail(256).map(|u| cache.get(ui.ctx(), &u)) {
Some(Lookup::Ready(t)) => {
let image = egui::Image::new(&t).uv(cover_uv(t.size_vec2()));
let image = if attached { image.tint(Color32::from_gray(90)) } else { image };
+55 -8
View File
@@ -1,23 +1,30 @@
//! The lightbox (`lightbox-modal`): the media large on black, prev/next through the feed, and a
//! side column with the prompt, details and actions.
//! The lightbox (`lightbox-modal`): the media large on black (audio as a player panel), prev/next
//! through the feed, and a side column with the prompt, details and actions.
use egui::{Align2, Color32, FontId, Id, Key, Rect, Sense, Ui, pos2, vec2};
use crate::backend::audio::AudioPlayer;
use crate::backend::media_cache::{Lookup, MediaCache};
use crate::backend::video::{self, VideoPlayer};
use crate::backend::wire::Prompt;
use crate::feed::grid::ModelNames;
use crate::feed::types::{FeedItem, MediaKind};
use crate::theme;
use crate::ui::icons::{self, Icon};
use crate::ui::{creator_icons, widgets};
use crate::ui::{creator_icons, waveform, widgets};
const SIDE_WIDTH: f32 = 280.0;
/// The audio panel's widest (`max-w-xl`) and its padding.
const AUDIO_PANEL_MAX: f32 = 576.0;
const AUDIO_PANEL_PAD: f32 = 24.0;
pub struct Lightbox {
pub kind: MediaKind,
pub token: String,
confirm_delete: bool,
prompt_expanded: bool,
/// The in-app player for videos (when FFmpeg is installed).
player: Option<VideoPlayer>,
}
#[derive(Debug, PartialEq)]
@@ -35,10 +42,11 @@ pub enum LightboxAction {
impl Lightbox {
pub fn new(kind: MediaKind, token: String) -> Self {
Self { kind, token, confirm_delete: false, prompt_expanded: false }
Self { kind, token, confirm_delete: false, prompt_expanded: false, player: None }
}
pub fn show(&mut self, ctx: &egui::Context, item: &FeedItem, order: &[String], prompt: Option<&Prompt>, names: &dyn ModelNames, cache: &mut MediaCache) -> Vec<LightboxAction> {
#[allow(clippy::too_many_arguments)]
pub fn show(&mut self, ctx: &egui::Context, item: &FeedItem, order: &[String], prompt: Option<&Prompt>, names: &dyn ModelNames, cache: &mut MediaCache, audio: &mut AudioPlayer) -> Vec<LightboxAction> {
let mut actions = Vec::new();
let screen = ctx.content_rect();
let size = vec2((screen.width() - 80.0).min(1200.0), (screen.height() - 80.0).min(760.0));
@@ -62,7 +70,7 @@ impl Lightbox {
ui.horizontal_top(|ui| {
let media_rect = Rect::from_min_size(ui.cursor().min, vec2(ui.available_width() - SIDE_WIDTH - 16.0, size.y - 32.0));
ui.allocate_rect(media_rect, Sense::hover());
self.media(ui, media_rect, item, cache, &prev, &next, &mut actions);
self.media(ui, media_rect, item, cache, audio, &prev, &next, &mut actions);
ui.add_space(16.0);
ui.vertical(|ui| {
ui.set_width(SIDE_WIDTH);
@@ -80,7 +88,29 @@ impl Lightbox {
}
#[allow(clippy::too_many_arguments)]
fn media(&self, ui: &mut Ui, rect: Rect, item: &FeedItem, cache: &mut MediaCache, prev: &Option<String>, next: &Option<String>, actions: &mut Vec<LightboxAction>) {
fn media(&mut self, ui: &mut Ui, rect: Rect, item: &FeedItem, cache: &mut MediaCache, audio: &mut AudioPlayer, prev: &Option<String>, next: &Option<String>, actions: &mut Vec<LightboxAction>) {
if item.kind == MediaKind::Audio {
self.player = None;
ui.painter().rect_filled(rect, 0.0, Color32::BLACK);
audio_panel(ui, rect, item, audio);
self.nav_buttons(ui, rect, prev, next, actions);
return;
}
if item.kind == MediaKind::Video && video::ffmpeg_available() {
if self.player.as_ref().is_none_or(|p| p.url() != item.full_url) {
self.player = Some(VideoPlayer::new(&item.full_url));
}
let poster = match item.thumbnail.as_deref().map(|u| cache.get(ui.ctx(), u)) {
Some(Lookup::Ready(t)) => Some(t),
_ => None,
};
if let Some(player) = &mut self.player {
player.ui(ui, rect, poster.as_ref());
}
self.nav_buttons(ui, rect, prev, next, actions);
return;
}
self.player = None;
ui.painter().rect_filled(rect, 0.0, Color32::BLACK);
let url = if item.kind == MediaKind::Image { Some(item.full_url.as_str()) } else { item.thumbnail.as_deref() };
match url.map(|u| cache.get(ui.ctx(), u)) {
@@ -109,6 +139,11 @@ impl Lightbox {
actions.push(LightboxAction::Play(item.full_url.clone()));
}
}
self.nav_buttons(ui, rect, prev, next, actions);
}
/// Previous / next arrows at the media's sides (shown while hovering it).
fn nav_buttons(&self, ui: &mut Ui, rect: Rect, prev: &Option<String>, next: &Option<String>, actions: &mut Vec<LightboxAction>) {
let hovering = ui.rect_contains_pointer(rect);
for (target, left, tip) in [(prev, true, "Previous item"), (next, false, "Next item")] {
let Some(token) = target else {
@@ -214,7 +249,8 @@ impl Lightbox {
ui.add_space(16.0);
let w = (ui.available_width() - 8.0) / 2.0;
if item.prompt_token.is_some() && widgets::button(ui, Some(Icon::RotateCw), "Recreate", widgets::Kind::Primary, ui.available_width(), theme::CONTROL_H).clicked() {
// Audio can't be recreated yet (`Audio v1`).
if item.prompt_token.is_some() && item.kind != MediaKind::Audio && widgets::button(ui, Some(Icon::RotateCw), "Recreate", widgets::Kind::Primary, ui.available_width(), theme::CONTROL_H).clicked() {
actions.push(LightboxAction::Recreate);
}
ui.horizontal_wrapped(|ui| {
@@ -259,6 +295,17 @@ impl Lightbox {
}
}
/// Audio in the lightbox: a bordered panel with the music tile and the full-size player.
fn audio_panel(ui: &mut Ui, rect: Rect, item: &FeedItem, audio: &mut AudioPlayer) {
let width = (rect.width() - 48.0).min(AUDIO_PANEL_MAX);
let panel = Rect::from_center_size(rect.center(), vec2(width, AUDIO_PANEL_PAD * 2.0 + 64.0 + 20.0 + 44.0));
ui.painter().rect(panel, theme::RADIUS, theme::fade(theme::CONTROLS, 0.6), theme::hairline(), egui::StrokeKind::Inside);
waveform::music_tile(ui, Rect::from_center_size(pos2(panel.center().x, panel.top() + AUDIO_PANEL_PAD + 32.0), vec2(64.0, 64.0)), false);
let bar = Rect::from_min_max(pos2(panel.left() + AUDIO_PANEL_PAD, panel.bottom() - AUDIO_PANEL_PAD - 44.0), pos2(panel.right() - AUDIO_PANEL_PAD, panel.bottom() - AUDIO_PANEL_PAD));
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(bar));
waveform::player(&mut child, Id::new(("lightbox-player", &item.token)), audio, &item.full_url, item.duration_secs, false);
}
fn info_row(ui: &mut Ui, label: &str, value: impl FnOnce(&mut Ui)) {
ui.horizontal(|ui| {
let (rect, _) = ui.allocate_exact_size(vec2(96.0, 20.0), Sense::hover());
+2
View File
@@ -1,3 +1,5 @@
pub mod account;
pub mod characters;
pub mod library_picker;
pub mod lightbox;
pub mod providers;
+261
View File
@@ -0,0 +1,261 @@
//! Connecting provider accounts: the "Set up {provider}" dialog (`ProviderSetupModal`), the
//! Midjourney sign-in window (a system webview, used only to capture the login cookies, like
//! the Tauri app's), and the accounts rows in Settings.
use std::path::Path;
use cookie_store_wrapper::cookie_store::CookieStore;
use egui::{Id, Rect, Sense, Ui, vec2};
use log::warn;
use midjourney_client::credentials::cookie_store_has_auth_cookies::cookie_store_has_auth_cookies;
use midjourney_client::utils::midjourney_browser_profile::MIDJOURNEY_USER_AGENT;
use url::Url;
use wry::raw_window_handle::HasWindowHandle;
use wry::{WebContext, WebView, WebViewBuilder};
use crate::backend::providers::Provider;
use crate::overlays::account::text_field;
use crate::theme;
use crate::ui::icons::{self, Icon};
use crate::ui::widgets;
const MIDJOURNEY_URL: &str = "https://www.midjourney.com/";
/// The login cookies live on both hosts.
const MIDJOURNEY_ORIGINS: [&str; 2] = ["https://www.midjourney.com/", "https://midjourney.com/"];
/// How often the sign-in window is checked for the login cookies.
const COOKIE_CHECK_SECS: f64 = 2.0;
const LOGIN_WIDTH: f32 = 560.0;
/// What the setup dialog asks for.
#[derive(Debug, PartialEq)]
pub enum SetupAction {
Close,
ConnectMidjourney,
SaveFalKey,
}
/// "Set up {provider}": Midjourney signs in through its window; fal takes an API key here (the
/// Tauri app has no place for it yet).
pub fn setup_dialog(ctx: &egui::Context, provider: Provider, fal_key: &mut String) -> Option<SetupAction> {
let name = provider.label();
let mut action = None;
let close = widgets::modal(ctx, Id::new("provider-setup"), 460.0, |ui| {
ui.horizontal(|ui| {
widgets::hud(ui, "Connect account", theme::FAINT);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if widgets::ghost(ui, Icon::X, None, "Close", false).clicked() {
action = Some(SetupAction::Close);
}
});
});
ui.add_space(6.0);
ui.label(widgets::heading(&format!("Set up {name}"), 26.0));
ui.add_space(4.0);
ui.label(egui::RichText::new(format!("Add your {name} account to ArtCraft!")).size(14.0).family(theme::medium()));
ui.add_space(10.0);
let description = match provider {
Provider::Fal => format!("Paste your {name} API key to generate with your own {name} account. You can add all of your AI accounts to ArtCraft to use them all in one place and build the ultimate AI art tool."),
_ => format!("You can add your {name} account to ArtCraft by simply logging in. Use your credits and account directly within ArtCraft. You can add all of your AI accounts to ArtCraft to use them all in one place and build the ultimate AI art tool."),
};
ui.label(egui::RichText::new(description).size(13.0).color(theme::MUTED));
ui.add_space(16.0);
if provider == Provider::Fal {
text_field(ui, fal_key, true, "FAL API key");
ui.add_space(10.0);
}
let ready = provider != Provider::Fal || !fal_key.trim().is_empty();
if ui.add_enabled_ui(ready, |ui| widgets::button(ui, Some(Icon::ChevronRight), &format!("Set up {name}"), widgets::Kind::Primary, ui.available_width(), 40.0)).inner.clicked() {
action = Some(if provider == Provider::Fal { SetupAction::SaveFalKey } else { SetupAction::ConnectMidjourney });
}
});
if close && action.is_none() {
action = Some(SetupAction::Close);
}
action
}
/// What the accounts rows in Settings show.
pub struct AccountsView {
/// The Midjourney account's email ("" when unknown), when one is connected.
pub midjourney: Option<String>,
/// The saved fal key, masked.
pub fal_key_hint: Option<String>,
}
#[derive(Debug, PartialEq)]
pub enum AccountAction {
ConnectMidjourney,
DisconnectMidjourney,
SaveFalKey,
RemoveFalKey,
}
/// The provider accounts in Settings (`MidjourneyAccountBlock`, plus the fal key).
pub fn accounts_section(ui: &mut Ui, view: &AccountsView, fal_key: &mut String) -> Option<AccountAction> {
let mut action = None;
let detail = match view.midjourney.as_deref() {
Some("") => "Connected",
Some(email) => email,
None => "Not logged in",
};
let (label, kind) = if view.midjourney.is_some() { ("Disconnect", widgets::Kind::Danger) } else { ("Connect", widgets::Kind::Secondary) };
if account_row(ui, "Midjourney account", detail, |ui| widgets::button(ui, None, label, kind, 0.0, theme::CONTROL_H).clicked()) {
action = Some(if view.midjourney.is_some() { AccountAction::DisconnectMidjourney } else { AccountAction::ConnectMidjourney });
}
ui.add_space(10.0);
match &view.fal_key_hint {
Some(hint) => {
if account_row(ui, "FAL API key", hint, |ui| widgets::button(ui, None, "Remove", widgets::Kind::Danger, 0.0, theme::CONTROL_H).clicked()) {
action = Some(AccountAction::RemoveFalKey);
}
},
None => {
account_row(ui, "FAL API key", "Not set", |_| false);
ui.add_space(6.0);
ui.horizontal(|ui| {
let save_w = 90.0;
ui.allocate_ui(vec2(ui.available_width() - save_w - 8.0, theme::CONTROL_H), |ui| text_field(ui, fal_key, true, "Paste your FAL API key"));
let enabled = !fal_key.trim().is_empty();
if ui.add_enabled_ui(enabled, |ui| widgets::button(ui, None, "Save", widgets::Kind::Secondary, save_w, theme::CONTROL_H)).inner.clicked() {
action = Some(AccountAction::SaveFalKey);
}
});
},
}
action
}
/// A mono label over a detail line, with its button at the right. Returns the button's click.
fn account_row(ui: &mut Ui, label: &str, detail: &str, button: impl FnOnce(&mut Ui) -> bool) -> bool {
ui.horizontal(|ui| {
ui.vertical(|ui| {
ui.set_width(300.0);
widgets::hud(ui, label, theme::MUTED);
ui.label(egui::RichText::new(detail).size(13.0));
});
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), button).inner
})
.inner
}
/// The Midjourney sign-in window: a webview over a modal, checked every two seconds for the
/// login cookies (both auth tokens), which it hands back once present.
pub struct MidjourneyLogin {
context: WebContext,
webview: Option<WebView>,
error: Option<String>,
next_check: f64,
}
/// What came of the sign-in window.
pub enum LoginEvent {
Captured(CookieStore),
Closed,
}
impl MidjourneyLogin {
/// The window keeps its own browser profile under the data folder.
pub fn new(data_root: &Path) -> Self {
Self { context: WebContext::new(Some(data_root.join("webview").join("midjourney"))), webview: None, error: None, next_check: 0.0 }
}
pub fn show(&mut self, ctx: &egui::Context, window: &impl HasWindowHandle) -> Option<LoginEvent> {
let mut closed = false;
let mut slot = Rect::NOTHING;
let height = (ctx.content_rect().height() - 120.0).clamp(420.0, 800.0);
let dismissed = widgets::modal(ctx, Id::new("midjourney-login"), LOGIN_WIDTH, |ui| {
ui.horizontal(|ui| {
ui.label(widgets::heading("Login to Midjourney", 22.0));
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if widgets::ghost(ui, Icon::X, None, "Close", false).clicked() {
closed = true;
}
});
});
ui.add_space(10.0);
let (rect, _) = ui.allocate_exact_size(vec2(ui.available_width(), height), Sense::hover());
ui.painter().rect_filled(rect, 0.0, theme::SUNKEN);
match &self.error {
Some(err) => {
let mut note = ui.new_child(egui::UiBuilder::new().max_rect(rect.shrink(24.0)).layout(egui::Layout::top_down(egui::Align::Center)));
note.add_space(rect.height() / 2.0 - 48.0);
note.label(egui::RichText::new(format!("Couldn't open the sign-in window: {err}")).size(13.0).color(theme::BAD));
},
None => icons::paint_spinner(ui, Rect::from_center_size(rect.center(), vec2(28.0, 28.0)), theme::MUTED),
}
slot = rect;
});
if closed || dismissed {
self.close();
return Some(LoginEvent::Closed);
}
if self.error.is_some() {
return None;
}
let bounds = physical_bounds(slot, ctx.pixels_per_point());
match &self.webview {
Some(webview) => {
if let Err(err) = webview.set_bounds(bounds) {
warn!("Couldn't move the sign-in window: {err}");
}
},
None => match self.open(window, bounds) {
Ok(webview) => self.webview = Some(webview),
Err(err) => self.error = Some(err.to_string()),
},
}
let now = ctx.input(|i| i.time);
ctx.request_repaint_after(std::time::Duration::from_secs_f64(COOKIE_CHECK_SECS));
if now < self.next_check {
return None;
}
self.next_check = now + COOKIE_CHECK_SECS;
let cookies = self.webview.as_ref().and_then(login_cookies)?;
self.close();
Some(LoginEvent::Captured(cookies))
}
/// Opens on a blank page, clears what an earlier sign-in left behind, then goes to Midjourney
/// (with the user agent the saved cookies will be replayed with).
fn open(&mut self, window: &impl HasWindowHandle, bounds: wry::Rect) -> wry::Result<WebView> {
let webview = WebViewBuilder::new_with_web_context(&mut self.context).with_user_agent(MIDJOURNEY_USER_AGENT).with_url("about:blank").with_bounds(bounds).build_as_child(window)?;
webview.clear_all_browsing_data()?;
webview.load_url(MIDJOURNEY_URL)?;
webview.focus()?;
Ok(webview)
}
fn close(&mut self) {
if let Some(webview) = self.webview.take() {
let _ = webview.focus_parent();
}
}
}
/// The login cookies, once the page is on Midjourney and both auth tokens are set.
fn login_cookies(webview: &WebView) -> Option<CookieStore> {
let host = webview.url().ok().and_then(|u| Url::parse(&u).ok()).and_then(|u| u.host_str().map(str::to_owned))?;
log::debug!("The Midjourney sign-in window is on {host}");
if host != "www.midjourney.com" && host != "midjourney.com" {
return None;
}
let mut store = CookieStore::empty();
for origin in MIDJOURNEY_ORIGINS {
let Ok(origin) = Url::parse(origin) else {
continue;
};
match webview.cookies_for_url(origin.as_str()) {
// The cookie's own text keeps its path, domain, Secure, HttpOnly, SameSite and expiry.
Ok(cookies) => cookies.iter().for_each(|c| {
store.apply_set_cookie_header(&c.to_string(), &origin);
}),
Err(err) => warn!("Couldn't read the sign-in cookies: {err}"),
}
}
cookie_store_has_auth_cookies(&store).then_some(store)
}
fn physical_bounds(rect: Rect, pixels_per_point: f32) -> wry::Rect {
let r = rect * pixels_per_point;
wry::Rect { position: wry::dpi::PhysicalPosition::new(r.left().round() as i32, r.top().round() as i32).into(), size: wry::dpi::PhysicalSize::new(r.width().round().max(1.0) as u32, r.height().round().max(1.0) as u32).into() }
}
+67 -8
View File
@@ -7,6 +7,7 @@ use std::sync::Arc;
use egui::{Align2, Color32, Id, Mesh, Order, Rect, Ui, pos2, vec2};
use serde_json::{Map, Value};
use crate::backend::audio::AudioPlayer;
use crate::backend::media_cache::MediaCache;
use crate::backend::{Backend, Modality, UploadedMedia};
use crate::feed::grid::{self, FeedAction, FeedView, RatioCache, ViewMode};
@@ -29,28 +30,55 @@ pub struct Env<'a> {
pub ctx: egui::Context,
pub backend: &'a Backend,
pub cache: &'a mut MediaCache,
/// Plays reference clips and feed tracks.
pub audio: &'a mut AudioPlayer,
pub toasts: &'a mut Toasts,
pub catalog: &'a Catalog,
pub signed_in: bool,
pub enter_to_generate: bool,
pub view_mode: ViewMode,
pub autoplay: bool,
pub ratios: &'a mut RatioCache,
pub prompts: &'a std::collections::HashMap<String, String>,
pub requests: &'a mut Vec<AppRequest>,
/// The user's characters, newest first.
pub characters: &'a [crate::backend::wire::Character],
}
/// Things only the app can do.
#[derive(Debug)]
pub enum AppRequest {
SignIn,
Open { kind: MediaKind, token: String },
PickFromLibrary { kind: RefKind, slot: ImageSlot, max: usize, page: MediaKind },
MakeVideo { token: String },
Recreate { kind: MediaKind, token: String },
Open {
kind: MediaKind,
token: String,
},
PickFromLibrary {
kind: RefKind,
slot: ImageSlot,
max: usize,
page: MediaKind,
},
MakeVideo {
token: String,
},
Recreate {
kind: MediaKind,
token: String,
},
Share(String),
Download(Vec<String>),
CopyText(String),
Preview(String),
/// Play or stop an audio reference.
ToggleAudio {
ref_id: u64,
url: String,
},
/// Open a file in the system's player.
PlayExternally(String),
/// Open the characters dialog (it mentions the picked one in the video prompt).
OpenCharacters,
LoadMoreLibrary(FeedKey),
}
@@ -59,6 +87,7 @@ pub enum AppRequest {
pub enum FeedKey {
Image,
Video,
Audio,
Library,
}
@@ -67,6 +96,7 @@ impl FeedKey {
match kind {
MediaKind::Image => FeedKey::Image,
MediaKind::Video => FeedKey::Video,
MediaKind::Audio => FeedKey::Audio,
}
}
@@ -75,9 +105,33 @@ impl FeedKey {
match self {
FeedKey::Image => "image",
FeedKey::Video => "video",
FeedKey::Library => "image,video",
FeedKey::Audio => "audio",
FeedKey::Library => "image,video,audio",
}
}
/// What the feed's items default to (the library mixes kinds; each file says its own).
pub fn kind(self) -> MediaKind {
match self {
FeedKey::Image | FeedKey::Library => MediaKind::Image,
FeedKey::Video => MediaKind::Video,
FeedKey::Audio => MediaKind::Audio,
}
}
/// The backend's name for this feed's library requests.
pub fn name(self) -> &'static str {
match self {
FeedKey::Image => "image",
FeedKey::Video => "video",
FeedKey::Audio => "audio",
FeedKey::Library => "library",
}
}
pub fn from_name(name: &str) -> Option<Self> {
[FeedKey::Image, FeedKey::Video, FeedKey::Audio, FeedKey::Library].into_iter().find(|k| k.name() == name)
}
}
/// The page around the feed: the splash empty state (or the feed), a fade behind the prompt box,
@@ -87,9 +141,9 @@ pub fn create_shell(ui: &mut Ui, env: &mut Env<'_>, kind: MediaKind, title: &str
let area = ui.max_rect();
let bottom_offset = *box_height + 40.0;
if feed.has_content() {
let view = FeedView { id: Id::new(("feed", kind.label())), mode: env.view_mode, pending: &feed.pending, failed: &feed.failed, items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: kind == MediaKind::Image, bottom_padding: bottom_offset + 24.0 };
let view = FeedView { id: Id::new(("feed", kind.label())), mode: env.view_mode, pending: &feed.pending, failed: &feed.failed, items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: kind == MediaKind::Image, autoplay: env.autoplay, bottom_padding: bottom_offset + 24.0 };
let mut feed_ui = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0))));
let actions = grid::show(&mut feed_ui, &view, env.cache, env.ratios, env.catalog);
let actions = grid::show(&mut feed_ui, &view, env.cache, env.ratios, env.audio, env.catalog);
handle_feed_actions(env, kind, feed, actions);
bottom_fade(ui, area);
} else {
@@ -313,6 +367,11 @@ pub struct CostState {
impl CostState {
/// Requests an estimate for `fields` once they've been stable for a moment.
pub fn update(&mut self, ui: &Ui, backend: &Backend, modality: Modality, fields: &Map<String, Value>) {
self.update_with(ui, fields, |key, fields| backend.estimate_cost(modality, key, fields));
}
/// [`update`](Self::update), asking `request(key, fields)` for the estimate.
pub fn update_with(&mut self, ui: &Ui, fields: &Map<String, Value>, request: impl FnOnce(String, Map<String, Value>)) {
let key = Value::Object(fields.clone()).to_string();
if key == self.key {
return;
@@ -324,7 +383,7 @@ impl CostState {
self.key = key.clone();
self.credits = None;
self.pending_key = None;
backend.estimate_cost(modality, key, fields.clone());
request(key, fields.clone());
} else {
ui.ctx().request_repaint_after(std::time::Duration::from_secs_f64(COST_DEBOUNCE_SECS));
}
+883
View File
@@ -0,0 +1,883 @@
//! Create Audio (`CreateAudio` + `PromptBoxAudio`): a song, sound or sample from a prompt, with
//! the model's options (style, instrumental, lyrics, loop, beat & key, tuning), audio or image
//! references, the cost, and the audio feed.
use std::ops::RangeInclusive;
use std::path::PathBuf;
use egui::{Align2, Color32, FontId, Id, Rect, Response, Sense, Stroke, TextureHandle, Ui, pos2, vec2};
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value, json};
use crate::backend::media_cache::Lookup;
use crate::backend::providers::Provider;
use crate::backend::{EnqueueMeta, Modality};
use crate::feed::store::FeedStore;
use crate::feed::types::MediaKind;
use crate::models::{self, AudioCaps, Catalog, DEFAULT_AUDIO_MODEL, MUSICAL_KEYS, ModelInfo};
use crate::pages::common::{self, AppRequest, CostState, Env};
use crate::prompt_box::deck::cover_uv;
use crate::prompt_box::model_selector::model_selector;
use crate::prompt_box::pickers;
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, RefStatus, References};
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot};
use crate::theme;
use crate::ui::icons::{self, Icon};
use crate::ui::toast::Toasts;
use crate::ui::widgets::{self, Leading};
const PLACEHOLDER: &str = "Describe the music or sound you want...";
const STYLE_PLACEHOLDER: &str = "Style: e.g. dreamy synth-pop, female vocals";
/// The audio references' total length cap (`AUDIO_REF_MAX_DURATION_SECONDS`).
const MAX_REF_SECS: f32 = 600.0;
/// Beat & Key and Tuning (`SoundsSettingsPopover`, `AudioTuningPopover`).
const BPM_RANGE: RangeInclusive<u16> = 40..=240;
const DEFAULT_BPM: u16 = 120;
const SPEED_RANGE: RangeInclusive<f32> = 0.5..=2.0;
const VOLUME_RANGE: RangeInclusive<f32> = 0.5..=2.0;
const PITCH_RANGE: RangeInclusive<i8> = -12..=12;
const POPOVER_WIDTH: f32 = 240.0;
/// Shown when generate fails on the server's side (`OMNI_GENERATE_OUTAGE_MESSAGE`).
const OUTAGE_MESSAGE: &str = "Couldn't start generation - the service is temporarily unavailable. Try again shortly.";
/// What the page remembers between launches.
#[derive(Clone, Debug, Serialize, Deserialize)]
#[serde(default)]
pub struct AudioSettings {
pub prompt: String,
pub style_prompt: String,
pub model: Option<String>,
pub instrumental: bool,
pub keep_lyrics: bool,
pub loopable: bool,
/// `None` = Auto.
pub bpm: Option<u16>,
pub musical_key: String,
/// `None` = the model's default.
pub sample_rate_hz: Option<u32>,
pub speed: f32,
pub volume: f32,
/// Semitones.
pub pitch: i8,
}
impl Default for AudioSettings {
fn default() -> Self {
Self { prompt: String::new(), style_prompt: String::new(), model: None, instrumental: false, keep_lyrics: false, loopable: false, bpm: None, musical_key: "auto".to_owned(), sample_rate_hz: None, speed: 1.0, volume: 1.0, pitch: 0 }
}
}
#[derive(Default)]
pub struct CreateAudioPage {
pub settings: AudioSettings,
pub refs: References,
pub feed: FeedStore,
pub cost: CostState,
pub generating: bool,
box_state: PromptBoxState,
box_height: f32,
}
/// What the reference row asks for.
#[derive(Debug, PartialEq)]
enum RowAction {
AddAudio,
AudioFromLibrary,
AddImage,
Remove(u64),
Toggle(u64),
}
impl CreateAudioPage {
pub fn new(settings: AudioSettings) -> Self {
let mut page = Self { settings, box_height: 138.0, ..Default::default() };
page.box_state.editor.focus_requested = true;
page
}
pub fn ui(&mut self, ui: &mut Ui, env: &mut Env<'_>) {
let model = common::selected_model(&env.catalog.audio_page, self.settings.model.as_deref(), DEFAULT_AUDIO_MODEL).cloned();
if let Some(m) = &model {
let fields = self.fields(m, true);
self.cost.update(ui, env.backend, Modality::Audio, &fields);
}
let mut feed = std::mem::take(&mut self.feed);
let mut box_height = self.box_height;
common::create_shell(ui, env, MediaKind::Audio, "Create Audio", "Describe a song, a sound, or a sample", &mut feed, &mut box_height, |ui, env| self.prompt_box(ui, env, model.as_ref()));
self.feed = feed;
self.box_height = box_height;
}
fn prompt_box(&mut self, ui: &mut Ui, env: &mut Env<'_>, model: Option<&ModelInfo>) {
let missing_ref = model.is_some_and(|m| self.missing_required_ref(m));
let accepts: Vec<RefKind> = model.map_or_else(Vec::new, |m| [(RefKind::Audio, m.audio_refs_max), (RefKind::Image, m.image_refs_max)].into_iter().filter(|(_, n)| *n > 0).map(|(k, _)| k).collect());
let props = PromptBoxProps { id: Id::new("audio-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: None, deck: DeckMode::None, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !missing_ref, generating: self.generating, generate_tooltip: "Generate", warning: missing_ref.then_some("Audio track required"), banner: None, extra_input: !self.settings.style_prompt.is_empty() };
let playing = self.refs.audios.iter().map(|r| r.id).find(|id| env.audio.is_playing(&ref_key(*id)));
let image = self.refs.images.first().and_then(|r| r.preview.as_deref()).and_then(|key| match env.cache.get(&env.ctx, key) {
Lookup::Ready(t) => Some(t),
_ => None,
});
let models = &env.catalog.audio_page;
let mut prompt = std::mem::take(&mut self.settings.prompt);
let settings = &mut self.settings;
let refs = &self.refs;
let mut picked_model = None;
let mut row_actions = Vec::new();
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
(Slot::ToolbarLeft, _) => {
if let Some((id, _)) = model_selector(ui, Id::new("audio-model"), models, model, false, None) {
picked_model = Some(id);
}
if let Some(m) = model {
audio_toolbar(ui, &m.audio, settings);
}
},
(Slot::AbovePrompt, Some(m)) => reference_row(ui, m, refs, playing, image.as_ref(), &mut row_actions),
(Slot::BelowPrompt, Some(m)) if m.audio.style_prompt => style_row(ui, &mut settings.style_prompt),
_ => {},
};
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
self.settings.prompt = prompt;
if let Some(id) = picked_model {
self.settings.model = Some(id);
}
if let Some(m) = model {
for action in row_actions {
self.row_action(action, env, m);
}
}
for action in actions {
self.handle(action, env, model);
}
}
fn handle(&mut self, action: PromptBoxAction, env: &mut Env<'_>, model: Option<&ModelInfo>) {
match action {
PromptBoxAction::Generate => self.generate(env, model),
// Toggles and tuning stay as they are.
PromptBoxAction::ClearAll => {
self.settings.prompt.clear();
self.settings.style_prompt.clear();
self.refs.clear();
},
PromptBoxAction::DroppedFiles(paths) => {
if let Some(m) = model {
self.add_dropped(env, m, &paths);
}
},
PromptBoxAction::PastedImage(png) => match model {
Some(m) if m.image_refs_max > 0 => {
self.make_room_for_image(env);
common::upload_png(env, &mut self.refs, png, ImageSlot::Reference);
},
_ => env.toasts.error("Image references aren't available here"),
},
PromptBoxAction::Deck(_) | PromptBoxAction::MentionPicked(..) => {},
}
}
fn row_action(&mut self, action: RowAction, env: &mut Env<'_>, m: &ModelInfo) {
match action {
RowAction::AddAudio => {
let paths = common::pick_files(&[RefKind::Audio], m.audio_refs_max > 1);
self.add_audio(env, m, &paths);
},
RowAction::AudioFromLibrary => {
let max = m.audio_refs_max.saturating_sub(self.refs.audios.len()).max(1);
env.requests.push(AppRequest::PickFromLibrary { kind: RefKind::Audio, slot: ImageSlot::Reference, max, page: MediaKind::Audio });
},
RowAction::AddImage => {
if let Some(path) = common::pick_files(&[RefKind::Image], false).first() {
self.make_room_for_image(env);
common::upload_path(env, &mut self.refs, path, RefKind::Image, ImageSlot::Reference);
}
},
RowAction::Remove(id) => {
if env.audio.is_playing(&ref_key(id)) {
env.audio.stop();
}
self.refs.remove(id);
},
RowAction::Toggle(id) => {
if let Some(url) = self.refs.audios.iter().find(|r| r.id == id).and_then(|r| r.full_url.clone()) {
env.requests.push(AppRequest::ToggleAudio { ref_id: id, url });
}
},
}
}
/// Dropped files (`usePromptBoxDrop`): audio wins over images; only the first image counts.
fn add_dropped(&mut self, env: &mut Env<'_>, m: &ModelInfo, paths: &[PathBuf]) {
let audios: Vec<PathBuf> = paths.iter().filter(|p| RefKind::from_path(p) == Some(RefKind::Audio)).cloned().collect();
let image = paths.iter().find(|p| RefKind::from_path(p) == Some(RefKind::Image));
if !audios.is_empty() {
self.add_audio(env, m, &audios);
} else if let Some(path) = image {
if m.image_refs_max == 0 {
env.toasts.error("Image references aren't available here");
return;
}
self.make_room_for_image(env);
common::upload_path(env, &mut self.refs, path, RefKind::Image, ImageSlot::Reference);
} else if !paths.is_empty() {
env.toasts.error(format!("Only {} files can be added here", accepted_kinds(m)));
}
}
/// Uploads audio tracks into the free slots (dropping an image reference: they don't mix).
fn add_audio(&mut self, env: &mut Env<'_>, m: &ModelInfo, paths: &[PathBuf]) {
if paths.is_empty() {
return;
}
if m.audio_refs_max == 0 {
env.toasts.error("Audio references aren't available here");
return;
}
let free = m.audio_refs_max.saturating_sub(self.refs.audios.len());
if paths.len() > free {
env.toasts.error(format!("Max {} audio track{}", m.audio_refs_max, if m.audio_refs_max == 1 { "" } else { "s" }));
}
if free == 0 {
return;
}
self.make_room_for_audio(env.toasts);
for path in paths.iter().take(free) {
common::upload_path(env, &mut self.refs, path, RefKind::Audio, ImageSlot::Reference);
}
}
/// Library picks, ready to attach: audio within the slots and the total-length cap.
pub fn attach_from_library(&mut self, toasts: &mut Toasts, catalog: &Catalog, kind: RefKind, picks: Vec<RefMedia>) {
let Some(m) = common::selected_model(&catalog.audio_page, self.settings.model.as_deref(), DEFAULT_AUDIO_MODEL) else {
return;
};
if kind != RefKind::Audio || picks.is_empty() {
common::attach_from_library(&mut self.refs, kind, ImageSlot::Reference, picks);
return;
}
self.make_room_for_audio(toasts);
let mut total = self.refs.total_secs(RefKind::Audio);
let mut accepted = Vec::new();
for pick in picks.into_iter().take(m.audio_refs_max.saturating_sub(self.refs.audios.len())) {
if total + pick.duration_secs > MAX_REF_SECS {
toasts.error(duration_message());
break;
}
total += pick.duration_secs;
accepted.push(pick);
}
common::attach_from_library(&mut self.refs, kind, ImageSlot::Reference, accepted);
}
/// After an upload lands: keep the audio references under the total-length cap.
pub fn check_durations(&mut self, toasts: &mut Toasts, ref_id: u64) {
if self.refs.audios.iter().any(|r| r.id == ref_id) && self.refs.total_secs(RefKind::Audio) > MAX_REF_SECS + 0.5 {
self.refs.remove(ref_id);
toasts.error(duration_message());
}
}
/// Audio and image references don't mix: adding audio drops the image.
fn make_room_for_audio(&mut self, toasts: &mut Toasts) {
if !self.refs.images.is_empty() {
self.refs.images.clear();
toasts.error("Removed image reference \u{2014} it can't be combined with audio");
}
}
/// One image at most, and never with audio.
fn make_room_for_image(&mut self, env: &mut Env<'_>) {
if !self.refs.audios.is_empty() {
if self.refs.audios.iter().any(|r| env.audio.is_playing(&ref_key(r.id))) {
env.audio.stop();
}
self.refs.audios.clear();
env.toasts.error("Removed audio reference \u{2014} it can't be combined with an image");
}
self.refs.images.clear();
}
fn missing_required_ref(&self, m: &ModelInfo) -> bool {
m.audio.requires_audio_ref && self.refs.audios.len() != 1
}
fn generate(&mut self, env: &mut Env<'_>, model: Option<&ModelInfo>) {
let Some(m) = model.filter(|_| !self.generating && !self.settings.prompt.trim().is_empty()) else {
return;
};
if self.missing_required_ref(m) {
env.toasts.error(format!("{} needs an audio track to work from \u{2014} add one first", m.name));
return;
}
if !env.signed_in {
env.requests.push(AppRequest::SignIn);
return;
}
if self.refs.any_uploading() {
env.toasts.error("Please wait for your references to finish uploading");
return;
}
self.generating = true;
let meta = EnqueueMeta { prompt: self.settings.prompt.trim().to_owned(), model_id: m.id.clone(), batch_count: 1, ref_image: None, provider: Provider::Artcraft };
env.backend.generate(Modality::Audio, self.fields(m, false), meta);
}
pub fn on_enqueued(&mut self, env: &mut Env<'_>, job_tokens: &[String], meta: &EnqueueMeta) {
self.generating = false;
self.feed.add_enqueued(MediaKind::Audio, job_tokens, meta, env.catalog);
env.toasts.success("Audio generation enqueued!");
}
pub fn on_enqueue_failed(&mut self, env: &mut Env<'_>, message: &str, status: Option<u16>) {
self.generating = false;
env.toasts.error(failure_message(message, status));
}
/// The OmniGen request (`buildAudioGenerationBody`): only what the model takes, nothing null.
/// The estimate counts references with placeholders and needs no prompt.
fn fields(&self, m: &ModelInfo, estimate: bool) -> Map<String, Value> {
let (s, caps) = (&self.settings, &m.audio);
let mut f = Map::new();
f.insert("model".into(), json!(m.id));
let tokens = |list: &[RefMedia]| -> Vec<String> {
if estimate {
vec!["placeholder".to_owned(); list.len()]
} else {
list.iter().filter_map(|r| r.token.clone()).collect()
}
};
for (key, max, list) in [("audio_media_tokens", m.audio_refs_max, &self.refs.audios), ("image_media_tokens", m.image_refs_max, &self.refs.images)] {
let list = tokens(list.as_slice());
if max > 0 && !list.is_empty() {
f.insert(key.into(), json!(list));
}
}
if let Some(hz) = caps.sample_rate(s.sample_rate_hz).filter(|_| !caps.sample_rates.is_empty()) {
f.insert("sample_rate_hz".into(), json!(hz));
}
if estimate {
return f;
}
let (prompt, style) = (s.prompt.trim(), s.style_prompt.trim());
if !prompt.is_empty() {
f.insert("prompt".into(), json!(prompt));
}
if caps.style_prompt && !style.is_empty() {
f.insert("style_prompt".into(), json!(style));
}
for (supported, key, value) in [(caps.keep_lyrics, "keep_lyrics", s.keep_lyrics), (caps.instrumental, "is_instrumental", s.instrumental), (caps.loopable, "is_loopable", s.loopable)] {
if supported {
f.insert(key.into(), json!(value));
}
}
if let Some(bpm) = s.bpm.filter(|_| caps.bpm) {
f.insert("bpm".into(), json!(bpm));
}
if caps.musical_key {
f.insert("musical_key".into(), json!(musical_key(&s.musical_key).0));
}
for (supported, key, value) in [(caps.speed, "speed", s.speed), (caps.volume, "volume", s.volume)] {
if supported {
f.insert(key.into(), json!(hundredths(value)));
}
}
if caps.pitch {
f.insert("pitch".into(), json!(s.pitch));
}
f
}
}
/// The audio pickers after the model (`PromptBoxAudio`'s toolbar): the toggles the model offers,
/// Beat & Key, and Tuning.
fn audio_toolbar(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
let toggles = [(caps.instrumental, &mut s.instrumental, "audio-instrumental", Icon::MicOff, "Instrumental"), (caps.keep_lyrics, &mut s.keep_lyrics, "audio-keep-lyrics", Icon::Mic, "Keep lyrics"), (caps.loopable, &mut s.loopable, "audio-loop", Icon::Repeat, "Loop")];
for (supported, value, id, icon, label) in toggles {
if supported && pickers::toggle(ui, Id::new(id), icon, label, *value, &format!("{label}: {}", if *value { "ON" } else { "OFF" })) {
*value = !*value;
}
}
if caps.bpm || caps.musical_key {
beat_and_key(ui, caps, s);
}
if caps.has_tuning() {
tuning(ui, caps, s);
}
}
/// The Beat & Key popover: a BPM slider with Auto, and the musical key grid.
fn beat_and_key(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
let mut parts = Vec::new();
if caps.bpm {
parts.push(s.bpm.map_or_else(|| "Auto BPM".to_owned(), |b| format!("{b} BPM")));
}
if caps.musical_key {
parts.push(musical_key(&s.musical_key).2.to_owned());
}
let label = if parts.is_empty() { "Beat".to_owned() } else { parts.join(" \u{b7} ") };
pickers::popover_pill(ui, Id::new("audio-beat"), Leading::Icon(Icon::Drum), &label, "Beat & Key", |ui| {
ui.set_width(POPOVER_WIDTH);
widgets::menu_header(ui, "Beat & Key");
padded(ui, |ui| {
if caps.bpm {
ui.horizontal(|ui| {
setting_name(ui, "BPM");
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
setting_value(ui, &s.bpm.map_or_else(|| "Auto".to_owned(), |b| b.to_string()));
if s.bpm.is_some() && ui.add(egui::Button::new(egui::RichText::new("Auto").size(11.5).color(theme::ACCENT_INK)).frame(false)).on_hover_cursor(egui::CursorIcon::PointingHand).clicked() {
s.bpm = None;
}
});
});
let mut bpm = s.bpm.unwrap_or(DEFAULT_BPM);
if full_slider(ui, &mut bpm, BPM_RANGE, 1.0).changed() {
s.bpm = Some(bpm);
}
}
if caps.musical_key {
ui.add_space(6.0);
setting_name(ui, "Musical key");
ui.spacing_mut().item_spacing = vec2(4.0, 4.0);
let width = ui.available_width();
let (auto, keys) = MUSICAL_KEYS.split_first().expect("keys");
if choice_chip(ui, auto.2, s.musical_key == auto.0, width, auto.1).clicked() {
s.musical_key = auto.0.to_owned();
}
let cell = (width - 12.0) / 4.0;
for row in keys.chunks(4) {
ui.horizontal(|ui| {
for (value, label, short) in row {
if choice_chip(ui, short, s.musical_key == *value, cell, label).clicked() {
s.musical_key = (*value).to_owned();
}
}
});
}
}
});
});
}
/// The Tuning popover: sample rate chips and the speed, volume and pitch sliders.
fn tuning(ui: &mut Ui, caps: &AudioCaps, s: &mut AudioSettings) {
pickers::popover_pill(ui, Id::new("audio-tuning"), Leading::Icon(Icon::SlidersHorizontal), "Tuning", "Tuning", |ui| {
ui.set_width(POPOVER_WIDTH);
widgets::menu_header(ui, "Tuning");
padded(ui, |ui| {
if !caps.sample_rates.is_empty() {
setting_name(ui, "Sample rate");
let current = caps.sample_rate(s.sample_rate_hz);
ui.horizontal_wrapped(|ui| {
ui.spacing_mut().item_spacing = vec2(4.0, 4.0);
for hz in &caps.sample_rates {
let label = models::sample_rate_label(*hz);
if choice_chip(ui, &label, current == Some(*hz), 72.0, &label).clicked() {
s.sample_rate_hz = Some(*hz);
}
}
});
ui.add_space(6.0);
}
if caps.speed {
labeled_slider(ui, "Speed", &format!("{:.2}\u{d7}", s.speed), &mut s.speed, SPEED_RANGE, 0.05);
}
if caps.volume {
labeled_slider(ui, "Volume", &format!("{:.2}\u{d7}", s.volume), &mut s.volume, VOLUME_RANGE, 0.05);
}
if caps.pitch {
labeled_slider(ui, "Pitch", &format!("{}{} st", if s.pitch > 0 { "+" } else { "" }, s.pitch), &mut s.pitch, PITCH_RANGE, 1.0);
}
});
});
}
/// The references above the prompt (`AudioReferenceRow`): the audio tracks (play, remove), add or
/// pick more while there's room, and the image when the model takes one.
fn reference_row(ui: &mut Ui, m: &ModelInfo, refs: &References, playing: Option<u64>, image: Option<&TextureHandle>, actions: &mut Vec<RowAction>) {
if m.audio_refs_max == 0 && m.image_refs_max == 0 {
return;
}
ui.horizontal_wrapped(|ui| {
ui.spacing_mut().item_spacing = vec2(8.0, 6.0);
if m.audio_refs_max > 0 {
icons::icon(ui, Icon::Music, 14.0, theme::MUTED);
ui.label(egui::RichText::new(format!("Audio Track ({}/{})", refs.audios.len(), m.audio_refs_max)).size(12.5).color(theme::MUTED));
if m.audio.requires_audio_ref && refs.audios.is_empty() {
ui.label(egui::RichText::new("required").size(12.5).color(theme::BAD));
}
for (i, r) in refs.audios.iter().enumerate() {
audio_tile(ui, i, r, playing == Some(r.id), actions);
}
if refs.audios.len() < m.audio_refs_max {
let uploading = refs.audios.iter().any(|r| r.status == RefStatus::Uploading);
if ui.add_enabled_ui(!uploading, |ui| widgets::pill(ui, Leading::Icon(Icon::Upload), if uploading { "Uploading\u{2026}" } else { "Add audio" }, false)).inner.clicked() {
actions.push(RowAction::AddAudio);
}
if widgets::pill(ui, Leading::Icon(Icon::Folder), "From library", false).clicked() {
actions.push(RowAction::AudioFromLibrary);
}
}
}
if m.image_refs_max > 0 {
ui.add_space(8.0);
ui.label(egui::RichText::new("Image").size(12.5).color(theme::MUTED));
match refs.images.first() {
Some(r) => image_tile(ui, r, image, actions),
None => {
if widgets::pill(ui, Leading::Icon(Icon::Image), "Add image", false).clicked() {
actions.push(RowAction::AddImage);
}
},
}
}
});
ui.add_space(8.0);
hairline(ui);
ui.add_space(8.0);
}
/// One audio reference (`AudioRefTile`): play / stop, "Audio 1 · 12s", remove.
fn audio_tile(ui: &mut Ui, index: usize, r: &RefMedia, playing: bool, actions: &mut Vec<RowAction>) {
let uploading = r.status == RefStatus::Uploading;
let text = if uploading { format!("Audio {}", index + 1) } else { format!("Audio {} \u{b7} {}s", index + 1, r.duration_secs.round() as i64) };
let galley = ui.painter().layout_no_wrap(text, FontId::new(12.5, theme::medium()), theme::INK);
let (rect, _) = ui.allocate_exact_size(vec2(6.0 + 24.0 + 8.0 + galley.size().x + 6.0 + 20.0 + 6.0, theme::CONTROL_H), Sense::hover());
ui.painter().rect(rect, theme::RADIUS, theme::CONTROLS, Stroke::new(1.0, theme::LINE), egui::StrokeKind::Inside);
let play = Rect::from_center_size(pos2(rect.left() + 18.0, rect.center().y), vec2(24.0, 24.0));
if uploading {
icons::paint_spinner(ui, play.shrink(4.0), theme::MUTED);
} else {
let resp = ui.interact(play, Id::new(("audio-ref-play", r.id)), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
let label = if playing { "Stop" } else { "Play" };
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
let fill = match (playing, resp.hovered()) {
(true, _) => theme::ACCENT,
(false, true) => theme::WASH_HOVER,
(false, false) => theme::WASH,
};
ui.painter().circle_filled(play.center(), 12.0, fill);
if playing {
ui.painter().rect_filled(Rect::from_center_size(play.center(), vec2(8.0, 8.0)), 1.0, Color32::WHITE);
} else {
icons::paint(ui.painter(), play.shrink(7.0).translate(vec2(1.0, 0.0)), Icon::Play, theme::INK);
}
if resp.clicked() {
actions.push(RowAction::Toggle(r.id));
}
}
ui.painter().galley(pos2(play.right() + 8.0, rect.center().y - galley.size().y / 2.0), galley, theme::INK);
let remove = Rect::from_center_size(pos2(rect.right() - 16.0, rect.center().y), vec2(20.0, 20.0));
if x_button(ui, remove, Id::new(("audio-ref-remove", r.id)), "Remove audio", true) {
actions.push(RowAction::Remove(r.id));
}
}
/// The image reference: a 36 px thumbnail with a remove × on hover.
fn image_tile(ui: &mut Ui, r: &RefMedia, texture: Option<&TextureHandle>, actions: &mut Vec<RowAction>) {
let (rect, _) = ui.allocate_exact_size(vec2(36.0, 36.0), Sense::hover());
ui.painter().rect(rect, theme::RADIUS, theme::CONTROLS, Stroke::new(1.0, theme::LINE), egui::StrokeKind::Inside);
if let Some(t) = texture {
egui::Image::new(t).uv(cover_uv(t.size_vec2())).corner_radius(theme::RADIUS).paint_at(ui, rect.shrink(1.0));
}
if r.status == RefStatus::Uploading {
icons::paint_spinner(ui, rect.shrink(9.0), Color32::WHITE);
}
let remove = Rect::from_min_size(pos2(rect.right() - 14.0, rect.top() - 4.0), vec2(18.0, 18.0));
let shown = ui.rect_contains_pointer(rect.union(remove));
if x_button(ui, remove, Id::new(("image-ref-remove", r.id)), "Remove image", shown) {
actions.push(RowAction::Remove(r.id));
}
}
/// The style line under the prompt (`StylePromptRow`).
fn style_row(ui: &mut Ui, style: &mut String) {
ui.add_space(6.0);
hairline(ui);
ui.add_space(6.0);
ui.horizontal(|ui| {
icons::icon(ui, Icon::Tags, 14.0, theme::DIM);
let clear_w = if style.is_empty() { 0.0 } else { 28.0 };
ui.add(egui::TextEdit::singleline(style).id_salt("audio-style").hint_text(STYLE_PLACEHOLDER).frame(egui::Frame::NONE).font(FontId::new(14.0, egui::FontFamily::Proportional)).desired_width(ui.available_width() - clear_w));
if !style.is_empty() {
let (rect, _) = ui.allocate_exact_size(vec2(20.0, 20.0), Sense::hover());
if x_button(ui, rect, Id::new("audio-style-clear"), "Clear style", true) {
style.clear();
}
}
});
}
/// A small round × at `rect`, painted while `shown` (it stays clickable for keyboards and tests).
fn x_button(ui: &mut Ui, rect: Rect, id: Id, label: &str, shown: bool) -> bool {
let resp = ui.interact(rect, id, Sense::click());
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
if shown || resp.hovered() {
let fill = if resp.hovered() { theme::fade(theme::DANGER, 0.7) } else { Color32::from_black_alpha(128) };
ui.painter().circle_filled(rect.center(), rect.width() / 2.0, fill);
icons::paint(ui.painter(), rect.shrink(rect.width() * 0.27), Icon::X, Color32::WHITE);
}
resp.on_hover_text(label).on_hover_cursor(egui::CursorIcon::PointingHand).clicked()
}
/// A selectable chip in the popovers (keys, sample rates): outlined, filled when selected.
fn choice_chip(ui: &mut Ui, label: &str, selected: bool, width: f32, tooltip: &str) -> Response {
let (rect, resp) = ui.allocate_exact_size(vec2(width, 28.0), Sense::click());
let (fill, border) = match (selected, resp.hovered()) {
(true, _) => (theme::WASH_HOVER, theme::fade(Color32::WHITE, 0.8)),
(false, true) => (theme::WASH, theme::LINE_STRONG),
(false, false) => (Color32::TRANSPARENT, theme::LINE),
};
ui.painter().rect(rect, theme::RADIUS, fill, Stroke::new(1.0, border), egui::StrokeKind::Inside);
ui.painter().text(rect.center(), Align2::CENTER_CENTER, label, FontId::new(12.5, theme::medium()), if selected { theme::INK } else { theme::MUTED });
resp.widget_info(|| egui::WidgetInfo::selected(egui::WidgetType::Button, true, selected, label));
resp.on_hover_text(tooltip).on_hover_cursor(egui::CursorIcon::PointingHand)
}
/// A setting's name, its value at the right, and a slider across the popover.
fn labeled_slider<N: egui::emath::Numeric>(ui: &mut Ui, name: &str, value: &str, number: &mut N, range: RangeInclusive<N>, step: f64) {
ui.horizontal(|ui| {
setting_name(ui, name);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| setting_value(ui, value));
});
full_slider(ui, number, range, step);
ui.add_space(4.0);
}
fn full_slider<N: egui::emath::Numeric>(ui: &mut Ui, number: &mut N, range: RangeInclusive<N>, step: f64) -> Response {
ui.style_mut().spacing.slider_width = ui.available_width();
ui.add(egui::Slider::new(number, range).step_by(step).show_value(false))
}
fn setting_name(ui: &mut Ui, name: &str) {
ui.label(egui::RichText::new(name).size(12.0).color(theme::MUTED));
}
fn setting_value(ui: &mut Ui, value: &str) {
ui.label(egui::RichText::new(value).size(12.0).family(theme::semibold()).color(theme::INK));
}
fn padded(ui: &mut Ui, content: impl FnOnce(&mut Ui)) {
egui::Frame::new().inner_margin(egui::Margin::symmetric(8, 6)).show(ui, content);
}
fn hairline(ui: &mut Ui) {
let (line, _) = ui.allocate_exact_size(vec2(ui.available_width(), 1.0), Sense::hover());
ui.painter().rect_filled(line, 0.0, theme::fade(Color32::WHITE, 0.1));
}
/// The key a stored value names (Auto for anything unknown): value, label, short label.
fn musical_key(value: &str) -> &'static (&'static str, &'static str, &'static str) {
MUSICAL_KEYS.iter().find(|k| k.0 == value).unwrap_or(&MUSICAL_KEYS[0])
}
/// The audio player's key for a reference clip.
fn ref_key(id: u64) -> String {
format!("ref:{id}")
}
/// A slider's value to two decimals (0.05 steps drift in `f32`).
fn hundredths(value: f32) -> f64 {
(f64::from(value) * 100.0).round() / 100.0
}
/// "audio", "audio or image" (for the drop toasts).
fn accepted_kinds(m: &ModelInfo) -> String {
let kinds: Vec<&str> = [("audio", m.audio_refs_max), ("image", m.image_refs_max)].into_iter().filter(|(_, n)| *n > 0).map(|(k, _)| k).collect();
if kinds.is_empty() {
"reference".to_owned()
} else {
kinds.join(" or ")
}
}
fn duration_message() -> String {
format!("Total audio duration cannot exceed {}s", MAX_REF_SECS as u32)
}
/// What a failed generate tells the user: out of credits, an outage, or the server's message.
fn failure_message(message: &str, status: Option<u16>) -> String {
match status {
Some(402) => "Not enough credits for this generation".to_owned(),
Some(code) if code >= 500 => OUTAGE_MESSAGE.to_owned(),
_ if message.is_empty() => "Failed to start audio generation".to_owned(),
_ => message.to_owned(),
}
}
#[cfg(test)]
mod tests {
use super::*;
fn model(json: Value) -> ModelInfo {
models::audio_model(&serde_json::from_value(json).expect("audio model"))
}
/// The server's audio models (`configs/omni_gen/audio_models.rs`).
fn suno_music() -> ModelInfo {
model(json!({ "model": "suno_music", "style_prompt_supported": true, "instrumental_toggle_supported": true }))
}
fn suno_sounds() -> ModelInfo {
model(json!({ "model": "suno_sounds", "loopable_toggle_supported": true, "bpm_supported": true, "musical_key_supported": true }))
}
fn suno_remix() -> ModelInfo {
model(json!({ "model": "suno_remix", "style_prompt_supported": true, "keep_lyrics_supported": true, "audio_references_supported": true, "audio_references_max": 1 }))
}
fn seed_audio() -> ModelInfo {
model(json!({ "model": "seed_audio_1p0", "audio_references_supported": true, "audio_references_max": 3, "image_references_supported": true, "image_references_max": 1, "sample_rate_hz_options": [8000, 16000, 24000, 32000, 44100, 48000], "sample_rate_hz_default": 24000, "speed_supported": true, "volume_supported": true, "pitch_supported": true }))
}
fn page(settings: AudioSettings) -> CreateAudioPage {
CreateAudioPage { settings, ..Default::default() }
}
fn audio_ref(token: &str, secs: f32) -> RefMedia {
RefMedia::from_library(RefKind::Audio, token.into(), None, Some(format!("https://cdn/{token}.mp3")), secs)
}
mod request_bodies {
use super::*;
#[test]
fn suno_music_sends_only_what_it_takes() {
let p = page(AudioSettings { prompt: " lofi beat ".into(), style_prompt: " chill ".into(), instrumental: true, keep_lyrics: true, bpm: Some(90), ..Default::default() });
assert_eq!(Value::Object(p.fields(&suno_music(), false)), json!({ "model": "suno_music", "prompt": "lofi beat", "style_prompt": "chill", "is_instrumental": true }));
}
#[test]
fn suno_sounds_sends_its_toggles_beat_and_key() {
let mut p = page(AudioSettings { prompt: "door creak".into(), ..Default::default() });
assert_eq!(Value::Object(p.fields(&suno_sounds(), false)), json!({ "model": "suno_sounds", "prompt": "door creak", "is_loopable": false, "musical_key": "auto" }), "Auto BPM is left to the model");
p.settings.bpm = Some(128);
p.settings.musical_key = "c_minor".into();
let f = p.fields(&suno_sounds(), false);
assert_eq!((f["bpm"].clone(), f["musical_key"].clone()), (json!(128), json!("c_minor")));
}
#[test]
fn seed_audio_sends_references_and_tuning() {
let mut p = page(AudioSettings { prompt: "rain".into(), speed: 1.05, volume: 0.5, pitch: -3, sample_rate_hz: Some(11025), ..Default::default() });
p.refs.audios = vec![audio_ref("mf_a", 10.0), audio_ref("mf_b", 20.0)];
assert_eq!(Value::Object(p.fields(&seed_audio(), false)), json!({ "model": "seed_audio_1p0", "prompt": "rain", "audio_media_tokens": ["mf_a", "mf_b"], "sample_rate_hz": 24000, "speed": 1.05, "volume": 0.5, "pitch": -3 }));
}
#[test]
fn estimates_count_references_without_the_prompt() {
let mut p = page(AudioSettings { prompt: "rain".into(), sample_rate_hz: Some(44100), ..Default::default() });
p.refs.audios = vec![RefMedia::uploading(RefKind::Audio, None), audio_ref("mf_b", 5.0)];
assert_eq!(Value::Object(p.fields(&seed_audio(), true)), json!({ "model": "seed_audio_1p0", "audio_media_tokens": ["placeholder", "placeholder"], "sample_rate_hz": 44100 }));
assert_eq!(Value::Object(p.fields(&suno_music(), true)), json!({ "model": "suno_music" }), "references the model can't take don't count");
}
}
mod rules {
use super::*;
#[test]
fn remixes_need_exactly_one_track() {
let mut p = page(AudioSettings::default());
assert!(p.missing_required_ref(&suno_remix()));
assert!(!p.missing_required_ref(&suno_music()));
p.refs.audios.push(audio_ref("mf_a", 30.0));
assert!(!p.missing_required_ref(&suno_remix()));
p.refs.audios.push(audio_ref("mf_b", 30.0));
assert!(p.missing_required_ref(&suno_remix()), "two tracks are too many");
}
#[test]
fn library_picks_respect_slots_the_cap_and_drop_the_image() {
let mut catalog = Catalog::default();
catalog.set_audio(vec![seed_audio()]);
let mut p = page(AudioSettings { model: Some("seed_audio_1p0".into()), ..Default::default() });
p.refs.images.push(RefMedia::from_library(RefKind::Image, "mf_img".into(), None, None, 0.0));
let mut toasts = Toasts::default();
p.attach_from_library(&mut toasts, &catalog, RefKind::Audio, vec![audio_ref("mf_a", 400.0), audio_ref("mf_b", 300.0), audio_ref("mf_c", 10.0)]);
let tokens: Vec<_> = p.refs.audios.iter().filter_map(|r| r.token.as_deref()).collect();
assert_eq!(tokens, ["mf_a"], "the second would pass 600s, so picking stops there");
assert!(p.refs.images.is_empty(), "audio and an image don't mix");
}
#[test]
fn uploads_past_the_cap_are_dropped() {
let mut p = page(AudioSettings::default());
p.refs.audios = vec![audio_ref("mf_a", 590.0), audio_ref("mf_b", 30.0)];
let late = p.refs.audios[1].id;
p.check_durations(&mut Toasts::default(), late);
assert_eq!(p.refs.audios.len(), 1);
}
#[test]
fn failures_explain_credits_and_outages() {
assert_eq!(failure_message("Payment required", Some(402)), "Not enough credits for this generation");
assert_eq!(failure_message("boom", Some(503)), OUTAGE_MESSAGE);
assert_eq!(failure_message("Prompt too long", Some(400)), "Prompt too long");
assert_eq!(failure_message("", None), "Failed to start audio generation");
}
}
/// The toolbar headlessly (egui_kittest): toggles, Beat & Key and Tuning.
mod toolbar {
use egui_kittest::Harness;
use egui_kittest::kittest::{NodeT, Queryable};
use super::*;
struct State {
caps: AudioCaps,
settings: AudioSettings,
fonts_ready: bool,
}
#[test]
fn toggles_show_per_model_and_flip() {
let mut harness = build(suno_music().audio);
assert!(harness.query_by_label("Loop").is_none(), "Suno Music has no loop toggle");
harness.get_by_label("Instrumental").click();
harness.run();
assert!(harness.state().settings.instrumental);
}
#[test]
fn beat_and_key_sets_the_key_and_bpm() {
let mut harness = build(suno_sounds().audio);
harness.get_by_label("Auto BPM \u{b7} Auto").click();
harness.run();
harness.get_by_label("Cm").click();
harness.run();
assert_eq!(harness.state().settings.musical_key, "c_minor");
assert!(harness.query_by_label("Auto BPM \u{b7} Cm").is_some(), "the pill shows the key");
}
#[test]
fn tuning_picks_a_sample_rate() {
let mut harness = build(seed_audio().audio);
harness.get_by_label("Tuning").click();
harness.run();
// egui reports a selected button as toggled.
assert!(harness.query_by_label("24 kHz").is_some_and(|n| n.accesskit_node().toggled() == Some(egui::accesskit::Toggled::True)), "the default is selected");
harness.get_by_label("44.1 kHz").click();
harness.run();
assert_eq!(harness.state().settings.sample_rate_hz, Some(44100));
}
fn build(caps: AudioCaps) -> Harness<'static, State> {
let mut harness = Harness::builder().with_size(egui::vec2(1000.0, 600.0)).build_ui_state(
|ui, s: &mut State| {
if !s.fonts_ready {
crate::theme::apply(ui.ctx());
s.fonts_ready = true;
return;
}
ui.add_space(400.0);
ui.horizontal(|ui| audio_toolbar(ui, &s.caps, &mut s.settings));
},
State { caps, settings: AudioSettings::default(), fonts_ready: false },
);
harness.run();
harness
}
}
}
+84 -44
View File
@@ -1,10 +1,14 @@
//! Create Image (`TextToImage` + `PromptBoxImage`): prompt, reference images, model, aspect
//! ratio, resolution, quality and count; the cost; and the image feed.
//! Create Image (`TextToImage` + `PromptBoxImage`): prompt, reference images, model (and the
//! provider it runs on), aspect ratio, resolution, quality and count; the cost; and the image
//! feed.
use std::collections::HashMap;
use egui::{Id, Ui};
use serde::{Deserialize, Serialize};
use serde_json::{Map, Value, json};
use crate::backend::providers::Provider;
use crate::backend::wire::Prompt;
use crate::backend::{EnqueueMeta, Modality};
use crate::feed::store::FeedStore;
@@ -15,7 +19,7 @@ use crate::pages::toolbar;
use crate::prompt_box::deck::{DeckAction, DeckLimits};
use crate::prompt_box::model_selector::model_selector;
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, editor};
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot, editor};
const PLACEHOLDER: &str = "Describe what you want in the image...";
@@ -29,6 +33,8 @@ pub struct ImageSettings {
pub resolution: Option<String>,
pub quality: Option<String>,
pub count: u16,
/// The provider picked for each model (the rest run on their default).
pub providers: HashMap<String, Provider>,
}
#[derive(Default)]
@@ -51,70 +57,88 @@ impl CreateImagePage {
pub fn ui(&mut self, ui: &mut Ui, env: &mut Env<'_>) {
let models = &env.catalog.image_page;
let model = common::selected_model(models, self.settings.model.as_deref(), DEFAULT_IMAGE_MODEL).cloned();
if let Some(m) = &model {
self.reconcile(m);
let fields = self.fields(m, true);
self.cost.update(ui, env.backend, Modality::Image, &fields);
let base = common::selected_model(models, self.settings.model.as_deref(), DEFAULT_IMAGE_MODEL).cloned();
let provider = base.as_ref().map_or(Provider::Artcraft, |m| self.provider(m));
// What the model can do on its provider (direct Midjourney: four images, no references).
let model = base.as_ref().map(|m| provider.adapt(m));
if let (Some(base), Some(m)) = (&base, &model) {
self.reconcile(base);
let mut fields = self.fields(m, true);
if provider != Provider::Artcraft {
fields.insert("provider".into(), json!(provider));
}
let backend = env.backend;
self.cost.update_with(ui, &fields, |key, fields| backend.estimate_provider_cost(provider, key, fields));
}
let mut feed = std::mem::take(&mut self.feed);
let mut box_height = self.box_height;
common::create_shell(ui, env, MediaKind::Image, "Create Image", "Add a prompt, then generate", &mut feed, &mut box_height, |ui, env| self.prompt_box(ui, env, model.as_ref()));
common::create_shell(ui, env, MediaKind::Image, "Create Image", "Add a prompt, then generate", &mut feed, &mut box_height, |ui, env| self.prompt_box(ui, env, model.as_ref(), provider));
self.feed = feed;
self.box_height = box_height;
}
fn prompt_box(&mut self, ui: &mut Ui, env: &mut Env<'_>, model: Option<&ModelInfo>) {
/// The provider `model` runs on: the user's pick, else its default.
fn provider(&self, model: &ModelInfo) -> Provider {
Provider::resolve(&model.id, self.settings.providers.get(&model.id).copied())
}
fn prompt_box(&mut self, ui: &mut Ui, env: &mut Env<'_>, model: Option<&ModelInfo>, provider: Provider) {
let max_images = model.map_or(0, |m| m.image_refs_max);
let deck = if max_images > 0 { DeckMode::References(DeckLimits { max_images, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: env.signed_in }) } else { DeckMode::None };
let accepts: &[RefKind] = if max_images > 0 { &[RefKind::Image] } else { &[] };
let prompt_max = model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max);
let props = PromptBoxProps { id: Id::new("image-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: Some(prompt_max), deck, accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some(), generating: self.generating, generate_tooltip: "Generate", warning: None, banner: None };
let props = PromptBoxProps { id: Id::new("image-prompt-box"), placeholder: PLACEHOLDER, enter_to_generate: env.enter_to_generate, mentions: &[], max_length: Some(prompt_max), deck, accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some(), generating: self.generating, generate_tooltip: "Generate", warning: None, banner: None, extra_input: false };
let models = &env.catalog.image_page;
let settings = &mut self.settings;
let mut picked_model = None;
let mut left = |ui: &mut Ui| {
if let Some(id) = model_selector(ui, Id::new("image-model"), models, model, false) {
picked_model = Some(id);
}
let Some(m) = model else {
return;
};
let aspect = ModelInfo::resolve(settings.aspect_ratio.as_deref(), &m.aspect_ratios, m.aspect_default.as_ref());
if let Some(v) = toolbar::aspect_ratio(ui, Id::new("image-aspect"), m, aspect.as_deref().unwrap_or("")) {
settings.aspect_ratio = Some(v);
}
let resolution = ModelInfo::resolve(settings.resolution.as_deref(), &m.resolutions, m.resolution_default.as_ref());
if let Some(v) = toolbar::resolution(ui, Id::new("image-resolution"), m, resolution.as_deref().unwrap_or("")) {
settings.resolution = Some(v);
}
let quality = ModelInfo::resolve(settings.quality.as_deref(), &m.qualities, m.quality_default.as_ref());
if let Some(v) = toolbar::quality(ui, Id::new("image-quality"), m, quality.as_deref().unwrap_or("")) {
settings.quality = Some(v);
}
};
let count = settings.count;
let mut picked_model = None;
let mut picked_count = None;
let mut right = |ui: &mut Ui| {
if let Some(m) = model {
picked_count = toolbar::count(ui, Id::new("image-count"), m, count, "images");
}
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
(Slot::ToolbarLeft, _) => {
if let Some(pick) = model_selector(ui, Id::new("image-model"), models, model, false, Some(&settings.providers)) {
picked_model = Some(pick);
}
let Some(m) = model else {
return;
};
let aspect = ModelInfo::resolve(settings.aspect_ratio.as_deref(), &m.aspect_ratios, m.aspect_default.as_ref());
if let Some(v) = toolbar::aspect_ratio(ui, Id::new("image-aspect"), m, aspect.as_deref().unwrap_or("")) {
settings.aspect_ratio = Some(v);
}
let resolution = ModelInfo::resolve(settings.resolution.as_deref(), &m.resolutions, m.resolution_default.as_ref());
if let Some(v) = toolbar::resolution(ui, Id::new("image-resolution"), m, resolution.as_deref().unwrap_or("")) {
settings.resolution = Some(v);
}
let quality = ModelInfo::resolve(settings.quality.as_deref(), &m.qualities, m.quality_default.as_ref());
if let Some(v) = toolbar::quality(ui, Id::new("image-quality"), m, quality.as_deref().unwrap_or("")) {
settings.quality = Some(v);
}
},
(Slot::ToolbarRight, Some(m)) => {
if let Some(n) = toolbar::count(ui, Id::new("image-count"), m, count, "images") {
picked_count = Some(n);
}
},
_ => {},
};
let actions = prompt_box::show(ui, &mut settings.prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut left, &mut right);
if let Some(id) = picked_model {
let actions = prompt_box::show(ui, &mut settings.prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
if let Some((id, picked_provider)) = picked_model {
if let Some(p) = picked_provider {
self.settings.providers.insert(id.clone(), p);
}
self.settings.model = Some(id);
}
if let Some(n) = picked_count {
self.settings.count = n;
}
for action in actions {
self.handle(action, env, model, max_images);
self.handle(action, env, model, max_images, provider);
}
}
fn handle(&mut self, action: PromptBoxAction, env: &mut Env<'_>, model: Option<&ModelInfo>, max_images: usize) {
fn handle(&mut self, action: PromptBoxAction, env: &mut Env<'_>, model: Option<&ModelInfo>, max_images: usize, provider: Provider) {
match action {
PromptBoxAction::Generate => self.generate(env, model),
PromptBoxAction::Generate => self.generate(env, model, provider),
PromptBoxAction::ClearAll => {
self.settings.prompt.clear();
self.refs.clear();
@@ -143,6 +167,7 @@ impl CreateImagePage {
DeckAction::SwapFrames => {},
},
PromptBoxAction::DroppedFiles(paths) => self.add_dropped(env, &paths, max_images),
PromptBoxAction::MentionPicked(..) => {},
PromptBoxAction::PastedImage(png) => {
if self.refs.images.len() >= max_images {
env.toasts.error(max_images_message(max_images));
@@ -182,7 +207,7 @@ impl CreateImagePage {
}
}
fn generate(&mut self, env: &mut Env<'_>, model: Option<&ModelInfo>) {
fn generate(&mut self, env: &mut Env<'_>, model: Option<&ModelInfo>, provider: Provider) {
if self.generating || self.settings.prompt.trim().is_empty() {
return;
}
@@ -202,9 +227,24 @@ impl CreateImagePage {
env.toasts.error("Please wait for your references to finish uploading");
return;
}
// Other providers take the references as links (when the model takes references there).
let image_urls = if provider == Provider::Artcraft || model.image_refs_max == 0 {
Vec::new()
} else {
match self.refs.images.iter().map(|r| r.full_url.clone()).collect::<Option<Vec<String>>>() {
Some(urls) => urls,
None => {
env.toasts.error("A reference image is missing its link. Please add it again.");
return;
},
}
};
self.generating = true;
let meta = EnqueueMeta { prompt: self.settings.prompt.clone(), model_id: model.id.clone(), batch_count: u32::from(self.settings.count.max(1)), ref_image: self.refs.images.first().and_then(|r| r.preview.clone()) };
env.backend.generate(Modality::Image, self.fields(model, false), meta);
let meta = EnqueueMeta { prompt: self.settings.prompt.clone(), model_id: model.id.clone(), batch_count: u32::from(model.valid_batch(self.settings.count)), ref_image: self.refs.images.first().and_then(|r| r.preview.clone()), provider };
match provider {
Provider::Artcraft => env.backend.generate(Modality::Image, self.fields(model, false), meta),
_ => env.backend.generate_with_provider(provider, self.fields(model, false), image_urls, meta),
}
}
/// The OmniGen request (`GenerateImageRequest` after `api_fields`). `estimate` swaps
+122 -23
View File
@@ -9,6 +9,7 @@ use serde::{Deserialize, Serialize};
use serde_json::{Map, Value, json};
use crate::backend::wire::Prompt;
use crate::backend::providers::Provider;
use crate::backend::{EnqueueMeta, Modality};
use crate::feed::store::FeedStore;
use crate::feed::types::MediaKind;
@@ -20,7 +21,7 @@ use crate::prompt_box::editor::{self, MentionItem, MentionKind};
use crate::prompt_box::model_selector::model_selector;
use crate::prompt_box::pickers::{self, PickOption, Trigger};
use crate::prompt_box::refs::{ImageSlot, RefKind, RefMedia, References};
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState};
use crate::prompt_box::{self, DeckMode, PromptBoxAction, PromptBoxProps, PromptBoxState, Slot};
use crate::ui::icons::Icon;
use crate::ui::widgets::Leading;
@@ -67,6 +68,8 @@ pub struct CreateVideoPage {
pub generating: bool,
box_state: PromptBoxState,
box_height: f32,
/// Which character each mentioned name means, when the user picked one explicitly.
character_picks: std::collections::HashMap<String, String>,
}
impl CreateVideoPage {
@@ -104,10 +107,14 @@ impl CreateVideoPage {
(DeckMode::References(l), _) => [(RefKind::Image, l.max_images), (RefKind::Video, l.max_videos), (RefKind::Audio, l.max_audios)].into_iter().filter(|(_, n)| *n > 0).map(|(k, _)| k).collect(),
_ => Vec::new(),
};
let mentions = if reference { self.mentions() } else { Vec::new() };
let characters_on = model.is_some_and(supports_characters);
let mut mentions = if reference { self.mentions() } else { Vec::new() };
if characters_on {
mentions.extend(editor::character_mentions(env.characters.iter().map(|c| (c.name.as_str(), c.token.as_str(), c.avatar_url()))));
}
let needs_start = model.is_some_and(|m| self.needs_start_frame(m));
let banner = model.filter(|m| !m.text_to_video && !self.has_image_input()).map(|_| TEXT_ONLY_BANNER);
let props = PromptBoxProps { id: Id::new("video-prompt-box"), placeholder: if reference { PLACEHOLDER_REFERENCE } else { PLACEHOLDER }, enter_to_generate: env.enter_to_generate, mentions: &mentions, max_length: Some(model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max)), deck, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !needs_start, generating: self.generating, generate_tooltip: if needs_start { "Add a starting image before generating" } else { "Generate" }, warning: needs_start.then_some("Starting frame required"), banner };
let props = PromptBoxProps { id: Id::new("video-prompt-box"), placeholder: if reference { PLACEHOLDER_REFERENCE } else { PLACEHOLDER }, enter_to_generate: env.enter_to_generate, mentions: &mentions, max_length: Some(model.map_or(Some(DEFAULT_PROMPT_MAX), |m| m.prompt_max)), deck, accepts: &accepts, credits: self.cost.credits, generate_enabled: !self.settings.prompt.trim().is_empty() && model.is_some() && !needs_start, generating: self.generating, generate_tooltip: if needs_start { "Add a starting image before generating" } else { "Generate" }, warning: needs_start.then_some("Starting frame required"), banner, extra_input: false };
let models = &env.catalog.video_page;
let mut prompt = std::mem::take(&mut self.settings.prompt);
@@ -116,21 +123,28 @@ impl CreateVideoPage {
let with_refs = reference && !self.refs.images.is_empty();
let mut picked_model = None;
let mut picked_mode = None;
let mut left = |ui: &mut Ui| {
if let Some(id) = model_selector(ui, Id::new("video-model"), models, model, true) {
picked_model = Some(id);
}
if let Some(m) = model {
picked_mode = video_toolbar(ui, m, settings, with_refs).or(picked_mode);
}
};
let mut picked_count = None;
let mut right = |ui: &mut Ui| {
if let Some(m) = model {
picked_count = toolbar::count(ui, Id::new("video-count"), m, count, "videos");
}
let mut open_characters = false;
let mut slots = |ui: &mut Ui, slot: Slot| match (slot, model) {
(Slot::ToolbarLeft, _) => {
if let Some((id, _)) = model_selector(ui, Id::new("video-model"), models, model, true, None) {
picked_model = Some(id);
}
if let Some(m) = model {
picked_mode = video_toolbar(ui, m, settings, with_refs).or(picked_mode);
if characters_on && pickers::toggle(ui, Id::new("video-characters"), Icon::User, "@Characters", false, "Characters") {
open_characters = true;
}
}
},
(Slot::ToolbarRight, Some(m)) => {
if let Some(n) = toolbar::count(ui, Id::new("video-count"), m, count, "videos") {
picked_count = Some(n);
}
},
_ => {},
};
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut left, &mut right);
let actions = prompt_box::show(ui, &mut prompt, &self.refs, &mut self.box_state, &props, env.cache, &mut slots);
self.settings.prompt = prompt;
if let Some(id) = picked_model {
self.settings.model = Some(id);
@@ -141,6 +155,9 @@ impl CreateVideoPage {
if let Some(mode) = picked_mode {
switch_mode(&mut self.settings, &mut self.refs, mode);
}
if open_characters {
env.requests.push(AppRequest::OpenCharacters);
}
for action in actions {
self.handle(action, env, model, limits.as_ref());
}
@@ -176,14 +193,27 @@ impl CreateVideoPage {
let item = self.refs.images.remove(from);
self.refs.images.insert(to.min(self.refs.images.len()), item);
},
DeckAction::Preview(id) => {
if let Some(url) = self.refs.find_mut(id).and_then(|r| r.full_url.clone().or_else(|| r.preview.clone())) {
env.requests.push(AppRequest::Preview(url));
}
DeckAction::Preview(id) => match self.refs.find_mut(id) {
// Audio cards play and stop on click (the webapp's DeckCard).
Some(r) if r.kind == RefKind::Audio => {
if let Some(url) = r.full_url.clone() {
env.requests.push(AppRequest::ToggleAudio { ref_id: id, url });
}
},
Some(r) => {
if let Some(url) = r.full_url.clone().or_else(|| r.preview.clone()) {
env.requests.push(if r.kind == RefKind::Video { AppRequest::PlayExternally(url) } else { AppRequest::Preview(url) });
}
},
None => {},
},
DeckAction::SwapFrames => self.refs.swap_frames(),
},
PromptBoxAction::DroppedFiles(paths) => self.add_files(env, &paths, ImageSlot::Reference, limits),
PromptBoxAction::MentionPicked(label, Some(token)) => {
self.character_picks.insert(label.trim_start_matches('@').to_owned(), token);
},
PromptBoxAction::MentionPicked(_, None) => {},
PromptBoxAction::PastedImage(png) => match self.settings.input_mode {
InputMode::Keyframe => match self.free_keyframe(model) {
Some(slot) => common::upload_png(env, &mut self.refs, png, slot),
@@ -287,8 +317,28 @@ impl CreateVideoPage {
}
self.generating = true;
let ref_image = self.refs.first_frame.as_ref().or(self.refs.images.first()).and_then(|r| r.preview.clone());
let meta = EnqueueMeta { prompt: self.settings.prompt.clone(), model_id: model.id.clone(), batch_count: u32::from(model.valid_batch(self.settings.count)), ref_image };
env.backend.generate(Modality::Video, self.fields(model, false), meta);
let meta = EnqueueMeta { prompt: self.settings.prompt.clone(), model_id: model.id.clone(), batch_count: u32::from(model.valid_batch(self.settings.count)), ref_image, provider: Provider::Artcraft };
let mut fields = self.fields(model, false);
if supports_characters(model) {
let tokens = mentioned_characters(&self.settings.prompt, env.characters, &self.character_picks);
if !tokens.is_empty() {
fields.insert("reference_character_tokens".into(), json!(tokens));
}
}
env.backend.generate(Modality::Video, fields, meta);
}
/// The characters dialog picked `name`: mention it at the end of the prompt.
pub fn mention_character(&mut self, name: &str, token: &str) {
let p = &mut self.settings.prompt;
if !p.is_empty() && !p.ends_with(char::is_whitespace) {
p.push(' ');
}
p.push('@');
p.push_str(name);
p.push(' ');
self.character_picks.insert(name.to_owned(), token.to_owned());
self.box_state.editor.focus_requested = true;
}
/// The OmniGen request (`GenerateVideoRequest` after `api_fields`).
@@ -433,7 +483,7 @@ impl CreateVideoPage {
"audioref" => RefKind::Audio,
_ => RefKind::Image,
};
let item = RefMedia::from_library(kind, c.media_token.clone(), c.media_links.thumbnail(256), Some(c.media_links.cdn_url.clone()), 0.0);
let item = RefMedia::from_library(kind, c.media_token.clone(), c.media_links.thumbnail(256).filter(|_| kind != RefKind::Audio), Some(c.media_links.cdn_url.clone()), 0.0);
match (c.semantic.as_str(), kind) {
("vid_end_frame", _) => self.refs.last_frame = Some(item),
(_, RefKind::Image) if s.input_mode == InputMode::Keyframe && self.refs.first_frame.is_none() => self.refs.first_frame = Some(item),
@@ -506,6 +556,33 @@ fn switch_mode(s: &mut VideoSettings, refs: &mut References, mode: InputMode) {
s.input_mode = mode;
}
/// Whether the model takes `@Character` mentions (the listing says so; Seedance 2.0 always does).
fn supports_characters(m: &ModelInfo) -> bool {
m.character_refs_max > 0 || m.id == "seedance_2p0"
}
/// One token per character named in the prompt (`@Name`, not inside a longer word): the user's
/// explicit pick for that name, else the newest character with it.
fn mentioned_characters(prompt: &str, characters: &[crate::backend::wire::Character], picks: &std::collections::HashMap<String, String>) -> Vec<String> {
let mut names: Vec<&str> = characters.iter().map(|c| c.name.as_str()).filter(|n| !n.is_empty()).collect();
names.sort_by_key(|n| std::cmp::Reverse(n.len()));
names.dedup();
let mut tokens = Vec::new();
for name in names {
let needle = format!("@{name}");
let mentioned = prompt.match_indices(&needle).any(|(i, _)| !prompt[i + needle.len()..].chars().next().is_some_and(|c| c.is_alphanumeric() || c == '_'));
if !mentioned {
continue;
}
let candidates: Vec<&crate::backend::wire::Character> = characters.iter().filter(|c| c.name == name).collect();
let chosen = picks.get(name).and_then(|t| candidates.iter().find(|c| &c.token == t)).or(candidates.first());
if let Some(c) = chosen {
tokens.push(c.token.clone());
}
}
tokens
}
fn deck_limits(m: &ModelInfo, library: bool) -> DeckLimits {
DeckLimits { max_images: m.image_refs_max, max_videos: m.video_refs_max, max_video_secs: m.video_refs_max_secs.map(f32::from), max_audios: m.audio_refs_max, max_audio_secs: m.audio_refs_max_secs.map(f32::from), library }
}
@@ -539,3 +616,25 @@ fn frames_full_message(model: Option<&ModelInfo>) -> &'static str {
"The first frame is already set"
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::backend::wire::Character;
fn character(token: &str, name: &str) -> Character {
Character { token: token.into(), name: name.into(), ..Default::default() }
}
#[test]
fn mentions_resolve_to_one_token_per_name() {
// Newest first, as the server lists them.
let characters = [character("c3", "Bob"), character("c2", "Bob2"), character("c1", "Bob"), character("c0", "Ann")];
let none = std::collections::HashMap::new();
assert_eq!(mentioned_characters("@Bob2 waves", &characters, &none), ["c2"], "@Bob2 isn't @Bob");
assert_eq!(mentioned_characters("@Bob, then @Ann.", &characters, &none), ["c3", "c0"], "newest Bob by default");
let picks = std::collections::HashMap::from([("Bob".to_owned(), "c1".to_owned())]);
assert_eq!(mentioned_characters("hi @Bob", &characters, &picks), ["c1"], "the explicit pick wins");
assert!(mentioned_characters("no mentions", &characters, &none).is_empty());
}
}
+1
View File
@@ -1,4 +1,5 @@
pub mod common;
pub mod create_audio;
pub mod create_image;
pub mod create_video;
pub mod other;
+6 -5
View File
@@ -23,12 +23,13 @@ pub fn home(ui: &mut Ui, signed_in: bool, requests: &mut Vec<AppRequest>) -> Opt
ui.add_space(10.0);
ui.label(widgets::heading("Capable tools for artists.", 52.0));
ui.add_space(10.0);
ui.label(egui::RichText::new("Generate images and videos with the best models, steer them with references and keyframes, and keep everything in your library.").size(17.0).color(theme::MUTED));
ui.label(egui::RichText::new("Generate images, videos and audio with the best models, steer them with references and keyframes, and keep everything in your library.").size(17.0).color(theme::MUTED));
ui.add_space(28.0);
ui.horizontal(|ui| {
ui.spacing_mut().item_spacing.x = 16.0;
let card_w = (width - 16.0) / 2.0;
for (page, icon, title, text) in [(Page::CreateImage, Icon::Image, "Create Image", "Describe anything. See it in seconds."), (Page::CreateVideo, Icon::Video, "Create Video", "Describe a scene. See it in motion.")] {
let cards = [(Page::CreateImage, Icon::Image, "Create Image", "Describe anything. See it in seconds."), (Page::CreateVideo, Icon::Video, "Create Video", "Describe a scene. See it in motion."), (Page::CreateAudio, Icon::Music, "Create Audio", "Describe a song, a sound, or a sample.")];
let card_w = (width - 16.0 * (cards.len() - 1) as f32) / cards.len() as f32;
for (page, icon, title, text) in cards {
if card(ui, card_w, icon, title, text) {
open = Some(page);
}
@@ -73,9 +74,9 @@ pub fn library(ui: &mut Ui, env: &mut Env<'_>, feed: &mut FeedStore) {
placeholder(ui, area, "Library", "Your generations will appear here.", None);
return;
}
let view = FeedView { id: Id::new("library-feed"), mode: env.view_mode, pending: &[], failed: &[], items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: true, bottom_padding: 24.0 };
let view = FeedView { id: Id::new("library-feed"), mode: env.view_mode, pending: &[], failed: &[], items: &feed.items, has_more: feed.has_more, loading: feed.loading, selecting: feed.selecting, selected: &feed.selected, last_viewed: feed.last_viewed.as_deref(), prompts: env.prompts, make_video: true, autoplay: env.autoplay, bottom_padding: 24.0 };
let mut child = ui.new_child(egui::UiBuilder::new().max_rect(area.shrink2(vec2(12.0, 0.0)).with_min_y(area.top() + 2.0)));
for action in grid::show(&mut child, &view, env.cache, env.ratios, env.catalog) {
for action in grid::show(&mut child, &view, env.cache, env.ratios, env.audio, env.catalog) {
match action {
FeedAction::Open(token) => {
let kind = feed.find(&token).map_or(MediaKind::Image, |i| i.kind);
+15 -2
View File
@@ -309,12 +309,14 @@ fn paint_card(ui: &Ui, painter: &egui::Painter, rect: Rect, tilt: f32, item: &Re
image.paint_at(ui, rect.shrink(2.0));
},
None => {
let playing = item.kind == RefKind::Audio && playing_audio(ui) == Some(item.id);
let icon = match item.kind {
RefKind::Image => Icon::Image,
RefKind::Video => Icon::Video,
RefKind::Audio => Icon::Music,
RefKind::Audio if playing => Icon::Volume,
RefKind::Audio => Icon::Play,
};
icons::paint(painter, Rect::from_center_size(rect.center(), vec2(20.0, 20.0)), icon, theme::MUTED);
icons::paint(painter, Rect::from_center_size(rect.center(), vec2(20.0, 20.0)), icon, if playing { theme::ACCENT_INK } else { theme::MUTED });
},
}
if item.kind != RefKind::Image && tilt == 0.0 {
@@ -334,6 +336,17 @@ fn paint_card(ui: &Ui, painter: &egui::Painter, rect: Rect, tilt: f32, item: &Re
painter.add(Shape::closed_line(corners.to_vec(), Stroke::new(2.0, border)));
}
/// The audio reference playing right now (the app sets it each frame).
pub fn set_playing_audio(ctx: &egui::Context, id: Option<u64>) {
ctx.data_mut(|d| d.insert_temp(Id::new(PLAYING_AUDIO), id));
}
fn playing_audio(ui: &Ui) -> Option<u64> {
ui.data(|d| d.get_temp::<Option<u64>>(Id::new(PLAYING_AUDIO))).flatten()
}
const PLAYING_AUDIO: &str = "deck-playing-audio";
/// The hover-revealed round × in a card's top-right corner. Returns whether it was clicked.
fn remove_button(ui: &mut Ui, id: Id, card: Rect, card_hovered: bool) -> bool {
let rect = Rect::from_min_size(pos2(card.right() - 22.0, card.top() + 2.0), vec2(20.0, 20.0));
+33 -15
View File
@@ -22,12 +22,15 @@ const RIGHT_GUTTER: f32 = 32.0;
const IMAGE_COLORS: [Color32; 5] = [Color32::from_rgb(96, 165, 250), Color32::from_rgb(251, 146, 60), Color32::from_rgb(167, 139, 250), Color32::from_rgb(52, 211, 153), Color32::from_rgb(251, 113, 133)];
const VIDEO_COLORS: [Color32; 3] = [Color32::from_rgb(250, 204, 21), Color32::from_rgb(245, 158, 11), Color32::from_rgb(74, 222, 128)];
const AUDIO_COLORS: [Color32; 2] = [Color32::from_rgb(192, 132, 252), Color32::from_rgb(232, 121, 249)];
/// Teal, emerald, sky.
const CHARACTER_COLORS: [Color32; 3] = [Color32::from_rgb(45, 212, 191), Color32::from_rgb(34, 197, 94), Color32::from_rgb(14, 165, 233)];
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum MentionKind {
Image,
Video,
Audio,
Character,
}
/// Something the prompt can reference by name, e.g. `@Image2`.
@@ -39,6 +42,8 @@ pub struct MentionItem {
pub index: usize,
/// Media cache key of a thumbnail, if there is one.
pub preview: Option<String>,
/// The character this mention names (several can share a name).
pub token: Option<String>,
}
impl MentionItem {
@@ -48,6 +53,7 @@ impl MentionItem {
MentionKind::Image => IMAGE_COLORS[i % IMAGE_COLORS.len()],
MentionKind::Video => VIDEO_COLORS[i % VIDEO_COLORS.len()],
MentionKind::Audio => AUDIO_COLORS[i % AUDIO_COLORS.len()],
MentionKind::Character => CHARACTER_COLORS[i % CHARACTER_COLORS.len()],
}
}
}
@@ -93,6 +99,8 @@ pub struct EditorOutput {
pub submit: bool,
pub open_fullscreen: bool,
pub focused: bool,
/// A mention the user picked from the dropdown: (label, character token).
pub picked: Option<(String, Option<String>)>,
}
/// Draws the editor in the available width.
@@ -105,7 +113,7 @@ pub fn show(ui: &mut Ui, prompt: &mut String, state: &mut EditorState, opts: &Ed
// Keys the field mustn't see: dropdown navigation, and Enter when it generates.
if had_focus {
if !filtered.is_empty() {
handle_dropdown_keys(ui, id, prompt, state, &filtered);
out.picked = handle_dropdown_keys(ui, id, prompt, state, &filtered);
} else if opts.enter_to_generate && consume_plain_enter(ui) {
out.submit = true;
}
@@ -183,7 +191,9 @@ pub fn show(ui: &mut Ui, prompt: &mut String, state: &mut EditorState, opts: &Ed
}
if !filtered.is_empty() && out.focused {
mention_dropdown(ui, id, field_rect, prompt, state, &filtered, thumbnails);
if let Some(picked) = mention_dropdown(ui, id, field_rect, prompt, state, &filtered, thumbnails) {
out.picked = Some(picked);
}
}
out
}
@@ -214,12 +224,10 @@ fn consume_plain_enter(ui: &Ui) -> bool {
})
}
fn handle_dropdown_keys(ui: &Ui, id: Id, prompt: &mut String, state: &mut EditorState, filtered: &[&MentionItem]) {
fn handle_dropdown_keys(ui: &Ui, id: Id, prompt: &mut String, state: &mut EditorState, filtered: &[&MentionItem]) -> Option<(String, Option<String>)> {
let n = filtered.len();
let (down, up, accept, escape) = ui.input_mut(|i| (i.consume_key(Modifiers::NONE, Key::ArrowDown), i.consume_key(Modifiers::NONE, Key::ArrowUp), i.consume_key(Modifiers::NONE, Key::Enter) || i.consume_key(Modifiers::NONE, Key::Tab), i.consume_key(Modifiers::NONE, Key::Escape)));
let Some(query) = state.mention.as_mut() else {
return;
};
let query = state.mention.as_mut()?;
if down {
query.selected = (query.selected + 1) % n;
}
@@ -228,12 +236,14 @@ fn handle_dropdown_keys(ui: &Ui, id: Id, prompt: &mut String, state: &mut Editor
}
if escape {
state.mention = None;
return;
return None;
}
if accept {
let label = filtered[query.selected.min(n - 1)].label.clone();
insert_mention(ui.ctx(), id, prompt, state, &label);
if !accept {
return None;
}
let item = filtered[query.selected.min(n - 1)];
insert_mention(ui.ctx(), id, prompt, state, &item.label);
Some((item.label.clone(), item.token.clone()))
}
/// Opens, updates or closes the dropdown from the text before the cursor.
@@ -284,7 +294,7 @@ fn insert_mention(ctx: &egui::Context, id: Id, prompt: &mut String, state: &mut
}
}
fn mention_dropdown(ui: &Ui, id: Id, field: Rect, prompt: &mut String, state: &mut EditorState, filtered: &[&MentionItem], thumbnails: &mut dyn FnMut(&Ui, &str) -> Option<egui::TextureHandle>) {
fn mention_dropdown(ui: &Ui, id: Id, field: Rect, prompt: &mut String, state: &mut EditorState, filtered: &[&MentionItem], thumbnails: &mut dyn FnMut(&Ui, &str) -> Option<egui::TextureHandle>) -> Option<(String, Option<String>)> {
let selected = state.mention.as_ref().map_or(0, |q| q.selected);
let row_h = 44.0;
let h = 28.0 + row_h * filtered.len() as f32;
@@ -317,23 +327,25 @@ fn mention_dropdown(ui: &Ui, id: Id, field: Rect, prompt: &mut String, state: &m
MentionKind::Image => Icon::Image,
MentionKind::Video => Icon::Video,
MentionKind::Audio => Icon::Music,
MentionKind::Character => Icon::User,
};
icons::paint(ui.painter(), tile.shrink(9.0), icon, theme::fade(Color32::WHITE, 0.6));
},
}
ui.painter().text(pos2(tile.right() + 10.0, row.center().y), Align2::LEFT_CENTER, &item.label, FontId::new(14.0, theme::medium()), item.color());
if resp.clicked() {
picked = Some(item.label.clone());
picked = Some((item.label.clone(), item.token.clone()));
}
}
});
if let (Some(i), Some(q)) = (hovered, state.mention.as_mut()) {
q.selected = i;
}
if let Some(label) = picked {
insert_mention(ui.ctx(), id, prompt, state, &label);
if let Some((label, _)) = &picked {
insert_mention(ui.ctx(), id, prompt, state, label);
ui.memory_mut(|m| m.request_focus(id));
}
picked
}
/// The three-line grip that resizes the field (`.promptbox-resize-wrap::after`).
@@ -382,14 +394,20 @@ fn highlight(text: &str, mentions: &[MentionItem]) -> LayoutJob {
job
}
/// `@Name` mentions for characters (newest first), coloured by position.
pub fn character_mentions<'a>(characters: impl Iterator<Item = (&'a str, &'a str, Option<String>)>) -> Vec<MentionItem> {
characters.enumerate().map(|(i, (name, token, avatar))| MentionItem { label: format!("@{name}"), kind: MentionKind::Character, index: i + 1, preview: avatar, token: Some(token.to_owned()) }).collect()
}
/// Mention labels for `n` items of `kind` (`@Image1`, `@Image2`, …).
pub fn mention_items(kind: MentionKind, previews: impl Iterator<Item = Option<String>>) -> Vec<MentionItem> {
let name = match kind {
MentionKind::Image => "Image",
MentionKind::Video => "Video",
MentionKind::Audio => "Audio",
MentionKind::Character => "Character",
};
previews.enumerate().map(|(i, preview)| MentionItem { label: format!("@{name}{}", i + 1), kind, index: i + 1, preview }).collect()
previews.enumerate().map(|(i, preview)| MentionItem { label: format!("@{name}{}", i + 1), kind, index: i + 1, preview, token: None }).collect()
}
#[cfg(test)]
+35 -14
View File
@@ -58,6 +58,21 @@ pub struct PromptBoxProps<'a> {
pub warning: Option<&'a str>,
/// An info strip above the prompt.
pub banner: Option<&'a str>,
/// Something clear-all also clears besides the prompt and references (the audio style).
pub extra_input: bool,
}
/// Where a page draws into the box.
#[derive(Clone, Copy, Debug, PartialEq, Eq)]
pub enum Slot {
/// Rows above the prompt (the audio box's references).
AbovePrompt,
/// Rows below the prompt (the audio box's style).
BelowPrompt,
/// The toolbar's left: the model selector and pickers.
ToolbarLeft,
/// The toolbar's right, before clear-all: the count picker.
ToolbarRight,
}
/// Something the page should do.
@@ -68,6 +83,8 @@ pub enum PromptBoxAction {
ClearAll,
DroppedFiles(Vec<PathBuf>),
PastedImage(Vec<u8>),
/// A mention picked from the dropdown: (label, character token).
MentionPicked(String, Option<String>),
}
/// UI state that outlives a frame.
@@ -79,11 +96,10 @@ pub struct PromptBoxState {
focused: bool,
}
/// Draws the box in `ui` (which should be as wide as the box). `toolbar_left` holds the model
/// selector and pickers; `toolbar_right` the count picker. Both may be drawn twice in a frame
/// (box and focus mode), each under its own id.
#[allow(clippy::too_many_arguments)]
pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, toolbar_left: &mut dyn FnMut(&mut Ui), toolbar_right: &mut dyn FnMut(&mut Ui)) -> Vec<PromptBoxAction> {
/// Draws the box in `ui` (which should be as wide as the box). `slots` draws the page's parts
/// into each [`Slot`]; the toolbar's left may be drawn twice in a frame (box and focus mode), each
/// under its own id.
pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, slots: &mut dyn FnMut(&mut Ui, Slot)) -> Vec<PromptBoxAction> {
let mut actions = Vec::new();
let frame = egui::Frame::new().fill(theme::fade(theme::CONTROLS, 0.95)).stroke(Stroke::new(1.0, if state.focused { theme::ACCENT } else { theme::LINE })).corner_radius(theme::RADIUS).inner_margin(16);
let response = frame.show(ui, |ui| {
@@ -94,25 +110,30 @@ pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut Pro
ui.spacing_mut().item_spacing.x = 12.0;
deck_widget(ui, props, refs, cache, false, &mut actions);
ui.vertical(|ui| {
slots(ui, Slot::AbovePrompt);
let out = editor_ui(ui, prompt, state, props, cache, None);
state.focused = out.focused;
if out.submit {
actions.push(PromptBoxAction::Generate);
}
if let Some((label, token)) = out.picked {
actions.push(PromptBoxAction::MentionPicked(label, token));
}
if out.open_fullscreen {
state.fullscreen = true;
}
slots(ui, Slot::BelowPrompt);
});
});
ui.add_space(14.0);
ui.horizontal(|ui| {
ui.spacing_mut().item_spacing.x = 8.0;
toolbar_left(ui);
slots(ui, Slot::ToolbarLeft);
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
ui.spacing_mut().item_spacing.x = 6.0;
generate_cluster(ui, props, &mut actions);
clear_all_button(ui, prompt, refs, state, &mut actions);
toolbar_right(ui);
clear_all_button(ui, prompt, refs, state, props.extra_input, &mut actions);
slots(ui, Slot::ToolbarRight);
if let Some(warning) = props.warning {
warning_label(ui, warning);
}
@@ -124,7 +145,7 @@ pub fn show(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut Pro
drop_target(ui, props, rect, &mut actions);
paste_images(ui, props, &mut actions);
if state.fullscreen {
focus_mode(ui.ctx(), prompt, refs, state, props, cache, toolbar_left, &mut actions);
focus_mode(ui.ctx(), prompt, refs, state, props, cache, slots, &mut actions);
}
if state.confirm_clear {
confirm_clear(ui.ctx(), props.id, state, &mut actions);
@@ -167,10 +188,10 @@ fn generate_cluster(ui: &mut Ui, props: &PromptBoxProps<'_>, actions: &mut Vec<P
}
/// The eraser button and the hairline after it (`PromptClearAllButton`).
fn clear_all_button(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, actions: &mut Vec<PromptBoxAction>) {
fn clear_all_button(ui: &mut Ui, prompt: &mut String, refs: &References, state: &mut PromptBoxState, extra_input: bool, actions: &mut Vec<PromptBoxAction>) {
let (rule, _) = ui.allocate_exact_size(vec2(1.0, 20.0), Sense::hover());
ui.painter().rect_filled(rule, 0.0, theme::fade(theme::INK, 0.15));
let clearable = !prompt.is_empty() || !refs.is_empty();
let clearable = !prompt.is_empty() || !refs.is_empty() || extra_input;
let resp = ui.add_enabled_ui(clearable, |ui| widgets::ghost(ui, Icon::Eraser, None, "Clear all", false)).inner;
if resp.clicked() {
if refs.is_empty() {
@@ -307,7 +328,7 @@ fn paste_images(ui: &mut Ui, props: &PromptBoxProps<'_>, actions: &mut Vec<Promp
/// Focus mode: a large editor in a modal, with the deck and toolbar at hand.
#[allow(clippy::too_many_arguments)]
fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, toolbar_left: &mut dyn FnMut(&mut Ui), actions: &mut Vec<PromptBoxAction>) {
fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state: &mut PromptBoxState, props: &PromptBoxProps<'_>, cache: &mut MediaCache, slots: &mut dyn FnMut(&mut Ui, Slot), actions: &mut Vec<PromptBoxAction>) {
let height = ctx.content_rect().height() * 0.7;
let mut done = false;
let closed = widgets::modal(ctx, props.id.with("focus-mode"), 896.0, |ui| {
@@ -330,12 +351,12 @@ fn focus_mode(ctx: &egui::Context, prompt: &mut String, refs: &References, state
ui.push_id("focus-deck", |ui| deck_widget(ui, props, refs, cache, true, actions));
ui.add_space(12.0);
ui.horizontal(|ui| {
ui.push_id("focus-toolbar", |ui| ui.horizontal(|ui| toolbar_left(ui)));
ui.push_id("focus-toolbar", |ui| ui.horizontal(|ui| slots(ui, Slot::ToolbarLeft)));
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if widgets::button(ui, None, "Done", widgets::Kind::Primary, 0.0, theme::CONTROL_H).clicked() {
done = true;
}
ui.push_id("focus-clear", |ui| clear_all_button(ui, prompt, refs, state, actions));
ui.push_id("focus-clear", |ui| clear_all_button(ui, prompt, refs, state, props.extra_input, actions));
});
});
});
+188 -38
View File
@@ -1,19 +1,31 @@
//! The embedded model selector (`ClassyModelSelector variant="embedded"`): a pill with the
//! maker's logo and the model's name, opening "Select Model". Long lists group by family, each
//! family opening a flyout of its variants on hover.
//! family opening a flyout of its variants on hover. Models that run on more than one provider
//! show the provider's logo, and hovering them opens "Select Provider".
use egui::{Align2, Id, Order, Popup, Rect, Ui, pos2};
use std::collections::HashMap;
use egui::{Align2, Id, Order, Popup, Rect, Stroke, Ui, pos2, vec2};
use crate::backend::providers::Provider;
use crate::models::{self, ModelInfo};
use crate::prompt_box::pickers;
use crate::theme;
use crate::ui::widgets::{self, Leading, RowLead};
use crate::ui::{creator_icons, icons};
const PANEL_WIDTH: f32 = 280.0;
const FLYOUT_WIDTH: f32 = 300.0;
const PROVIDER_FLYOUT_WIDTH: f32 = 220.0;
/// Hovering a model this long opens its providers (`providerTooltipDelayMs`).
const PROVIDER_DELAY_SECS: f64 = 0.3;
/// Draws the selector; returns the id of a newly picked model.
pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Option<&ModelInfo>, show_badges: bool) -> Option<String> {
/// What was picked: a model, and the provider when one was chosen with it.
pub type Pick = (String, Option<Provider>);
/// Draws the selector; returns a newly picked model (and provider). `providers` is the page's
/// provider choice per model (`None` where providers can't be chosen).
pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Option<&ModelInfo>, show_badges: bool, providers: Option<&HashMap<String, Provider>>) -> Option<Pick> {
let popup_id = ui.make_persistent_id(id).with("popup");
let open = Popup::is_id_open(ui.ctx(), popup_id);
let label = selected.map_or("Select model", |m| m.name.as_str());
@@ -26,6 +38,7 @@ pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Optio
}
});
let resp = resp.inner.on_hover_text("Model");
let rows = Rows { id, selected, show_badges, providers };
let mut picked = None;
let hovered_family_id = id.with("family");
pickers::popover(&resp, popup_id, |ui| {
@@ -42,12 +55,12 @@ pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Optio
let subtitle = if contains_selected { selected.map(|s| s.name.as_str()) } else { None };
let is_open = hovered.as_deref() == Some(family.name.as_str());
if family.models.len() == 1 {
let row = widgets::menu_row(ui, RowLead::Logo(&tex), &first.name, non_empty(&first.description), contains_selected, false, false);
if row.hovered() {
let (row_hovered, pick) = rows.row(ui, first);
if row_hovered {
hovered = None;
}
if row.clicked() {
picked = Some(first.id.clone());
if pick.is_some() {
picked = pick;
}
continue;
}
@@ -56,8 +69,8 @@ pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Optio
hovered = Some(family.name.clone());
}
if hovered.as_deref() == Some(family.name.as_str()) {
if let Some(id) = flyout(ui, id, row.rect, &family.name, &family.models, selected, show_badges) {
picked = Some(id);
if let Some(pick) = rows.family_flyout(ui, row.rect, &family.name, &family.models) {
picked = Some(pick);
}
}
}
@@ -65,8 +78,8 @@ pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Optio
},
None => {
for m in models {
if model_row(ui, m, selected, show_badges) {
picked = Some(m.id.clone());
if let (_, Some(pick)) = rows.row(ui, m) {
picked = Some(pick);
}
}
},
@@ -74,21 +87,109 @@ pub fn model_selector(ui: &mut Ui, id: Id, models: &[ModelInfo], selected: Optio
});
if picked.is_some() {
Popup::close_id(ui.ctx(), popup_id);
ui.data_mut(|d| d.remove::<Option<String>>(hovered_family_id));
ui.data_mut(|d| {
d.remove::<Option<String>>(hovered_family_id);
d.remove::<Option<(String, f64)>>(id.with("provider-hover"));
});
}
picked.filter(|p| selected.is_none_or(|s| &s.id != p))
picked.filter(|(model, provider)| provider.is_some() || selected.is_none_or(|s| &s.id != model))
}
/// A family's variants, to the right of its row.
fn flyout(ui: &Ui, id: Id, row: Rect, family: &str, list: &[&ModelInfo], selected: Option<&ModelInfo>, show_badges: bool) -> Option<String> {
/// What every row needs to know.
struct Rows<'a> {
id: Id,
selected: Option<&'a ModelInfo>,
show_badges: bool,
providers: Option<&'a HashMap<String, Provider>>,
}
impl Rows<'_> {
/// A family's variants, to the right of its row.
fn family_flyout(&self, ui: &Ui, row: Rect, family: &str, list: &[&ModelInfo]) -> Option<Pick> {
let mut picked = None;
egui::Area::new(self.id.with(("flyout", family))).order(Order::Tooltip).pivot(Align2::LEFT_TOP).fixed_pos(pos2(row.right() + 10.0, row.top() - 32.0)).show(ui.ctx(), |ui| {
widgets::popover_frame().show(ui, |ui| {
ui.set_width(FLYOUT_WIDTH);
widgets::menu_header(ui, family);
for m in list {
if let (_, Some(pick)) = self.row(ui, m) {
picked = Some(pick);
}
}
});
});
picked
}
/// One model: logo tile, name, description (and capability badges for video), and the
/// provider chip and flyout for models with a choice. Returns whether the row is hovered, and
/// a pick.
fn row(&self, ui: &mut Ui, m: &ModelInfo) -> (bool, Option<Pick>) {
let tex = creator_icons::texture(ui.ctx(), &m.creator);
let is_selected = self.selected.is_some_and(|s| s.id == m.id);
let badges = if self.show_badges { capability_badges(m) } else { String::new() };
let subtitle = match (m.description.is_empty(), badges.is_empty()) {
(true, true) => None,
(false, true) => Some(m.description.clone()),
(true, false) => Some(badges),
(false, false) => Some(format!("{} \u{b7} {badges}", m.description)),
};
let allowed = Provider::for_model(&m.id);
let Some(picks) = self.providers.filter(|_| allowed.len() > 1) else {
let row = widgets::menu_row(ui, RowLead::Logo(&tex), &m.name, subtitle.as_deref(), is_selected, false, false);
if row.hovered() {
ui.data_mut(|d| d.remove::<Option<(String, f64)>>(self.id.with("provider-hover")));
}
return (row.hovered(), row.clicked().then(|| (m.id.clone(), None)));
};
let current = Provider::resolve(&m.id, picks.get(&m.id).copied());
let hover_id = self.id.with("provider-hover");
let now = ui.input(|i| i.time);
let hovered: Option<(String, f64)> = ui.data(|d| d.get_temp(hover_id)).flatten();
let open = hovered.as_ref().is_some_and(|(id, since)| *id == m.id && now - since >= PROVIDER_DELAY_SECS);
let row = widgets::menu_row(ui, RowLead::Logo(&tex), &m.name, subtitle.as_deref(), false, false, open);
provider_chip(ui, row.rect, current, is_selected);
if row.hovered() && hovered.as_ref().is_none_or(|(id, _)| *id != m.id) {
ui.data_mut(|d| d.insert_temp(hover_id, Some((m.id.clone(), now))));
}
if row.hovered() && !open {
ui.ctx().request_repaint_after(std::time::Duration::from_secs_f64(PROVIDER_DELAY_SECS));
}
let mut picked = row.clicked().then(|| (m.id.clone(), None));
if open {
if let Some(provider) = provider_flyout(ui, self.id.with(("providers", &m.id)), row.rect, allowed, current) {
picked = Some((m.id.clone(), Some(provider)));
}
}
(row.hovered(), picked)
}
}
/// The provider's logo on a chip at the row's right; on the brand blue for the selected model.
fn provider_chip(ui: &Ui, row: Rect, provider: Provider, selected: bool) {
let chip = Rect::from_center_size(pos2(row.right() - 22.0, row.center().y), vec2(26.0, 20.0));
let fill = if selected { theme::fade(theme::ACCENT, 0.6) } else { theme::WASH };
ui.painter().rect_filled(chip, theme::RADIUS, fill);
let tex = creator_icons::texture(ui.ctx(), provider.icon());
egui::Image::new(&tex).tint(theme::INK).paint_at(ui, Rect::from_center_size(chip.center(), vec2(12.0, 12.0)));
}
/// "Select Provider" (`ProviderTooltipContent`), to the right of a model's row.
fn provider_flyout(ui: &Ui, id: Id, row: Rect, allowed: &[Provider], current: Provider) -> Option<Provider> {
let mut picked = None;
egui::Area::new(id.with(("flyout", family))).order(Order::Tooltip).pivot(Align2::LEFT_TOP).fixed_pos(pos2(row.right() + 10.0, row.top() - 32.0)).show(ui.ctx(), |ui| {
egui::Area::new(id).order(Order::Tooltip).pivot(Align2::LEFT_TOP).fixed_pos(pos2(row.right() + 10.0, row.top() - 32.0)).show(ui.ctx(), |ui| {
widgets::popover_frame().show(ui, |ui| {
ui.set_width(FLYOUT_WIDTH);
widgets::menu_header(ui, family);
for m in list {
if model_row(ui, m, selected, show_badges) {
picked = Some(m.id.clone());
ui.set_width(PROVIDER_FLYOUT_WIDTH);
widgets::menu_header(ui, "Select Provider");
for &provider in allowed {
let tex = creator_icons::texture(ui.ctx(), provider.icon());
let selected = provider == current;
let resp = widgets::menu_row(ui, RowLead::SmallLogo(&tex), provider.label(), None, selected, false, false);
if selected {
ui.painter().vline(resp.rect.left() + 1.5, resp.rect.y_range(), Stroke::new(3.0, theme::ACCENT));
}
if resp.clicked() {
picked = Some(provider);
}
}
});
@@ -96,20 +197,6 @@ fn flyout(ui: &Ui, id: Id, row: Rect, family: &str, list: &[&ModelInfo], selecte
picked
}
/// One model: logo tile, name, description (and capability badges for video).
fn model_row(ui: &mut Ui, m: &ModelInfo, selected: Option<&ModelInfo>, show_badges: bool) -> bool {
let tex = creator_icons::texture(ui.ctx(), &m.creator);
let is_selected = selected.is_some_and(|s| s.id == m.id);
let badges = if show_badges { capability_badges(m) } else { String::new() };
let subtitle = match (m.description.is_empty(), badges.is_empty()) {
(true, true) => None,
(false, true) => Some(m.description.clone()),
(true, false) => Some(badges),
(false, false) => Some(format!("{} \u{b7} {badges}", m.description)),
};
widgets::menu_row(ui, RowLead::Logo(&tex), &m.name, subtitle.as_deref(), is_selected, false, false).clicked()
}
/// The video picker's capability badges ("Audio Support", "Start/End", "Reference").
fn capability_badges(m: &ModelInfo) -> String {
let mut badges = Vec::new();
@@ -125,6 +212,69 @@ fn capability_badges(m: &ModelInfo) -> String {
badges.join(" \u{b7} ")
}
fn non_empty(s: &str) -> Option<&str> {
(!s.is_empty()).then_some(s)
/// The provider submenu headlessly (egui_kittest).
#[cfg(test)]
mod tests {
use egui_kittest::Harness;
use egui_kittest::kittest::Queryable;
use super::*;
struct State {
models: Vec<ModelInfo>,
picks: HashMap<String, Provider>,
picked: Vec<Pick>,
fonts_ready: bool,
}
#[test]
fn hovering_a_model_offers_its_providers() {
let mut harness = build();
harness.get_by_label("Seedream 4").click();
harness.run();
let row = harness.get_all_by_label("Nano Banana Pro").last().expect("the model's row");
row.hover();
for _ in 0..4 {
harness.step();
}
harness.run();
assert!(harness.query_by_label("ArtCraft").is_some(), "the submenu lists the providers");
harness.get_by_label("FAL").click();
harness.run();
assert_eq!(harness.state().picked, vec![("nano_banana_pro".to_owned(), Some(Provider::Fal))]);
}
#[test]
fn models_without_a_choice_have_no_submenu() {
let mut harness = build();
harness.get_by_label("Seedream 4").click();
harness.run();
harness.get_all_by_label("Seedream 4").last().expect("row").hover();
for _ in 0..4 {
harness.step();
}
harness.run();
assert!(harness.query_by_label("FAL").is_none(), "no provider list");
}
fn build() -> Harness<'static, State> {
let models = ["seedream_4", "nano_banana_pro"].iter().map(|id| ModelInfo { id: (*id).into(), name: models::display_name(id), creator: models::creator_for(id).into(), ..Default::default() }).collect();
let state = State { models, picks: HashMap::new(), picked: Vec::new(), fonts_ready: false };
let mut harness = Harness::builder().with_size(egui::vec2(1000.0, 700.0)).build_ui_state(
|ui, s: &mut State| {
if !s.fonts_ready {
theme::apply(ui.ctx());
s.fonts_ready = true;
return;
}
ui.add_space(500.0);
if let Some(pick) = model_selector(ui, Id::new("test-model"), &s.models, s.models.first(), false, Some(&s.picks)) {
s.picked.push(pick);
}
},
state,
);
harness.run();
harness
}
}
+9 -4
View File
@@ -90,14 +90,19 @@ pub fn popover(anchor: &Response, popup_id: Id, content: impl FnOnce(&mut Ui)) {
});
}
/// A toolbar pill that opens a popover of custom `content` above it.
pub fn popover_pill(ui: &mut Ui, id: Id, lead: Leading<'_>, label: &str, tooltip: &str, content: impl FnOnce(&mut Ui)) {
let popup_id = ui.make_persistent_id(id).with("popup");
let open = Popup::is_id_open(ui.ctx(), popup_id);
let resp = ui.push_id(id, |ui| widgets::pill(ui, lead, label, open)).inner.on_hover_text(tooltip);
popover(&resp, popup_id, content);
}
/// The duration popover: a clock pill reading `5s`, opening a 1-second-step slider between the
/// model's minimum and maximum. Returns the new value.
pub fn duration_picker(ui: &mut Ui, id: Id, current: u32, min: u32, max: u32) -> Option<u32> {
let popup_id = ui.make_persistent_id(id).with("popup");
let open = Popup::is_id_open(ui.ctx(), popup_id);
let resp = ui.push_id(id, |ui| widgets::pill(ui, Leading::Icon(Icon::Clock), &format!("{current}s"), open)).inner.on_hover_text("Duration");
let mut value = current;
popover(&resp, popup_id, |ui| {
popover_pill(ui, id, Leading::Icon(Icon::Clock), &format!("{current}s"), "Duration", |ui| {
ui.set_width(240.0);
widgets::menu_header(ui, "Duration");
ui.add_space(4.0);
+9 -8
View File
@@ -11,7 +11,7 @@ use crate::pages::toolbar;
use crate::prompt_box::deck::DeckLimits;
use crate::prompt_box::editor::{MentionKind, mention_items};
use crate::prompt_box::refs::{RefKind, References};
use crate::prompt_box::{DeckMode, PromptBoxProps, PromptBoxState, show};
use crate::prompt_box::{DeckMode, PromptBoxProps, PromptBoxState, Slot, show};
struct State {
// Keeps the media cache's runtime alive.
@@ -45,15 +45,16 @@ fn build(state: State) -> Harness<'static, State> {
return;
}
let mentions = if s.with_mentions { mention_items(MentionKind::Image, [None, None].into_iter()) } else { Vec::new() };
let props = PromptBoxProps { id: egui::Id::new("test-box"), placeholder: "Describe what you want in the image...", enter_to_generate: s.enter_to_generate, mentions: &mentions, max_length: Some(Some(1000)), deck: DeckMode::References(DeckLimits { max_images: 4, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: false }), accepts: &[RefKind::Image], credits: Some(15), generate_enabled: !s.prompt.trim().is_empty(), generating: false, generate_tooltip: "Generate", warning: None, banner: None };
let props = PromptBoxProps { id: egui::Id::new("test-box"), placeholder: "Describe what you want in the image...", enter_to_generate: s.enter_to_generate, mentions: &mentions, max_length: Some(Some(1000)), deck: DeckMode::References(DeckLimits { max_images: 4, max_videos: 0, max_video_secs: None, max_audios: 0, max_audio_secs: None, library: false }), accepts: &[RefKind::Image], credits: Some(15), generate_enabled: !s.prompt.trim().is_empty(), generating: false, generate_tooltip: "Generate", warning: None, banner: None, extra_input: false };
let (model, aspect) = (&s.model, &mut s.aspect);
let mut left = |ui: &mut egui::Ui| {
if let Some(v) = toolbar::aspect_ratio(ui, egui::Id::new("test-aspect"), model, aspect) {
*aspect = v;
let mut slots = |ui: &mut egui::Ui, slot: Slot| {
if slot == Slot::ToolbarLeft {
if let Some(v) = toolbar::aspect_ratio(ui, egui::Id::new("test-aspect"), model, aspect) {
*aspect = v;
}
}
};
let mut right = |_: &mut egui::Ui| {};
let actions = show(ui, &mut s.prompt, &s.refs, &mut s.box_state, &props, &mut s.cache, &mut left, &mut right);
let actions = show(ui, &mut s.prompt, &s.refs, &mut s.box_state, &props, &mut s.cache, &mut slots);
s.actions.extend(actions.iter().map(|a| format!("{a:?}")));
},
state,
@@ -128,7 +129,7 @@ fn typing_at_offers_mentions_and_enter_inserts_one() {
harness.key_press(egui::Key::Enter);
harness.run();
assert_eq!(harness.state().prompt, "put @Image2 ");
assert!(harness.state().actions.is_empty(), "Enter picked a mention, it didn't generate");
assert_eq!(harness.state().actions, [r#"MentionPicked("@Image2", None)"#], "Enter picked a mention, it didn't generate");
}
#[test]
+23 -6
View File
@@ -87,6 +87,7 @@ impl Page {
"home" => Page::Home,
"create-image" | "image" => Page::CreateImage,
"create-video" | "video" => Page::CreateVideo,
"create-audio" | "audio" => Page::CreateAudio,
"library" => Page::Library,
_ => return None,
})
@@ -94,7 +95,7 @@ impl Page {
/// Pages the native app has; the rest open a "coming soon" page.
pub fn is_ready(self) -> bool {
matches!(self, Page::Home | Page::CreateImage | Page::CreateVideo | Page::Library)
matches!(self, Page::Home | Page::CreateImage | Page::CreateVideo | Page::CreateAudio | Page::Library)
}
/// The same tool in the web app.
@@ -133,6 +134,7 @@ pub enum ShellAction {
SignOut,
OpenSettings,
SetViewMode(ViewMode),
SetAutoplay(bool),
ToggleSelect,
}
@@ -286,8 +288,17 @@ fn account_row(ui: &mut Ui, name: &str, credits: Option<u64>) -> egui::Response
resp.on_hover_cursor(egui::CursorIcon::PointingHand).on_hover_text("Settings")
}
/// The content panel's header: breadcrumbs, and the select and grid/list toggles on feed pages.
pub fn page_header(ui: &mut Ui, page: Page, feed_toggles: Option<(ViewMode, bool)>) -> Option<ShellAction> {
/// The feed toggles a page's header shows.
pub struct FeedToggles {
pub mode: ViewMode,
pub selecting: bool,
/// Shown on pages with videos (`GalleryAutoplayToggle`).
pub autoplay: Option<bool>,
}
/// The content panel's header: breadcrumbs, and the select, previews and grid/list toggles on
/// feed pages.
pub fn page_header(ui: &mut Ui, page: Page, feed_toggles: Option<FeedToggles>) -> Option<ShellAction> {
let mut action = None;
let rect = ui.max_rect();
ui.painter().hline(rect.x_range(), rect.bottom() - 0.5, theme::hairline());
@@ -302,12 +313,18 @@ pub fn page_header(ui: &mut Ui, page: Page, feed_toggles: Option<(ViewMode, bool
let (crumb, _) = ui.allocate_exact_size(galley.size() + vec2(12.0, 8.0), Sense::hover());
ui.painter().rect_filled(crumb, 0.0, theme::INK);
ui.painter().galley(crumb.center() - galley.size() / 2.0, galley, theme::BG);
if let Some((mode, selecting)) = feed_toggles {
if let Some(t) = feed_toggles {
ui.with_layout(egui::Layout::right_to_left(egui::Align::Center), |ui| {
if let Some(m) = view_toggle(ui, mode) {
if let Some(m) = view_toggle(ui, t.mode) {
action = Some(ShellAction::SetViewMode(m));
}
if segmented(ui, &[(Icon::SquareCheck, if selecting { "Exit selection" } else { "Select items" }, selecting)]).is_some() {
if let Some(playing) = t.autoplay {
// Right-to-left: stills first so "play" ends up on the left.
if let Some(i) = segmented(ui, &[(Icon::Image, "Still thumbnails", !playing), (Icon::Play, "Play video previews", playing)]) {
action = Some(ShellAction::SetAutoplay(i == 1));
}
}
if segmented(ui, &[(Icon::SquareCheck, if t.selecting { "Exit selection" } else { "Select items" }, t.selecting)]).is_some() {
action = Some(ShellAction::ToggleSelect);
}
});
+54
View File
@@ -18,6 +18,7 @@ pub enum Icon {
Coins,
Copy,
Download,
Drum,
Eraser,
Eye,
ExternalLink,
@@ -35,17 +36,24 @@ pub enum Icon {
LogIn,
LogOut,
Maximize,
Mic,
MicOff,
Music,
PanelLeft,
Pause,
Pencil,
Play,
Plus,
Repeat,
RotateCw,
Settings,
SlidersHorizontal,
Sparkles,
SquareCheck,
Tags,
Trash,
Upload,
User,
Video,
Volume,
VolumeOff,
@@ -157,6 +165,14 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.line(p, &[(7.0, 10.0), (12.0, 15.0), (17.0, 10.0)]);
g.line(p, &[(4.0, 21.0), (20.0, 21.0)]);
},
Icon::Drum => {
g.line(p, &[(2.0, 2.0), (10.0, 8.0)]);
g.line(p, &[(22.0, 2.0), (14.0, 8.0)]);
g.closed(p, &[(2.0, 9.0), (6.0, 6.5), (12.0, 5.5), (18.0, 6.5), (22.0, 9.0), (18.0, 11.5), (12.0, 12.5), (6.0, 11.5)]);
g.line(p, &[(2.0, 9.0), (2.0, 17.0), (6.0, 19.5), (12.0, 20.5), (18.0, 19.5), (22.0, 17.0), (22.0, 9.0)]);
g.line(p, &[(7.0, 11.8), (7.0, 19.8)]);
g.line(p, &[(17.0, 11.8), (17.0, 19.8)]);
},
Icon::Eraser => {
g.closed(p, &[(7.0, 21.0), (3.5, 17.5), (14.0, 7.0), (21.0, 14.0), (14.0, 21.0)]);
g.line(p, &[(22.0, 21.0), (7.0, 21.0)]);
@@ -248,6 +264,14 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.line(p, &[(21.0, 3.0), (14.0, 10.0)]);
g.line(p, &[(3.0, 21.0), (10.0, 14.0)]);
},
Icon::Mic | Icon::MicOff => {
g.rect(p, 9.0, 2.0, 6.0, 13.0, 3.0);
g.line(p, &[(19.0, 10.0), (19.0, 12.0), (17.0, 16.0), (12.0, 19.0), (7.0, 16.0), (5.0, 12.0), (5.0, 10.0)]);
g.line(p, &[(12.0, 19.0), (12.0, 22.0)]);
if icon == Icon::MicOff {
g.line(p, &[(2.0, 2.0), (22.0, 22.0)]);
}
},
Icon::Music => {
g.line(p, &[(9.0, 18.0), (9.0, 5.0), (21.0, 3.0), (21.0, 16.0)]);
g.circle(p, 6.0, 18.0, 3.0);
@@ -257,6 +281,10 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.rect(p, 3.0, 3.0, 18.0, 18.0, 2.0);
g.line(p, &[(9.0, 3.0), (9.0, 21.0)]);
},
Icon::Pause => {
p.rect_filled(Rect::from_min_size(g.p(6.0, 4.0), vec2(4.0, 16.0) * g.scale), 1.0, color);
p.rect_filled(Rect::from_min_size(g.p(14.0, 4.0), vec2(4.0, 16.0) * g.scale), 1.0, color);
},
Icon::Pencil => {
g.closed(p, &[(17.0, 3.0), (21.0, 7.0), (7.5, 20.5), (2.0, 22.0), (3.5, 16.5)]);
g.line(p, &[(15.0, 5.0), (19.0, 9.0)]);
@@ -269,6 +297,12 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.line(p, &[(5.0, 12.0), (19.0, 12.0)]);
g.line(p, &[(12.0, 5.0), (12.0, 19.0)]);
},
Icon::Repeat => {
g.line(p, &[(17.0, 2.0), (21.0, 6.0), (17.0, 10.0)]);
g.line(p, &[(3.0, 11.0), (3.0, 10.0), (5.0, 7.0), (7.0, 6.0), (21.0, 6.0)]);
g.line(p, &[(7.0, 22.0), (3.0, 18.0), (7.0, 14.0)]);
g.line(p, &[(21.0, 13.0), (21.0, 14.0), (19.0, 17.0), (17.0, 18.0), (3.0, 18.0)]);
},
Icon::RotateCw => {
g.arc(p, 12.0, 12.0, 9.0, -60.0, 250.0);
g.line(p, &[(21.0, 3.0), (21.0, 8.5), (15.5, 8.5)]);
@@ -281,6 +315,17 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.line(p, &[(12.0 + 7.5 * a.cos(), 12.0 + 7.5 * a.sin()), (12.0 + 10.0 * a.cos(), 12.0 + 10.0 * a.sin())]);
}
},
Icon::SlidersHorizontal => {
g.line(p, &[(21.0, 4.0), (14.0, 4.0)]);
g.line(p, &[(10.0, 4.0), (3.0, 4.0)]);
g.line(p, &[(21.0, 12.0), (12.0, 12.0)]);
g.line(p, &[(8.0, 12.0), (3.0, 12.0)]);
g.line(p, &[(21.0, 20.0), (16.0, 20.0)]);
g.line(p, &[(12.0, 20.0), (3.0, 20.0)]);
g.line(p, &[(14.0, 2.0), (14.0, 6.0)]);
g.line(p, &[(8.0, 10.0), (8.0, 14.0)]);
g.line(p, &[(16.0, 18.0), (16.0, 22.0)]);
},
Icon::Sparkles => {
let star = |cx: f32, cy: f32, r: f32| {
let k = r * 0.28;
@@ -289,6 +334,11 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
star(10.0, 12.0, 8.0);
star(19.0, 5.0, 3.0);
},
Icon::Tags => {
g.closed(p, &[(3.0, 5.0), (5.0, 3.0), (10.6, 3.0), (18.0, 10.4), (18.0, 12.6), (12.6, 18.0), (10.4, 18.0), (3.0, 10.6)]);
g.line(p, &[(15.0, 3.0), (22.0, 10.0), (22.0, 12.0), (17.0, 17.0)]);
g.dot(p, 7.5, 7.5, 1.1);
},
Icon::Trash => {
g.line(p, &[(3.0, 6.0), (21.0, 6.0)]);
g.line(p, &[(19.0, 6.0), (19.0, 20.0), (17.0, 22.0), (7.0, 22.0), (5.0, 20.0), (5.0, 6.0)]);
@@ -299,6 +349,10 @@ pub fn paint(painter: &Painter, rect: Rect, icon: Icon, color: Color32) {
g.line(p, &[(7.0, 8.0), (12.0, 3.0), (17.0, 8.0)]);
g.line(p, &[(4.0, 21.0), (20.0, 21.0)]);
},
Icon::User => {
g.circle(p, 12.0, 8.0, 4.5);
g.arc(p, 12.0, 22.0, 8.0, 180.0, 360.0);
},
Icon::Video => {
g.rect(p, 2.0, 6.0, 14.0, 12.0, 2.0);
g.closed(p, &[(16.0, 10.5), (22.0, 7.0), (22.0, 17.0), (16.0, 13.5)]);
+1
View File
@@ -1,5 +1,6 @@
pub mod creator_icons;
pub mod icons;
pub mod toast;
pub mod waveform;
pub mod widgets;
pub mod window;
+194
View File
@@ -0,0 +1,194 @@
//! The track player (`WaveformAudioPlayer`): a round white play button, the waveform (click to
//! seek, played part in blue), the time, and a volume flyout. Placeholder bars stand in while the
//! waveform decodes; a plain progress line if it can't.
use egui::{Color32, FontId, Id, Popup, Rect, Sense, Stroke, Ui, pos2, vec2};
use crate::backend::audio::{self, AudioPlayer, WaveformLookup};
use crate::prompt_box::pickers;
use crate::theme;
use crate::ui::icons::{self, Icon};
/// Bars drawn while the waveform loads (`PLACEHOLDER_BAR_COUNT`).
const PLACEHOLDER_BARS: usize = 48;
/// Waveform bars: width and the step between them.
const BAR_WIDTH: f32 = 2.0;
const BAR_STEP: f32 = 3.5;
const WAVE_INK: Color32 = Color32::from_rgba_premultiplied(89, 89, 89, 89);
const PROGRESS_INK: Color32 = Color32::from_rgb(0x2d, 0x81, 0xff);
/// How often a playing track redraws its progress.
const FRAME: std::time::Duration = std::time::Duration::from_millis(33);
/// Draws the player across `ui`'s width; `duration_hint` (seconds) is used until the waveform
/// knows the real length. `compact` is the list rows' smaller size. Returns the player's rect.
pub fn player(ui: &mut Ui, id: Id, audio: &mut AudioPlayer, url: &str, duration_hint: Option<f32>, compact: bool) -> Rect {
let (button_d, wave_h): (f32, f32) = if compact { (32.0, 32.0) } else { (40.0, 44.0) };
let (rect, _) = ui.allocate_exact_size(vec2(ui.available_width(), wave_h.max(button_d)), Sense::hover());
// Clicks between the controls stay with the player (they don't open the card underneath).
ui.interact(rect, id.with("bg"), Sense::click());
let ctx = ui.ctx().clone();
let waveform = audio.waveform(&ctx, url);
let playback = audio.track(url);
let duration = match &waveform {
WaveformLookup::Ready(w) => Some(w.duration),
_ => duration_hint,
}
.filter(|d| *d > 0.0);
let position = playback.map_or(0.0, |p| p.position);
let playing = playback.is_some_and(|p| !p.paused && !p.loading);
if playing {
ctx.request_repaint_after(FRAME);
}
// Play / pause.
let button = Rect::from_center_size(pos2(rect.left() + button_d / 2.0, rect.center().y), vec2(button_d, button_d));
let resp = ui.interact(button, id.with("play"), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
let label = if playing { "Pause" } else { "Play" };
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, label));
ui.painter().circle_filled(button.center(), button_d / 2.0, if resp.hovered() { theme::fade(Color32::WHITE, 0.88) } else { Color32::WHITE });
let glyph = Rect::from_center_size(button.center(), vec2(button_d * 0.4, button_d * 0.4));
match playback {
Some(p) if p.loading => icons::paint_spinner(ui, glyph, Color32::BLACK),
_ if playing => icons::paint(ui.painter(), glyph, Icon::Pause, Color32::BLACK),
_ => icons::paint(ui.painter(), glyph.translate(vec2(button_d * 0.04, 0.0)), Icon::Play, Color32::BLACK),
}
if resp.clicked() {
audio.toggle_track(&ctx, url);
}
// Volume, then the time, from the right.
let volume_rect = Rect::from_center_size(pos2(rect.right() - 14.0, rect.center().y), vec2(28.0, 28.0));
volume(ui, id, volume_rect, audio);
let time = format!("{} / {}", format_time(position), duration.map_or_else(|| "-:--".to_owned(), format_time));
let galley = ui.painter().layout_no_wrap(time, FontId::new(11.0, theme::mono()), theme::MUTED);
let time_x = volume_rect.left() - 8.0 - galley.size().x;
ui.painter().galley(pos2(time_x, rect.center().y - galley.size().y / 2.0), galley, theme::MUTED);
// The waveform between them.
let wave = Rect::from_min_max(pos2(button.right() + 10.0, rect.center().y - wave_h / 2.0), pos2(time_x - 10.0, rect.center().y + wave_h / 2.0));
if wave.width() > 8.0 {
let progress = duration.map_or(0.0, |d| (position / d).clamp(0.0, 1.0));
let wave_resp = ui.interact(wave, id.with("wave"), Sense::click()).on_hover_cursor(egui::CursorIcon::PointingHand);
match &waveform {
WaveformLookup::Ready(w) => paint_bars(ui, wave, &audio::resample(&w.peaks, bar_count(wave.width())), progress),
WaveformLookup::Loading => paint_bars(ui, wave, &placeholder_bars(url), progress),
WaveformLookup::Failed => paint_line(ui, wave, progress),
}
if let Some(pointer) = wave_resp.interact_pointer_pos().filter(|_| wave_resp.clicked()) {
let frac = ((pointer.x - wave.left()) / wave.width()).clamp(0.0, 1.0);
audio.seek_track(&ctx, url, duration.map_or(0.0, |d| frac * d));
}
}
rect
}
/// "m:ss".
pub fn format_time(secs: f32) -> String {
let s = secs.max(0.0) as u64;
format!("{}:{:02}", s / 60, s % 60)
}
/// The volume button and its 0–100 slider flyout.
fn volume(ui: &mut Ui, id: Id, rect: Rect, audio: &mut AudioPlayer) {
let popup_id = id.with("volume-popup");
let open = Popup::is_id_open(ui.ctx(), popup_id);
let resp = ui.interact(rect, id.with("volume"), Sense::click()).on_hover_text("Volume").on_hover_cursor(egui::CursorIcon::PointingHand);
resp.widget_info(|| egui::WidgetInfo::labeled(egui::WidgetType::Button, true, "Volume"));
if resp.hovered() || open {
ui.painter().rect_filled(rect, theme::RADIUS, theme::WASH);
}
let level = audio.track_volume();
icons::paint(ui.painter(), rect.shrink(7.0), if level <= 0.0 { Icon::VolumeOff } else { Icon::Volume }, if resp.hovered() || open { theme::INK } else { theme::DIM });
let mut percent = (level * 100.0).round() as u32;
pickers::popover(&resp, popup_id, |ui| {
ui.horizontal(|ui| {
ui.add_space(6.0);
ui.style_mut().spacing.slider_width = 120.0;
ui.add(egui::Slider::new(&mut percent, 0..=100).show_value(false));
ui.label(egui::RichText::new(percent.to_string()).size(11.0).family(theme::mono()).color(theme::MUTED));
ui.add_space(4.0);
});
});
if percent != (level * 100.0).round() as u32 {
audio.set_track_volume(percent as f32 / 100.0);
}
}
fn bar_count(width: f32) -> usize {
((width + BAR_STEP - BAR_WIDTH) / BAR_STEP).floor().max(1.0) as usize
}
/// Bars centred on the waveform's midline, blue up to `progress` (0–1).
fn paint_bars(ui: &Ui, rect: Rect, heights: &[f32], progress: f32) {
if heights.is_empty() {
return;
}
let step = rect.width() / heights.len() as f32;
let width = BAR_WIDTH.min(step * 0.7);
let split = rect.left() + rect.width() * progress;
let p = ui.painter();
for (i, h) in heights.iter().enumerate() {
let x = rect.left() + step * (i as f32 + 0.5);
let half = (h.clamp(0.0, 1.0) * rect.height() / 2.0).max(1.0);
let bar = Rect::from_center_size(pos2(x, rect.center().y), vec2(width, half * 2.0));
p.rect_filled(bar, 1.0, if x <= split { PROGRESS_INK } else { WAVE_INK });
}
}
/// The fallback when the waveform can't be decoded: a thin track with the played part in blue.
fn paint_line(ui: &Ui, rect: Rect, progress: f32) {
let track = Rect::from_center_size(rect.center(), vec2(rect.width(), 4.0));
ui.painter().rect_filled(track, 2.0, WAVE_INK);
ui.painter().rect_filled(Rect::from_min_size(track.min, vec2(track.width() * progress, track.height())), 2.0, PROGRESS_INK);
ui.painter().line_segment([pos2(track.left() + track.width() * progress, rect.top() + 6.0), pos2(track.left() + track.width() * progress, rect.bottom() - 6.0)], Stroke::new(1.0, PROGRESS_INK));
}
/// Deterministic stand-in bars for a track (seeded by its URL), 25–89 % tall.
pub fn placeholder_bars(url: &str) -> Vec<f32> {
let mut seed = url.chars().fold(0_u32, |s, c| s.wrapping_mul(31).wrapping_add(c as u32));
(0..PLACEHOLDER_BARS)
.map(|_| {
let percent = 25 + seed % 65;
seed = seed.wrapping_mul(1_103_515_245).wrapping_add(12_345);
percent as f32 / 100.0
})
.collect()
}
/// A tile with a music note (`MusicIcon` in its box), for audio without artwork.
pub fn music_tile(ui: &Ui, rect: Rect, round: bool) {
let p = ui.painter();
if round {
p.circle_filled(rect.center(), rect.width() / 2.0, theme::WASH_HOVER);
} else {
p.rect(rect, theme::RADIUS, theme::WASH, theme::hairline(), egui::StrokeKind::Inside);
}
icons::paint(p, Rect::from_center_size(rect.center(), rect.size() * 0.42), Icon::Music, theme::fade(Color32::WHITE, 0.75));
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn times_read_minutes_and_seconds() {
assert_eq!(format_time(0.0), "0:00");
assert_eq!(format_time(65.9), "1:05");
assert_eq!(format_time(600.0), "10:00");
}
#[test]
fn placeholder_bars_are_stable_per_track() {
let a = placeholder_bars("https://cdn/a.mp3");
assert_eq!(a.len(), PLACEHOLDER_BARS);
assert_eq!(a, placeholder_bars("https://cdn/a.mp3"));
assert_ne!(a, placeholder_bars("https://cdn/b.mp3"));
assert!(a.iter().all(|h| (0.25..0.9).contains(h)));
}
#[test]
fn bars_fill_the_width() {
assert_eq!(bar_count(2.0), 1);
assert_eq!(bar_count(352.0), 101);
}
}
+6
View File
@@ -252,6 +252,8 @@ pub enum RowLead<'a> {
Aspect(Option<(f32, f32)>),
/// A 36 px tile with a model maker's logo (white texture, tinted).
Logo(&'a egui::TextureHandle),
/// A logo at icon size (the provider list).
SmallLogo(&'a egui::TextureHandle),
}
/// One option in a popover: a lead, a title, an optional muted subtitle, and a check when
@@ -284,6 +286,10 @@ pub fn menu_row(ui: &mut Ui, lead: RowLead<'_>, title: &str, subtitle: Option<&s
egui::Image::new(texture).tint(ink).paint_at(ui, Rect::from_center_size(tile.center(), vec2(18.0, 18.0)));
x += 46.0;
},
RowLead::SmallLogo(texture) => {
egui::Image::new(texture).tint(ink).paint_at(ui, Rect::from_center_size(pos2(x + 8.0, rect.center().y), vec2(15.0, 15.0)));
x += 26.0;
},
}
let title_font = FontId::new(13.5, theme::medium());
match subtitle {
@@ -33,6 +33,7 @@ chrono.workspace = true
cookie.workspace = true
log.workspace = true
reqwest = { workspace = true, features = ["multipart", "gzip"] }
tokio = { workspace = true, features = ["fs"] }
url.workspace = true
# Serialization
@@ -60,7 +60,7 @@ pub async fn legacy_upload_media_file_from_file<P: AsRef<Path>>(
.build()
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_bytes = std::fs::read(args.path.as_ref())
let file_bytes = tokio::fs::read(args.path.as_ref()).await
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
@@ -69,7 +69,7 @@ pub async fn upload_image_media_file_from_file_with_idempotency_token<P: AsRef<P
.build()
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_bytes = std::fs::read(args.path.as_ref())
let file_bytes = tokio::fs::read(args.path.as_ref()).await
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
@@ -35,7 +35,7 @@ pub async fn upload_new_engine_asset_from_file<P: AsRef<Path>>(
let client = storyteller_client_builder(api_host)
.build()?;
let file_bytes = std::fs::read(path.as_ref())?;
let file_bytes = tokio::fs::read(path.as_ref()).await?;
let file_name = path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
let mut form = Form::new()
@@ -61,7 +61,7 @@ pub async fn upload_video_media_file_from_file<P: AsRef<Path>>(
let client = storyteller_client_builder(args.api_host)
.build()?;
let file_bytes = std::fs::read(args.path.as_ref())?;
let file_bytes = tokio::fs::read(args.path.as_ref()).await?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
let mut form = Form::new()
@@ -1,5 +1,3 @@
use std::fs;
use std::io::Write;
use std::path::{Path, PathBuf};
use log::{info, warn};
@@ -68,13 +66,7 @@ pub async fn download_media_file<P: AsRef<Path>>(
let bytes = download_bytes(cdn_url).await?;
// 4. Write to disk.
let mut file = fs::File::create(&output_path)
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
file.write_all(&bytes)
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
file.flush()
tokio::fs::write(&output_path, &bytes).await
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
info!("Downloaded {} bytes to {:?}", bytes.len(), output_path);
@@ -31,7 +31,7 @@ reqwest.workspace = true
serde.workspace = true
wreq.workspace = true
serde_derive.workspace = true
tokio.workspace = true
tokio = { workspace = true, features = ["fs"] }
uuid.workspace = true
[dev-dependencies]
@@ -6,3 +6,4 @@ pub mod image_generation_draft_context;
pub mod image_generation_draft_or_request;
pub mod image_generation_request;
pub mod providers;
pub mod router_image_request_to_artcraft_prompt;
@@ -1,21 +1,24 @@
//! The ArtCraft prompt record for an image request sent to another provider (Midjourney, fal),
//! created first so the results can be attributed to it.
use artcraft_api_defs::prompts::create_prompt::CreatePromptRequest;
use artcraft_router::api::router_aspect_ratio::RouterAspectRatio;
use artcraft_router::api::router_image_model::RouterImageModel;
use artcraft_router::api::router_resolution::RouterResolution;
use artcraft_router::api::router_provider::RouterProvider;
use artcraft_router::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
use enums::common::generation::common_aspect_ratio::CommonAspectRatio as EnumsAspectRatio;
use enums::common::generation::common_generation_mode::CommonGenerationMode;
use enums::common::generation::common_model_type::CommonModelType;
use enums::common::generation::common_resolution::CommonResolution as EnumsResolution;
use enums::common::generation_provider::GenerationProvider;
use uuid_utils::uuid::generate_random_uuid;
use crate::api::router_aspect_ratio::RouterAspectRatio;
use crate::api::router_image_model::RouterImageModel;
use crate::api::router_provider::RouterProvider;
use crate::api::router_resolution::RouterResolution;
use crate::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
pub fn router_image_request_to_artcraft_prompt(
request: &GenerateImageRequestBuilder,
) -> CreatePromptRequest {
CreatePromptRequest {
uuid_idempotency_token: generate_random_uuid(),
uuid_idempotency_token: uuid::Uuid::new_v4().to_string(),
positive_prompt: request.prompt.clone(),
negative_prompt: None,
model_type: image_model_to_common_model_type(request.model),
@@ -120,7 +123,7 @@ fn router_resolution_to_enums(res: RouterResolution) -> EnumsResolution {
#[cfg(test)]
mod tests {
use super::*;
use artcraft_router::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
use crate::client::request_mismatch_mitigation_strategy::RequestMismatchMitigationStrategy;
fn base_builder() -> GenerateImageRequestBuilder {
GenerateImageRequestBuilder {
@@ -165,7 +168,7 @@ mod tests {
#[test]
fn edit_mode_when_images_present() {
use artcraft_router::api::image_list_ref::ImageListRef;
use crate::api::image_list_ref::ImageListRef;
let builder = GenerateImageRequestBuilder {
image_inputs: Some(ImageListRef::Urls(vec!["https://example.com/img.jpg".to_string()])),
..base_builder()
@@ -24,7 +24,7 @@ pub(crate) async fn upload_to_kinovi_web(
.unwrap_or_else(|| "jpg".to_string());
let file_bytes = match maybe_local_path {
Some(local_path) => match std::fs::read(local_path) {
Some(local_path) => match tokio::fs::read(local_path).await {
Ok(bytes) => {
info!("Reusing predownloaded file for kinovi upload: {}", source_url);
bytes
@@ -14,14 +14,19 @@ browser_emulation.workspace = true
cloudflare_errors.workspace = true
cookie_store_wrapper = { workspace = true, features = ["wreq"] }
errors.workspace = true
uuid_utils.workspace = true
# External
anyhow.workspace = true
base64.workspace = true
chrono.workspace = true
futures.workspace = true
log.workspace = true
serde.workspace = true
serde_json.workspace = true
tempfile.workspace = true
tokio = { version = "1", features = ["full"] }
toml.workspace = true
url.workspace = true
wreq-util.workspace = true
wreq.workspace = true
@@ -0,0 +1,77 @@
//! Reading Midjourney's Firebase auth token (the `__Host-Midjourney.AuthUserTokenV3_i` cookie).
//! Its JWT payload is decoded WITHOUT verifying the signature: it's only used to learn the user
//! id, the email and when the token expires, never to trust anything.
use base64::Engine;
use base64::prelude::BASE64_URL_SAFE_NO_PAD;
use serde_json::Value;
use crate::credentials::cookie_names::AUTH_COOKIE_NAME_I;
use crate::credentials::midjourney_user_id::MidjourneyUserId;
/// The claim holding the Midjourney user id (the `singleplayer_{...}` channel id is built from
/// it).
const MIDJOURNEY_ID_CLAIM: &str = "midjourney_id";
/// Refresh this long before the token expires (seconds).
const REFRESH_MARGIN_SECS: i64 = 120;
/// The (unverified) payload of a `header.payload.signature` JWT.
pub fn jwt_claims(token: &str) -> Option<Value> {
let mut parts = token.split('.');
let (_header, payload, _signature) = (parts.next()?, parts.next()?, parts.next()?);
if parts.next().is_some() {
return None;
}
serde_json::from_slice(&BASE64_URL_SAFE_NO_PAD.decode(payload).ok()?).ok()
}
/// Whether the token is missing, unreadable, expired or about to expire.
pub fn token_needs_refresh(maybe_token: Option<&str>, now: i64) -> bool {
maybe_token.and_then(jwt_claims).and_then(|claims| claims.get("exp")?.as_i64()).is_none_or(|expires| expires <= now + REFRESH_MARGIN_SECS)
}
/// The Midjourney user id from the auth token's claims.
pub fn user_id_from_token(token: &str) -> Option<MidjourneyUserId> {
let claims = jwt_claims(token)?;
claims.get(MIDJOURNEY_ID_CLAIM).and_then(Value::as_str).filter(|id| !id.is_empty()).map(MidjourneyUserId::from_str)
}
/// The Midjourney user id from a raw `Cookie:` header (`"a=1; b=2"`), without hitting the
/// Cloudflare-gated index page.
pub fn extract_midjourney_user_id_from_cookie_header(cookie_header: &str) -> Option<MidjourneyUserId> {
let token = cookie_header.split(';').filter_map(|pair| pair.trim().split_once('=')).find(|(name, _)| *name == AUTH_COOKIE_NAME_I).map(|(_, value)| value)?;
user_id_from_token(token)
}
#[cfg(test)]
mod tests {
use super::*;
// Payload: {"midjourney_id":"26c9d38d-1f71-49c7-a356-b29dac58b54c","name":"u1"}
const JWT: &str = "eyJhbGciOiJIUzI1NiJ9.eyJtaWRqb3VybmV5X2lkIjoiMjZjOWQzOGQtMWY3MS00OWM3LWEzNTYtYjI5ZGFjNThiNTRjIiwibmFtZSI6InUxIn0.sig";
#[test]
fn extracts_user_id_from_auth_cookie() {
let header = format!("_fbp=abc; {}={}; other=x", AUTH_COOKIE_NAME_I, JWT);
let user_id = extract_midjourney_user_id_from_cookie_header(&header).unwrap();
assert_eq!(user_id.as_str(), "26c9d38d-1f71-49c7-a356-b29dac58b54c");
}
#[test]
fn returns_none_without_auth_cookie() {
assert!(extract_midjourney_user_id_from_cookie_header("_fbp=abc; other=x").is_none());
}
#[test]
fn refreshes_before_expiry_but_does_not_refresh_a_fresh_token() {
assert!(!token_needs_refresh(Some(&token(5000)), 1000));
assert!(token_needs_refresh(Some(&token(1120)), 1000));
assert!(token_needs_refresh(Some(&token(999)), 1000));
assert!(token_needs_refresh(Some("invalid"), 1000));
assert!(token_needs_refresh(None, 1000));
}
fn token(exp: i64) -> String {
format!("header.{}.signature", BASE64_URL_SAFE_NO_PAD.encode(serde_json::json!({ "exp": exp }).to_string()))
}
}
@@ -1,13 +1,17 @@
//! The saved Midjourney login (`midjourney_cookies.toml`): the browser cookies captured at
//! sign-in, the account they belong to, and when they were last refreshed or failed.
use std::io::Write;
use std::path::Path;
use browser_emulation::browser_profile::BrowserProfile;
use chrono::{DateTime, Utc};
use cookie_store_wrapper::cookie_store::CookieStore;
use serde::{Deserialize, Serialize};
use std::io::Write;
use std::path::Path;
use tempfile::NamedTempFile;
use uuid_utils::uuid::generate_random_uuid;
use crate::services::midjourney::utils::midjourney_browser_profile::MIDJOURNEY_USER_AGENT;
use crate::utils::midjourney_browser_profile::MIDJOURNEY_USER_AGENT;
/// Midjourney's slice of ArtCraftX's credential format. Other accounts retain
/// their existing storage. Websocket tokens are deliberately never persisted.
@@ -0,0 +1,287 @@
//! The Midjourney login shared by the desktop apps: kept in one file, refreshed before it
//! expires, and turned into a [`MidjourneySession`] for each submit or poll.
use std::path::PathBuf;
use std::sync::{Arc, RwLock};
use std::time::Duration;
use anyhow::{anyhow, bail};
use browser_emulation::browser_profile::BrowserProfile;
use chrono::Utc;
use cookie_store_wrapper::cookie_store::CookieStore;
use serde_json::Value;
use tokio::sync::Mutex;
use url::Url;
use crate::credentials::auth_token::{extract_midjourney_user_id_from_cookie_header, jwt_claims, token_needs_refresh};
use crate::credentials::cookie_names::{AUTH_COOKIE_NAME_I, AUTH_COOKIE_NAME_R};
use crate::credentials::cookie_store_has_auth_cookies::cookie_store_has_auth_cookies;
use crate::credentials::credential_file::{MidjourneyAccountInfo, MidjourneyCredential};
use crate::credentials::midjourney_user_id::MidjourneyUserId;
use crate::endpoints::refresh_token::{DEFAULT_FIREBASE_API_KEY, RefreshTokenArgs, RefreshTokenRequest, refresh_token};
use crate::error::midjourney_api_error::MidjourneyApiError;
use crate::error::midjourney_error::MidjourneyError;
use crate::recipes::get_user_info::{GetUserInfoArgs, get_user_info};
/// Where the auth cookies are scoped.
pub const MIDJOURNEY_ORIGIN: &str = "https://www.midjourney.com/";
/// Token refresh and account lookups give up after this long.
const NETWORK_TIMEOUT: Duration = Duration::from_secs(30);
#[derive(Clone)]
pub struct MidjourneyCredentialManager {
credential: Arc<RwLock<Option<MidjourneyCredential>>>,
// Serialize refresh, login, and logout so an old refresh cannot resurrect a
// signed-out session or overwrite credentials captured by a new login.
mutation: Arc<Mutex<()>>,
credential_path: PathBuf,
/// Older formats, removed on logout so they can't be migrated back.
legacy_paths: Vec<PathBuf>,
}
/// What a submit or poll needs: the cookies to send, whose account it is, and the browser to
/// present (the one the cookies were captured with).
pub struct MidjourneySession {
pub credential_id: String,
pub cookie_header: String,
pub user_id: MidjourneyUserId,
pub browser: BrowserProfile,
}
impl MidjourneyCredentialManager {
/// A manager that saves to `credential_path`, starting from `credential`.
pub fn new(credential_path: PathBuf, legacy_paths: Vec<PathBuf>, credential: Option<MidjourneyCredential>) -> Self {
Self { credential: Arc::new(RwLock::new(credential)), mutation: Arc::new(Mutex::new(())), credential_path, legacy_paths }
}
/// A manager for the login saved at `credential_path`, if there is one.
pub fn load(credential_path: PathBuf, legacy_paths: Vec<PathBuf>) -> anyhow::Result<Self> {
let credential = if credential_path.exists() {
let credential = MidjourneyCredential::load(&credential_path)?;
credential.save(&credential_path)?; // Persist a stable ID for hand-written files.
Some(credential)
} else {
None
};
Ok(Self::new(credential_path, legacy_paths, credential))
}
pub fn credential_path(&self) -> &PathBuf {
&self.credential_path
}
pub fn maybe_copy_cookie_store(&self) -> anyhow::Result<Option<CookieStore>> {
Ok(self.snapshot()?.map(|credential| credential.cookie.cookies))
}
pub fn maybe_account_info(&self) -> anyhow::Result<Option<MidjourneyAccountInfo>> {
Ok(self.snapshot()?.and_then(|credential| credential.maybe_user_info))
}
pub fn session_appears_active(&self) -> anyhow::Result<bool> {
Ok(self.snapshot()?.is_some_and(|credential| credential.cookie.maybe_relogin_required_since.is_none() && (cookie_store_has_auth_cookies(&credential.cookie.cookies) || !token_needs_refresh(credential.cookie.cookies.get_cookie_value(AUTH_COOKIE_NAME_I), Utc::now().timestamp()))))
}
pub async fn save_login(&self, cookies: CookieStore) -> anyhow::Result<()> {
let _guard = self.mutation.lock().await;
let mut credential = MidjourneyCredential::new(cookies);
populate_account_info(&mut credential);
self.store(credential)
}
pub async fn clear_credentials(&self) -> anyhow::Result<()> {
let _guard = self.mutation.lock().await;
// Remove every format, so a later startup cannot migrate a logged-out account.
for path in self.legacy_paths.iter().chain([&self.credential_path]) {
match tokio::fs::remove_file(path).await {
Ok(()) => {},
Err(err) if err.kind() == std::io::ErrorKind::NotFound => {},
Err(err) => return Err(err.into()),
}
}
*self.credential.write().map_err(|_| anyhow!("Midjourney credential lock poisoned"))? = None;
Ok(())
}
/// Resolve identity and renew expiring Firebase credentials before sending
/// generation or polling requests. The refresh token can outlive the ID JWT.
pub async fn session(&self) -> anyhow::Result<Option<MidjourneySession>> {
let _guard = self.mutation.lock().await;
let Some(mut credential) = self.snapshot()? else {
return Ok(None);
};
if credential.cookie.maybe_relogin_required_since.is_some() {
return Ok(None);
}
if token_needs_refresh(credential.cookie.cookies.get_cookie_value(AUTH_COOKIE_NAME_I), Utc::now().timestamp()) {
let Some(refresh) = credential.cookie.cookies.get_cookie_value(AUTH_COOKIE_NAME_R).map(str::to_owned) else {
self.mark_relogin(&mut credential)?;
return Ok(None);
};
let refreshed = tokio::time::timeout(NETWORK_TIMEOUT, refresh_token(RefreshTokenArgs { request: RefreshTokenRequest { refresh_token: &refresh, api_key: DEFAULT_FIREBASE_API_KEY }, browser: Some(credential.browser()) })).await?;
match refreshed {
Ok(response) => {
let origin = Url::parse(MIDJOURNEY_ORIGIN)?;
// Preserve the host-only Secure scope required by __Host- cookies.
for (name, value) in [(AUTH_COOKIE_NAME_I, response.id_token), (AUTH_COOKIE_NAME_R, response.refresh_token)] {
credential.cookie.cookies.apply_set_cookie_header(&format!("{name}={value}; Path=/; Secure; HttpOnly"), &origin);
}
credential.cookie.maybe_updated_at = Some(Utc::now());
credential.cookie.maybe_succeeded_at = Some(Utc::now());
populate_account_info(&mut credential);
self.store(credential.clone())?;
},
Err(err) if is_revoked_refresh(&err) => {
self.mark_relogin(&mut credential)?;
return Ok(None);
},
Err(err) => {
credential.cookie.maybe_failed_at = Some(Utc::now());
self.store(credential)?;
return Err(err.into());
},
}
}
populate_account_info(&mut credential);
let cookie_header = credential.cookie.cookies.cookie_header_for_url(&Url::parse(MIDJOURNEY_ORIGIN)?).unwrap_or_default();
let browser = credential.browser();
let mut maybe_user_id = extract_midjourney_user_id_from_cookie_header(&cookie_header).or_else(|| credential.maybe_user_info.as_ref()?.maybe_user_id.as_deref().map(MidjourneyUserId::from_str));
if maybe_user_id.is_none() {
let info = tokio::time::timeout(NETWORK_TIMEOUT, get_user_info(GetUserInfoArgs { cookie_header: &cookie_header, hostname: None, browser: Some(browser.clone()) })).await??;
maybe_user_id = info.user_id;
credential.maybe_user_info = Some(MidjourneyAccountInfo { maybe_user_id: maybe_user_id.as_ref().map(|id| id.as_str().to_string()), maybe_email: info.email, maybe_username: None });
self.store(credential.clone())?;
}
let Some(user_id) = maybe_user_id else {
bail!("Could not resolve the Midjourney account; please log in again")
};
Ok(Some(MidjourneySession { credential_id: credential.id, cookie_header, user_id, browser }))
}
fn snapshot(&self) -> anyhow::Result<Option<MidjourneyCredential>> {
Ok(self.credential.read().map_err(|_| anyhow!("Midjourney credential lock poisoned"))?.clone())
}
fn store(&self, credential: MidjourneyCredential) -> anyhow::Result<()> {
credential.save(&self.credential_path)?;
*self.credential.write().map_err(|_| anyhow!("Midjourney credential lock poisoned"))? = Some(credential);
Ok(())
}
fn mark_relogin(&self, credential: &mut MidjourneyCredential) -> anyhow::Result<()> {
let now = Utc::now();
credential.cookie.maybe_failed_at = Some(now);
credential.cookie.maybe_relogin_required_since.get_or_insert(now);
self.store(credential.clone())
}
}
/// Fills the account's id and email from the auth token's claims.
pub fn populate_account_info(credential: &mut MidjourneyCredential) {
let Some(claims) = credential.cookie.cookies.get_cookie_value(AUTH_COOKIE_NAME_I).and_then(jwt_claims) else {
return;
};
let info = credential.maybe_user_info.get_or_insert_with(MidjourneyAccountInfo::default);
if let Some(id) = claims.get("midjourney_id").and_then(Value::as_str).filter(|id| !id.is_empty()) {
info.maybe_user_id = Some(id.to_string());
}
if let Some(email) = claims.get("email").and_then(Value::as_str) {
info.maybe_email = Some(email.to_string());
}
}
fn is_revoked_refresh(error: &MidjourneyError) -> bool {
let MidjourneyError::Api(MidjourneyApiError::InvalidRequest(body) | MidjourneyApiError::Unauthorized(body)) = error else {
return false;
};
["TOKEN_EXPIRED", "INVALID_REFRESH_TOKEN", "USER_DISABLED", "USER_NOT_FOUND"].iter().any(|code| body.contains(code))
}
#[cfg(test)]
mod tests {
use base64::Engine;
use base64::prelude::BASE64_URL_SAFE_NO_PAD;
use tempfile::TempDir;
use super::*;
const TEST_USER_ID: &str = "00000000-0000-4000-8000-000000000001";
mod persistence {
use super::*;
#[tokio::test]
async fn expired_session_without_refresh_requires_login_and_new_login_clears_it() {
let (_dir, manager) = manager();
manager.save_login(cookies(10)).await.unwrap();
assert!(manager.session().await.unwrap().is_none());
assert!(!manager.session_appears_active().unwrap());
let dead = MidjourneyCredential::load(manager.credential_path()).unwrap();
assert!(dead.cookie.maybe_relogin_required_since.is_some());
manager.save_login(cookies(Utc::now().timestamp() + 3600)).await.unwrap();
assert!(manager.session().await.unwrap().is_some());
assert!(manager.session_appears_active().unwrap());
assert!(manager.snapshot().unwrap().unwrap().cookie.maybe_relogin_required_since.is_none());
}
#[tokio::test]
async fn loads_a_saved_login_and_logout_removes_every_format() {
let (dir, manager) = manager();
manager.save_login(cookies(Utc::now().timestamp() + 3600)).await.unwrap();
let legacy = dir.path().join("midjourney_state.json");
std::fs::write(&legacy, "{}").unwrap();
let reloaded = MidjourneyCredentialManager::load(manager.credential_path().clone(), vec![legacy.clone()]).unwrap();
let session = reloaded.session().await.unwrap().unwrap();
assert_eq!(session.user_id.as_str(), TEST_USER_ID);
reloaded.clear_credentials().await.unwrap();
assert!(!legacy.exists());
assert!(!manager.credential_path().exists());
assert!(reloaded.session().await.unwrap().is_none());
}
#[test]
fn cookie_attributes_and_user_agent_survive_toml_round_trip() {
let (_dir, manager) = manager();
let origin = Url::parse(MIDJOURNEY_ORIGIN).unwrap();
let mut cookies = CookieStore::empty();
cookies.apply_set_cookie_header("session=host; Path=/; Secure; HttpOnly; SameSite=Lax", &origin);
cookies.apply_set_cookie_header("session=domain; Domain=midjourney.com; Path=/api; Secure", &origin);
let credential = MidjourneyCredential::new(cookies);
let path = manager.credential_path();
credential.save(path).unwrap();
let loaded = MidjourneyCredential::load(path).unwrap();
assert_eq!(loaded.cookie.cookies, credential.cookie.cookies);
assert_eq!(loaded.browser(), credential.browser());
assert_eq!(loaded.cookie.cookies.cookie_header_for_url(&origin).as_deref(), Some("session=host"));
assert!(loaded.cookie.cookies.cookie_header_for_url(&Url::parse("http://www.midjourney.com/").unwrap()).is_none());
assert_eq!(loaded.cookie.cookies.cookie_header_for_url(&Url::parse("https://midjourney.com/api/test").unwrap()).as_deref(), Some("session=domain"));
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
assert_eq!(std::fs::metadata(path).unwrap().permissions().mode() & 0o777, 0o600);
}
}
}
#[test]
fn temporary_refresh_failures_do_not_force_logout() {
assert!(!is_revoked_refresh(&MidjourneyApiError::NetworkError("offline".into()).into()));
assert!(!is_revoked_refresh(&MidjourneyApiError::InvalidRequest("API_KEY_INVALID".into()).into()));
assert!(is_revoked_refresh(&MidjourneyApiError::InvalidRequest("INVALID_REFRESH_TOKEN".into()).into()));
}
fn manager() -> (TempDir, MidjourneyCredentialManager) {
let dir = tempfile::tempdir().unwrap();
let manager = MidjourneyCredentialManager::new(dir.path().join("midjourney_cookies.toml"), Vec::new(), None);
(dir, manager)
}
fn cookies(exp: i64) -> CookieStore {
CookieStore::from_cookie_header(&format!("{AUTH_COOKIE_NAME_I}={}", token(exp)), &Url::parse(MIDJOURNEY_ORIGIN).unwrap())
}
fn token(exp: i64) -> String {
let claims = serde_json::json!({"exp": exp, "midjourney_id": TEST_USER_ID});
format!("header.{}.signature", BASE64_URL_SAFE_NO_PAD.encode(claims.to_string()))
}
}
@@ -1,4 +1,7 @@
pub mod auth_token;
pub mod cookie_names;
pub mod cookie_store_has_auth_cookies;
pub mod credential_file;
pub mod credential_manager;
pub mod jwt_decode;
pub mod midjourney_user_id;
@@ -15,7 +15,7 @@ pub const MIDJOURNEY_USER_AGENT: &str = "Mozilla/5.0 (Macintosh; Intel Mac OS X
/// traffic (submit, imagine, index page, image downloads, websocket upgrade).
///
/// Safari fingerprint + the exact [`MIDJOURNEY_USER_AGENT`], to stay consistent
/// with the macOS WKWebView login window that captured the cookies.
/// with the login webview that captured the cookies.
pub fn midjourney_browser_profile() -> BrowserProfile {
BrowserProfile::safari_macos_with_user_agent(MIDJOURNEY_USER_AGENT)
}
@@ -1,2 +1,3 @@
pub mod get_image_url;
pub mod image_downloader_client;
pub mod image_downloader_client;
pub mod midjourney_browser_profile;
@@ -21,10 +21,6 @@ impl TaskId {
pub fn as_str(&self) -> &str {
&self.0
}
pub fn to_string(&self) -> String {
self.0.to_string()
}
}
impl Display for TaskId {
@@ -11,7 +11,7 @@ use errors::AnyhowResult;
use log::{debug, error, warn};
use serde_derive::{Deserialize, Serialize};
use std::fmt::{Debug, Display};
use std::{fs::File, io::Write, path::Path};
use std::path::Path;
use url::Url;
use wreq::Client;
@@ -271,9 +271,7 @@ pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &
pub async fn save_generations_to_dir(generations: &[Generation], dir: &str) -> AnyhowResult<()> {
let dir = Path::new(dir);
if !dir.exists() {
std::fs::create_dir_all(dir)?;
}
tokio::fs::create_dir_all(dir).await?;
for generation in generations {
let response = wreq::get(generation.url.clone()).send().await?;
let image_bytes = response.bytes().await?;
@@ -282,15 +280,13 @@ pub async fn save_generations_to_dir(generations: &[Generation], dir: &str) -> A
let ext = url.path().split(".").last().unwrap_or("png");
let path = Path::new(dir).join(format!("{}.{}", generation.id, ext));
let mut file = File::create(path)?;
file.write_all(&image_bytes)?;
tokio::fs::write(path, &image_bytes).await?;
let thumbnail_url = generation.encodings.thumbnail.path.clone();
let thumbnail_response = wreq::get(thumbnail_url).send().await?;
let thumbnail_bytes = thumbnail_response.bytes().await?;
let thumbnail_path = Path::new(dir).join(format!("{}_thumbnail.{}", generation.id, ext));
let mut thumbnail_file = File::create(thumbnail_path)?;
thumbnail_file.write_all(&thumbnail_bytes)?;
tokio::fs::write(thumbnail_path, &thumbnail_bytes).await?;
}
Ok(())
@@ -141,8 +141,8 @@ mod tests {
async fn test_requests() -> AnyhowResult<()> {
setup_test_logging(LevelFilter::Debug);
let refresh_token = "AMf-vByUgA0J9S93vULECe-sD50cbR85AFVYF60gKAmLFF1MBYwXlKRJCsv7z3oSpLtP0ApyOU5fAl52qHWM2U0yKgEu5gQ5UCUDt5SEG4UBRHWvHEmLLcu-Zl04Fq4ljocSEO3qJzuX1wyshmptZxlDPRkjdLYgyn-Kp03woI2yRPKLuQF5hQtABHabsNUh9Zhs129sgxuUlOuw4zsFI4L8UdE3bEuF3k6Mic7KlE3440YwyQ8Qtmk9zHtXIx2ob56N-1WTGSqwMUuQ1mX4OOvLve9bw2zDW3UzchomTWkh732886RPrP-DNRkPbO3FejUEMxuYQFWYX3EEeIEdfNGXFNGPL1dNniSXlSaKadpX699Ishjviv-x9o-pPQa3zoAj3NzWLyd8mn5rDBh9qAO_iDETaeo01Sjhb4mGYfAWZ-g4S6nW778";
let refresh_token = WorldLabsRefreshToken::new(refresh_token.to_string());
let refresh_token = std::env::var("WORLDLABS_REFRESH_TOKEN")?;
let refresh_token = WorldLabsRefreshToken::new(refresh_token);
let result = google_refresh_token(GoogleRefreshTokenArgs {
refresh_token: &refresh_token,
+1
View File
@@ -22,6 +22,7 @@ crate-type = ["staticlib", "cdylib", "rlib"]
tauri-build = { version = "2.6.2", features = [] }
[dependencies]
craft-update = { git = "https://github.com/storytold/craft-libs", rev = "2a24c10817173d26d0ea4689acbbade36f91b5dc" }
toml.workspace = true
browser_emulation.workspace = true
cookie_store_wrapper = { workspace = true, features = ["wreq"] }
@@ -1,2 +1 @@
pub mod router_image_request_to_artcraft_prompt;
pub mod router_video_request_to_artcraft_prompt;
@@ -19,7 +19,7 @@ use crate::core::api_adapters::models::image::tauri_image_model_to_generation_mo
use crate::core::api_adapters::models::image::tauri_image_model_to_router_model::tauri_image_model_to_router_model;
use crate::core::commands::enqueue::generate_error::GenerateError;
use crate::core::commands::enqueue::task_enqueue_success::TaskEnqueueSuccess;
use crate::core::commands::generate::common::router_image_request_to_artcraft_prompt::router_image_request_to_artcraft_prompt;
use artcraft_router::generate::generate_image::router_image_request_to_artcraft_prompt::router_image_request_to_artcraft_prompt;
use crate::core::commands::generate::generate_image::providers::artcraft_router::utils::convert_enums_to_router::{convert_aspect_ratio, convert_quality, convert_resolution};
use crate::core::commands::generate::generate_image::providers::artcraft_router::utils::map_media_files_to_urls::map_media_file_tokens_to_cdn_urls;
use crate::core::commands::generate::generate_image::tauri_generate_image_request::TauriGenerateImageRequest;
@@ -2,7 +2,7 @@ use crate::core::api_adapters::models::image::tauri_image_model_to_generation_mo
use crate::core::api_adapters::models::image::tauri_image_model_to_router_model::tauri_image_model_to_router_model;
use crate::core::commands::enqueue::generate_error::{BadInputReason, GenerateError};
use crate::core::commands::enqueue::task_enqueue_success::TaskEnqueueSuccess;
use crate::core::commands::generate::common::router_image_request_to_artcraft_prompt::router_image_request_to_artcraft_prompt;
use artcraft_router::generate::generate_image::router_image_request_to_artcraft_prompt::router_image_request_to_artcraft_prompt;
use crate::core::commands::generate::generate_image::providers::artcraft_router::utils::convert_enums_to_router::convert_aspect_ratio;
use crate::core::commands::generate::generate_image::tauri_generate_image_request::TauriGenerateImageRequest;
use crate::core::commands::generate::generate_image::tauri_image_model::TauriImageModel;
@@ -12,4 +12,5 @@ pub mod platform_info_command;
pub mod providers;
pub mod response;
pub mod task_queue;
pub mod deprecated;
pub mod deprecated;
pub mod updates;
@@ -0,0 +1,8 @@
use crate::updates::{UpdateInfo, Updates};
use std::sync::Arc;
use tauri::{AppHandle, State};
#[tauri::command]
pub async fn check_for_updates_command(app: AppHandle, updates: State<'_, Arc<Updates>>) -> Result<UpdateInfo, String> {
updates.inner().run("The update check stopped unexpectedly", move |state| state.check(&app)).await
}
@@ -0,0 +1,9 @@
use crate::updates::{require_confirmation, Updates};
use std::sync::Arc;
use tauri::{AppHandle, State};
#[tauri::command]
pub async fn install_update_command(app: AppHandle, updates: State<'_, Arc<Updates>>, confirmed: bool) -> Result<(), String> {
require_confirmation(confirmed)?;
updates.inner().run("The update installer stopped unexpectedly", move |state| state.install(&app)).await
}
@@ -0,0 +1,2 @@
pub mod check_for_updates_command;
pub mod install_update_command;
@@ -31,8 +31,6 @@ use sqlite_tasks::queries::task::Task;
use sqlite_tasks::queries::update_successful_task_status_with_metadata::{
update_successful_task_status_with_metadata, UpdateSuccessfulTaskArgs,
};
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use std::time::Duration;
use tauri::AppHandle;
@@ -255,8 +253,7 @@ async fn download_file(
let filename = format!("fal_{}_{}.{}", generate_random_uuid(), index, extension);
let download_path = tempdir.join(filename);
let mut file = File::create(&download_path)?;
file.write_all(&bytes)?;
tokio::fs::write(&download_path, bytes).await?;
Ok(download_path)
}
@@ -94,7 +94,7 @@ async fn download_generated_results(
}
ensure!(!urls.is_empty(), "Completed task has no downloadable results");
let path = checkpoint_path(root, task);
let mut checkpoint: AutoDownloadCheckpoint = match std::fs::read(&path) {
let mut checkpoint: AutoDownloadCheckpoint = match tokio::fs::read(&path).await {
Ok(bytes) => serde_json::from_slice(&bytes)?,
Err(err) if err.kind() == std::io::ErrorKind::NotFound => {
AutoDownloadCheckpoint { download_time: Local::now(), saved: BTreeMap::new() }
@@ -1,8 +1,6 @@
use errors::AnyhowResult;
use reqwest::Client;
use reqwest::Url;
use std::fs::OpenOptions;
use std::io::Write;
use std::path::Path;
const USER_AGENT: &str = "storyteller-client/1.0";
@@ -22,13 +20,7 @@ pub async fn simple_http_download<P: AsRef<Path>>(url: &Url, download_path: P) -
//let response : Response = reqwest::get(url).await?;
let bytes = response.bytes().await?;
let mut file = OpenOptions::new()
.create(true) // To create a new file
.write(true)
.truncate(true)
.open(download_path)?;
file.write_all(&bytes)?;
tokio::fs::write(download_path, bytes).await?;
Ok(())
}
+26 -2
View File
@@ -6,6 +6,11 @@ use crate::core::commands::generate::generate_mesh_command::generate_mesh_comman
pub mod core;
pub mod services;
pub mod version;
pub mod updates;
use std::sync::Arc;
use crate::core::commands::updates::check_for_updates_command::check_for_updates_command;
use crate::core::commands::updates::install_update_command::install_update_command;
use tauri::Manager;
@@ -155,11 +160,20 @@ pub fn run() {
println!("Initializing backend runtime...");
let builder = tauri::Builder::default()
.manage(Arc::new(updates::Updates::default()))
.on_window_event(|window, event| {
if window.try_state::<Arc<updates::Updates>>().is_some_and(|updates| updates.prevents_exit()) {
if let tauri::WindowEvent::CloseRequested { api, .. } = event {
api.prevent_close();
}
}
})
.plugin(tauri_plugin_dialog::init())
.plugin(tauri_plugin_http::init())
.plugin(tauri_plugin_opener::init())
.plugin(tauri_plugin_upload::init())
.setup(move |app| {
updates::show_previous_outcome(app.handle());
// TODO(bt): This is broken on windows
// log_environment_details();
@@ -225,6 +239,8 @@ pub fn run() {
// TODO: Break this out into another module, because RustRover/IntelliJ lags with these macros.
// My first attempt at naively doing this didn't work because the macros can't find their codegen'd targets.
let builder = builder.invoke_handler(tauri::generate_handler![
check_for_updates_command,
install_update_command,
check_sora_session_command,
download_directory_reveal_command,
download_media_file_command,
@@ -290,6 +306,14 @@ pub fn run() {
worldlabs_receive_bearer_command,
]);
builder.run(tauri::generate_context!("tauri.conf.json"))
.expect("error while running tauri application");
let app = builder.build(tauri::generate_context!("tauri.conf.json"))
.expect("error while building tauri application");
app.run(|handle, event| {
let updates = handle.state::<Arc<updates::Updates>>();
match event {
tauri::RunEvent::ExitRequested { api, .. } if updates.prevents_exit() => api.prevent_exit(),
tauri::RunEvent::Exit => updates.finish(),
_ => {}
}
});
}
+5
View File
@@ -2,5 +2,10 @@
#![cfg_attr(not(debug_assertions), windows_subsystem = "windows")]
fn main() {
if let Some(result) = artcraft_app_lib::updates::finish_helper() {
let failed = result.is_err();
if let Err(error) = result { log::error!("Update failed: {error}"); }
std::process::exit(i32::from(failed));
}
artcraft_app_lib::run();
}
@@ -1,337 +1,126 @@
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::midjourney::state::midjourney_credential::{MidjourneyAccountInfo, MidjourneyCredential};
use crate::services::midjourney::state::midjourney_user_info::MidjourneyUserInfo;
use crate::services::midjourney::state::serializable_midjourney_state::SerializableMidjourneyState;
use crate::services::midjourney::utils::extract_midjourney_user_id_from_cookies::extract_midjourney_user_id_from_cookie_header;
use anyhow::{anyhow, bail};
use base64::prelude::{Engine, BASE64_URL_SAFE_NO_PAD};
use browser_emulation::browser_profile::BrowserProfile;
use chrono::Utc;
//! The app's Midjourney login: the shared manager (`midjourney_client`, also used by the native
//! app) kept in the app's credentials directory, plus the one-time migration from the legacy
//! `midjourney_state.json`.
use std::ops::Deref;
use cookie_store_wrapper::cookie_store::CookieStore;
use midjourney_client::credentials::cookie_store_has_auth_cookies::cookie_store_has_auth_cookies;
use midjourney_client::credentials::credential_file::{MidjourneyAccountInfo, MidjourneyCredential};
use midjourney_client::credentials::credential_manager::{MIDJOURNEY_ORIGIN, MidjourneyCredentialManager as SharedCredentialManager, populate_account_info};
use midjourney_client::credentials::midjourney_user_id::MidjourneyUserId;
use midjourney_client::endpoints::refresh_token::{refresh_token, RefreshTokenArgs, RefreshTokenRequest, DEFAULT_FIREBASE_API_KEY};
use midjourney_client::error::midjourney_api_error::MidjourneyApiError;
use midjourney_client::error::midjourney_error::MidjourneyError;
use midjourney_client::recipes::get_user_info::{get_user_info, GetUserInfoArgs};
use serde_json::Value;
use std::sync::{Arc, RwLock};
use std::time::Duration;
use tokio::sync::Mutex;
use url::Url;
const AUTH_ID: &str = "__Host-Midjourney.AuthUserTokenV3_i";
const AUTH_REFRESH: &str = "__Host-Midjourney.AuthUserTokenV3_r";
const MIDJOURNEY_ORIGIN: &str = "https://www.midjourney.com/";
pub use midjourney_client::credentials::credential_manager::MidjourneySession;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::midjourney::state::midjourney_user_info::MidjourneyUserInfo;
use crate::services::midjourney::state::serializable_midjourney_state::SerializableMidjourneyState;
#[derive(Clone)]
pub struct MidjourneyCredentialManager {
credential: Arc<RwLock<Option<MidjourneyCredential>>>,
// Serialize refresh, login, and logout so an old refresh cannot resurrect a
// signed-out session or overwrite credentials captured by a new login.
mutation: Arc<Mutex<()>>,
app_data_root: AppDataRoot,
}
pub struct MidjourneyCredentialManager(SharedCredentialManager);
pub struct MidjourneySession {
pub credential_id: String,
pub cookie_header: String,
pub user_id: MidjourneyUserId,
pub browser: BrowserProfile,
impl Deref for MidjourneyCredentialManager {
type Target = SharedCredentialManager;
fn deref(&self) -> &Self::Target {
&self.0
}
}
impl MidjourneyCredentialManager {
pub fn initialize_empty(app_data_root: &AppDataRoot) -> Self {
Self { credential: Arc::new(RwLock::new(None)), mutation: Arc::new(Mutex::new(())), app_data_root: app_data_root.clone() }
let dir = app_data_root.credentials_dir();
Self(SharedCredentialManager::new(dir.get_midjourney_credential_path(), vec![dir.get_midjourney_state_path()], None))
}
pub fn initialize_from_disk_infallible(app_data_root: &AppDataRoot) -> Self {
let manager = Self::initialize_empty(app_data_root);
match load_or_migrate(app_data_root) {
Ok(credential) => {
if let Ok(mut current) = manager.credential.write() {
*current = credential;
}
Ok(manager) => manager,
Err(err) => {
log::warn!("Could not load Midjourney credentials: {}", err);
Self::initialize_empty(app_data_root)
},
Err(err) => log::warn!("Could not load Midjourney credentials: {}", err),
}
manager
}
pub fn maybe_copy_cookie_store(&self) -> anyhow::Result<Option<CookieStore>> {
Ok(self.snapshot()?.map(|credential| credential.cookie.cookies))
}
pub fn maybe_copy_user_info(&self) -> anyhow::Result<Option<MidjourneyUserInfo>> {
Ok(self.snapshot()?.and_then(|credential| credential.maybe_user_info).map(|info| MidjourneyUserInfo { user_id: info.maybe_user_id.map(MidjourneyUserId::from_string), email: info.maybe_email, websocket_token: None }))
}
pub fn session_appears_active(&self) -> anyhow::Result<bool> {
Ok(self.snapshot()?.is_some_and(|credential| credential.cookie.maybe_relogin_required_since.is_none() && (cookie_store_has_auth_cookies(&credential.cookie.cookies) || !token_needs_refresh(credential.cookie.cookies.get_cookie_value(AUTH_ID), Utc::now().timestamp()))))
}
pub async fn save_login(&self, cookies: CookieStore) -> anyhow::Result<()> {
let _guard = self.mutation.lock().await;
let mut credential = MidjourneyCredential::new(cookies);
populate_account_info(&mut credential);
self.store(credential)
}
pub async fn clear_credentials(&self) -> anyhow::Result<()> {
let _guard = self.mutation.lock().await;
// Remove both formats, so a later startup cannot migrate a logged-out account.
for path in [self.app_data_root.credentials_dir().get_midjourney_state_path(), self.credential_path()] {
match std::fs::remove_file(path) {
Ok(()) => {},
Err(err) if err.kind() == std::io::ErrorKind::NotFound => {},
Err(err) => return Err(err.into()),
}
}
*self.credential.write().map_err(|_| anyhow!("Midjourney credential lock poisoned"))? = None;
Ok(())
}
/// Resolve identity and renew expiring Firebase credentials before sending
/// generation or polling requests. The refresh token can outlive the ID JWT.
pub async fn session(&self) -> anyhow::Result<Option<MidjourneySession>> {
let _guard = self.mutation.lock().await;
let Some(mut credential) = self.snapshot()? else {
return Ok(None);
};
if credential.cookie.maybe_relogin_required_since.is_some() {
return Ok(None);
}
if token_needs_refresh(credential.cookie.cookies.get_cookie_value(AUTH_ID), Utc::now().timestamp()) {
let Some(refresh) = credential.cookie.cookies.get_cookie_value(AUTH_REFRESH).map(str::to_owned) else {
self.mark_relogin(&mut credential)?;
return Ok(None);
};
let refreshed = tokio::time::timeout(Duration::from_secs(30), refresh_token(RefreshTokenArgs { request: RefreshTokenRequest { refresh_token: &refresh, api_key: DEFAULT_FIREBASE_API_KEY }, browser: Some(credential.browser()) })).await?;
match refreshed {
Ok(response) => {
let origin = Url::parse(MIDJOURNEY_ORIGIN)?;
// Preserve the host-only Secure scope required by __Host- cookies.
for (name, value) in [(AUTH_ID, response.id_token), (AUTH_REFRESH, response.refresh_token)] {
credential.cookie.cookies.apply_set_cookie_header(&format!("{name}={value}; Path=/; Secure; HttpOnly"), &origin);
}
credential.cookie.maybe_updated_at = Some(Utc::now());
credential.cookie.maybe_succeeded_at = Some(Utc::now());
populate_account_info(&mut credential);
self.store(credential.clone())?;
},
Err(err) if is_revoked_refresh(&err) => {
self.mark_relogin(&mut credential)?;
return Ok(None);
},
Err(err) => {
credential.cookie.maybe_failed_at = Some(Utc::now());
self.store(credential)?;
return Err(err.into());
},
}
}
populate_account_info(&mut credential);
let cookie_header = credential.cookie.cookies.cookie_header_for_url(&Url::parse(MIDJOURNEY_ORIGIN)?).unwrap_or_default();
let browser = credential.browser();
let mut maybe_user_id = extract_midjourney_user_id_from_cookie_header(&cookie_header).or_else(|| credential.maybe_user_info.as_ref()?.maybe_user_id.as_deref().map(MidjourneyUserId::from_str));
if maybe_user_id.is_none() {
let info = tokio::time::timeout(Duration::from_secs(30), get_user_info(GetUserInfoArgs { cookie_header: &cookie_header, hostname: None, browser: Some(browser.clone()) })).await??;
maybe_user_id = info.user_id;
credential.maybe_user_info = Some(MidjourneyAccountInfo { maybe_user_id: maybe_user_id.as_ref().map(|id| id.as_str().to_string()), maybe_email: info.email, maybe_username: None });
self.store(credential.clone())?;
}
let Some(user_id) = maybe_user_id else {
bail!("Could not resolve the Midjourney account; please log in again")
};
Ok(Some(MidjourneySession { credential_id: credential.id, cookie_header, user_id, browser }))
}
fn snapshot(&self) -> anyhow::Result<Option<MidjourneyCredential>> {
Ok(self.credential.read().map_err(|_| anyhow!("Midjourney credential lock poisoned"))?.clone())
}
fn store(&self, credential: MidjourneyCredential) -> anyhow::Result<()> {
credential.save(&self.credential_path())?;
*self.credential.write().map_err(|_| anyhow!("Midjourney credential lock poisoned"))? = Some(credential);
Ok(())
}
fn mark_relogin(&self, credential: &mut MidjourneyCredential) -> anyhow::Result<()> {
let now = Utc::now();
credential.cookie.maybe_failed_at = Some(now);
credential.cookie.maybe_relogin_required_since.get_or_insert(now);
self.store(credential.clone())
}
fn credential_path(&self) -> std::path::PathBuf {
self.app_data_root.credentials_dir().get_midjourney_credential_path()
Ok(self.maybe_account_info()?.map(|info| MidjourneyUserInfo { user_id: info.maybe_user_id.map(MidjourneyUserId::from_string), email: info.maybe_email, websocket_token: None }))
}
}
fn load_or_migrate(root: &AppDataRoot) -> anyhow::Result<Option<MidjourneyCredential>> {
let path = root.credentials_dir().get_midjourney_credential_path();
if path.exists() {
let credential = MidjourneyCredential::load(&path)?;
credential.save(&path)?; // Persist a stable ID for hand-written files.
return Ok(Some(credential));
fn load_or_migrate(root: &AppDataRoot) -> anyhow::Result<MidjourneyCredentialManager> {
let dir = root.credentials_dir();
let (path, legacy_path) = (dir.get_midjourney_credential_path(), dir.get_midjourney_state_path());
if path.exists() || !legacy_path.exists() {
return Ok(MidjourneyCredentialManager(SharedCredentialManager::load(path, vec![legacy_path])?));
}
let legacy_path = root.credentials_dir().get_midjourney_state_path();
if !legacy_path.exists() {
return Ok(None);
}
let legacy: SerializableMidjourneyState = serde_json::from_str(&std::fs::read_to_string(legacy_path)?)?;
let legacy: SerializableMidjourneyState = serde_json::from_str(&std::fs::read_to_string(&legacy_path)?)?;
let Some(cookies) = legacy.user_cookies else {
return Ok(None);
return Ok(MidjourneyCredentialManager::initialize_empty(root));
};
let cookies = CookieStore::from_cookie_header(&cookies.to_cookie_store().to_cookie_string(), &Url::parse(MIDJOURNEY_ORIGIN)?);
let mut credential = MidjourneyCredential::new(cookies);
credential.maybe_user_info = legacy.user_info.map(|info| MidjourneyAccountInfo { maybe_user_id: info.user_id, maybe_email: info.email, maybe_username: None });
populate_account_info(&mut credential);
credential.save(&path)?;
Ok(Some(credential))
}
fn populate_account_info(credential: &mut MidjourneyCredential) {
let Some(claims) = credential.cookie.cookies.get_cookie_value(AUTH_ID).and_then(jwt_claims) else {
return;
};
let info = credential.maybe_user_info.get_or_insert_with(MidjourneyAccountInfo::default);
if let Some(id) = claims.get("midjourney_id").and_then(Value::as_str).filter(|id| !id.is_empty()) {
info.maybe_user_id = Some(id.to_string());
}
if let Some(email) = claims.get("email").and_then(Value::as_str) {
info.maybe_email = Some(email.to_string());
}
}
fn token_needs_refresh(maybe_token: Option<&str>, now: i64) -> bool {
maybe_token.and_then(jwt_claims).and_then(|claims| claims.get("exp")?.as_i64()).is_none_or(|expires| expires <= now + 120)
}
fn jwt_claims(token: &str) -> Option<Value> {
let payload = token.split('.').nth(1)?;
serde_json::from_slice(&BASE64_URL_SAFE_NO_PAD.decode(payload).ok()?).ok()
}
fn is_revoked_refresh(error: &MidjourneyError) -> bool {
let MidjourneyError::Api(MidjourneyApiError::InvalidRequest(body) | MidjourneyApiError::Unauthorized(body)) = error else {
return false;
};
["TOKEN_EXPIRED", "INVALID_REFRESH_TOKEN", "USER_DISABLED", "USER_NOT_FOUND"].iter().any(|code| body.contains(code))
Ok(MidjourneyCredentialManager(SharedCredentialManager::new(path, vec![legacy_path], Some(credential))))
}
#[cfg(test)]
mod tests {
use super::*;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use base64::prelude::{BASE64_URL_SAFE_NO_PAD, Engine};
use chrono::Utc;
use tempfile::TempDir;
use super::*;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
const TEST_USER_ID: &str = "00000000-0000-4000-8000-000000000001";
const AUTH_ID: &str = "__Host-Midjourney.AuthUserTokenV3_i";
const AUTH_REFRESH: &str = "__Host-Midjourney.AuthUserTokenV3_r";
mod persistence {
use super::*;
#[tokio::test]
async fn migrates_legacy_cookies_and_logout_does_not_touch_artcraft() {
let (_dir, root) = root();
let legacy = root.credentials_dir().get_midjourney_state_path();
let artcraft = root.credentials_dir().get_storyteller_session_cookie_file_path();
std::fs::write(&artcraft, "existing-artcraft-session").unwrap();
std::fs::write(
&legacy,
serde_json::json!({
"version": 1,
"user_cookies": {"cookies": [
{"name": AUTH_ID, "value": token(Utc::now().timestamp() + 3600)},
{"name": AUTH_REFRESH, "value": "test-refresh"}
]},
"user_info": {"user_id": TEST_USER_ID, "email": "test@example.invalid", "websocket_token": "old-socket-token"}
})
.to_string(),
)
.unwrap();
let manager = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
let session = manager.session().await.unwrap().unwrap();
assert_eq!(session.user_id.as_str(), TEST_USER_ID);
let contents = std::fs::read_to_string(root.credentials_dir().get_midjourney_credential_path()).unwrap();
assert!(contents.contains("midjourney_cookies"));
assert!(!contents.contains("old-socket-token"));
assert!(legacy.exists(), "migration preserves the old file until logout");
let reloaded = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
assert_eq!(reloaded.session().await.unwrap().unwrap().credential_id, session.credential_id);
manager.clear_credentials().await.unwrap();
assert!(!legacy.exists());
assert!(!root.credentials_dir().get_midjourney_credential_path().exists());
assert_eq!(std::fs::read_to_string(artcraft).unwrap(), "existing-artcraft-session");
let reloaded = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
assert!(reloaded.session().await.unwrap().is_none());
}
#[tokio::test]
async fn expired_session_without_refresh_requires_login_and_new_login_clears_it() {
let (_dir, root) = root();
let manager = MidjourneyCredentialManager::initialize_empty(&root);
manager.save_login(cookies(10)).await.unwrap();
assert!(manager.session().await.unwrap().is_none());
assert!(!manager.session_appears_active().unwrap());
let dead = MidjourneyCredential::load(&root.credentials_dir().get_midjourney_credential_path()).unwrap();
assert!(dead.cookie.maybe_relogin_required_since.is_some());
manager.save_login(cookies(Utc::now().timestamp() + 3600)).await.unwrap();
assert!(manager.session().await.unwrap().is_some());
assert!(manager.session_appears_active().unwrap());
assert!(manager.snapshot().unwrap().unwrap().cookie.maybe_relogin_required_since.is_none());
}
#[test]
fn cookie_attributes_and_user_agent_survive_toml_round_trip() {
let (_dir, root) = root();
let origin = Url::parse(MIDJOURNEY_ORIGIN).unwrap();
let mut cookies = CookieStore::empty();
cookies.apply_set_cookie_header("session=host; Path=/; Secure; HttpOnly; SameSite=Lax", &origin);
cookies.apply_set_cookie_header("session=domain; Domain=midjourney.com; Path=/api; Secure", &origin);
let credential = MidjourneyCredential::new(cookies);
let path = root.credentials_dir().get_midjourney_credential_path();
credential.save(&path).unwrap();
let loaded = MidjourneyCredential::load(&path).unwrap();
assert_eq!(loaded.cookie.cookies, credential.cookie.cookies);
assert_eq!(loaded.browser(), credential.browser());
assert_eq!(loaded.cookie.cookies.cookie_header_for_url(&origin).as_deref(), Some("session=host"));
assert!(loaded.cookie.cookies.cookie_header_for_url(&Url::parse("http://www.midjourney.com/").unwrap()).is_none());
assert_eq!(loaded.cookie.cookies.cookie_header_for_url(&Url::parse("https://midjourney.com/api/test").unwrap()).as_deref(), Some("session=domain"));
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
assert_eq!(std::fs::metadata(path).unwrap().permissions().mode() & 0o777, 0o600);
}
}
#[test]
fn corrupt_modern_credential_does_not_revive_legacy_state() {
let (_dir, root) = root();
std::fs::write(root.credentials_dir().get_midjourney_credential_path(), "not valid toml").unwrap();
std::fs::write(root.credentials_dir().get_midjourney_state_path(), "{}").unwrap();
assert!(load_or_migrate(&root).is_err());
}
#[tokio::test]
async fn migrates_legacy_cookies_and_logout_does_not_touch_artcraft() {
let (_dir, root) = root();
let legacy = root.credentials_dir().get_midjourney_state_path();
let artcraft = root.credentials_dir().get_storyteller_session_cookie_file_path();
std::fs::write(&artcraft, "existing-artcraft-session").unwrap();
std::fs::write(
&legacy,
serde_json::json!({
"version": 1,
"user_cookies": {"cookies": [
{"name": AUTH_ID, "value": token(Utc::now().timestamp() + 3600)},
{"name": AUTH_REFRESH, "value": "test-refresh"}
]},
"user_info": {"user_id": TEST_USER_ID, "email": "test@example.invalid", "websocket_token": "old-socket-token"}
})
.to_string(),
)
.unwrap();
let manager = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
let session = manager.session().await.unwrap().unwrap();
assert_eq!(session.user_id.as_str(), TEST_USER_ID);
let contents = std::fs::read_to_string(root.credentials_dir().get_midjourney_credential_path()).unwrap();
assert!(contents.contains("midjourney_cookies"));
assert!(!contents.contains("old-socket-token"));
assert!(legacy.exists(), "migration preserves the old file until logout");
let reloaded = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
assert_eq!(reloaded.session().await.unwrap().unwrap().credential_id, session.credential_id);
manager.clear_credentials().await.unwrap();
assert!(!legacy.exists());
assert!(!root.credentials_dir().get_midjourney_credential_path().exists());
assert_eq!(std::fs::read_to_string(artcraft).unwrap(), "existing-artcraft-session");
let reloaded = MidjourneyCredentialManager::initialize_from_disk_infallible(&root);
assert!(reloaded.session().await.unwrap().is_none());
}
mod refresh {
use super::*;
#[test]
fn refreshes_before_expiry_but_does_not_refresh_a_fresh_token() {
assert!(!token_needs_refresh(Some(&token(5000)), 1000));
assert!(token_needs_refresh(Some(&token(1120)), 1000));
assert!(token_needs_refresh(Some(&token(999)), 1000));
assert!(token_needs_refresh(Some("invalid"), 1000));
assert!(token_needs_refresh(None, 1000));
}
#[test]
fn temporary_refresh_failures_do_not_force_logout() {
assert!(!is_revoked_refresh(&MidjourneyApiError::NetworkError("offline".into()).into()));
assert!(!is_revoked_refresh(&MidjourneyApiError::InvalidRequest("API_KEY_INVALID".into()).into()));
assert!(is_revoked_refresh(&MidjourneyApiError::InvalidRequest("INVALID_REFRESH_TOKEN".into()).into()));
}
#[test]
fn corrupt_modern_credential_does_not_revive_legacy_state() {
let (_dir, root) = root();
std::fs::write(root.credentials_dir().get_midjourney_credential_path(), "not valid toml").unwrap();
std::fs::write(root.credentials_dir().get_midjourney_state_path(), "{}").unwrap();
assert!(load_or_migrate(&root).is_err());
}
fn root() -> (TempDir, AppDataRoot) {
@@ -341,10 +130,6 @@ mod tests {
(dir, root)
}
fn cookies(exp: i64) -> CookieStore {
CookieStore::from_cookie_header(&format!("{AUTH_ID}={}", token(exp)), &Url::parse(MIDJOURNEY_ORIGIN).unwrap())
}
fn token(exp: i64) -> String {
let claims = serde_json::json!({"exp": exp, "midjourney_id": TEST_USER_ID});
format!("header.{}.signature", BASE64_URL_SAFE_NO_PAD.encode(claims.to_string()))
@@ -1,4 +1,3 @@
pub (super) mod serializable_midjourney_state;
pub mod midjourney_credential_manager;
pub mod midjourney_user_info;
pub mod midjourney_credential;
@@ -169,7 +169,7 @@ mod tests {
use super::*;
use crate::core::commands::generate::generate_image::providers::midjourney::handle_midjourney::handle_midjourney;
use crate::core::commands::generate::generate_image::tauri_generate_image_request::TauriGenerateImageRequest;
use crate::services::midjourney::state::midjourney_credential::MidjourneyCredential;
use midjourney_client::credentials::credential_file::MidjourneyCredential;
use artcraft_client::credentials::storyteller_credential_set::StorytellerCredentialSet;
use artcraft_client::endpoints::media_files::list_batch_generated_redux_media_files::list_batch_generated_redux_media_files;
use artcraft_client::utils::api_host::ApiHost;
@@ -119,7 +119,7 @@ pub(super) async fn upload_midjourney_batch(maybe_app_handle: Option<&AppHandle>
}
}
if let Err(err) = std::fs::remove_file(MidjourneyUploadCheckpoint::path(app_data_root, local_task)) {
if let Err(err) = tokio::fs::remove_file(MidjourneyUploadCheckpoint::path(app_data_root, local_task)).await {
log::warn!("Could not remove completed Midjourney upload checkpoint: {}", err);
}
Ok(())
@@ -2,8 +2,6 @@ use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use errors::AnyhowResult;
use midjourney_client::utils::get_image_url::get_image_url;
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use url::Url;
use midjourney_client::utils::image_downloader_client::ImageDownloaderClient;
@@ -28,8 +26,7 @@ pub async fn download_midjourney_image(
let download_filename = format!("{}_{}.{}", job_id, index, ext);
let download_path = tempdir.join(download_filename);
let mut file = File::create(&download_path)?;
file.write_all(&image_bytes)?;
tokio::fs::write(&download_path, &image_bytes).await?;
Ok(download_path)
}
@@ -1,62 +0,0 @@
use base64::prelude::BASE64_URL_SAFE_NO_PAD;
use base64::Engine;
use midjourney_client::credentials::midjourney_user_id::MidjourneyUserId;
use serde_json::Value;
/// The Midjourney auth id-token cookie. Its (unverified) JWT payload carries
/// the Midjourney user id under `midjourney_id`.
const AUTH_ID_TOKEN_COOKIE: &str = "__Host-Midjourney.AuthUserTokenV3_i";
/// The JWT claim holding the Midjourney user id (the `singleplayer_{...}`
/// channel id is built from it).
const MIDJOURNEY_ID_CLAIM: &str = "midjourney_id";
/// Recover the Midjourney user id from the auth cookie's JWT, WITHOUT hitting
/// the Cloudflare-gated index page. `cookie_header` is a raw `Cookie:` header
/// string (`"a=1; b=2"`).
pub fn extract_midjourney_user_id_from_cookie_header(cookie_header: &str) -> Option<MidjourneyUserId> {
let token = cookie_value(cookie_header, AUTH_ID_TOKEN_COOKIE)?;
let claims = decode_jwt_payload(token)?;
let id = claims.get(MIDJOURNEY_ID_CLAIM).and_then(Value::as_str)?;
if id.is_empty() {
return None;
}
Some(MidjourneyUserId::from_str(id))
}
fn cookie_value<'a>(cookie_header: &'a str, name: &str) -> Option<&'a str> {
cookie_header.split(';').filter_map(|pair| pair.trim().split_once('=')).find(|(cookie_name, _)| *cookie_name == name).map(|(_, value)| value)
}
/// Decode the (unverified) payload of a `header.payload.signature` JWT.
fn decode_jwt_payload(token: &str) -> Option<Value> {
let mut parts = token.split('.');
let _header = parts.next()?;
let payload = parts.next()?;
let _signature = parts.next()?;
if parts.next().is_some() {
return None;
}
let bytes = BASE64_URL_SAFE_NO_PAD.decode(payload).ok()?;
serde_json::from_slice(&bytes).ok()
}
#[cfg(test)]
mod tests {
use super::*;
// Payload: {"midjourney_id":"26c9d38d-1f71-49c7-a356-b29dac58b54c","name":"u1"}
const JWT: &str = "eyJhbGciOiJIUzI1NiJ9.eyJtaWRqb3VybmV5X2lkIjoiMjZjOWQzOGQtMWY3MS00OWM3LWEzNTYtYjI5ZGFjNThiNTRjIiwibmFtZSI6InUxIn0.sig";
#[test]
fn extracts_user_id_from_auth_cookie() {
let header = format!("_fbp=abc; {}={}; other=x", AUTH_ID_TOKEN_COOKIE, JWT);
let user_id = extract_midjourney_user_id_from_cookie_header(&header).unwrap();
assert_eq!(user_id.as_str(), "26c9d38d-1f71-49c7-a356-b29dac58b54c");
}
#[test]
fn returns_none_without_auth_cookie() {
assert!(extract_midjourney_user_id_from_cookie_header("_fbp=abc; other=x").is_none());
}
}
@@ -1,3 +1 @@
pub mod download_midjourney_image;
pub mod midjourney_browser_profile;
pub mod extract_midjourney_user_id_from_cookies;
@@ -1,4 +1,4 @@
use crate::services::midjourney::utils::midjourney_browser_profile::MIDJOURNEY_USER_AGENT;
use midjourney_client::utils::midjourney_browser_profile::MIDJOURNEY_USER_AGENT;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::clear_all_webview_cookies::clear_all_webview_cookies;
use crate::services::midjourney::state::midjourney_credential_manager::MidjourneyCredentialManager;
@@ -14,7 +14,7 @@ pub async fn initialize_sora_jwt_bearer_token(app_data_root: &AppDataRoot) -> An
return Ok(()); // Bearer token already exists.
}
let cookie = std::fs::read_to_string(app_data_root.get_sora_cookie_file_path())?
let cookie = tokio::fs::read_to_string(app_data_root.get_sora_cookie_file_path()).await?
.trim()
.to_string();
@@ -22,7 +22,7 @@ pub async fn initialize_sora_jwt_bearer_token(app_data_root: &AppDataRoot) -> An
let bearer = generate_bearer_jwt_with_cookie(&cookie).await?;
std::fs::write(app_data_root.get_sora_bearer_token_file_path(), bearer)?;
tokio::fs::write(app_data_root.get_sora_bearer_token_file_path(), bearer).await?;
Ok(())
}
@@ -8,8 +8,7 @@ use openai_sora_client::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
use openai_sora_client::creds::sora_sentinel::SoraSentinel;
use openai_sora_client::creds::sora_sentinel_token::SoraSentinelToken;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use std::fs::{read_to_string, OpenOptions};
use std::io::Write;
use tokio::fs::{read_to_string, write};
pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDataRoot) -> AnyhowResult<SoraCredentialSet> {
let cookie_file = app_data_root.get_sora_cookie_file_path();
@@ -21,7 +20,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
return Err(anyhow!("Cookie file does not exist: {:?}", cookie_file));
}
let value = read_to_string(cookie_file)?
let value = read_to_string(cookie_file).await?
.trim()
.to_string();
@@ -32,14 +31,14 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
let mut sentinel_token = None;
if bearer_file.exists() {
let value = read_to_string(&bearer_file)?
let value = read_to_string(&bearer_file).await?
.trim()
.to_string();
bearer = Some(SoraJwtBearerToken::new(value)?);
}
if legacy_sentinel_file.exists() {
let value = read_to_string(&legacy_sentinel_file)?
let value = read_to_string(&legacy_sentinel_file).await?
.trim()
.to_string();
@@ -47,7 +46,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
}
if sentinel_token_file.exists() {
let value = read_to_string(&sentinel_token_file)?
let value = read_to_string(&sentinel_token_file).await?
.trim()
.to_string();
@@ -73,14 +72,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting bearer token value...");
let value = bearer.token_str();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&bearer_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&bearer_file, value.as_bytes()).await?;
}
}
@@ -89,14 +81,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting sentinel value...");
let value = sentinel.get_sentinel();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&legacy_sentinel_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&legacy_sentinel_file, value.as_bytes()).await?;
}
}
@@ -105,14 +90,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting sentinel token...");
let value = sentinel_token.to_persistent_storage_json()?;
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&sentinel_token_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&sentinel_token_file, value.as_bytes()).await?;
}
}
@@ -162,14 +162,7 @@ impl SoraCredentialManager {
self.holder.set_credentials(&creds)?;
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(self.app_data_root.get_sora_legacy_sentinel_file_path())?;
file.write_all(sentinel.as_bytes())?;
file.flush()?;
tokio::fs::write(self.app_data_root.get_sora_legacy_sentinel_file_path(), sentinel.as_bytes()).await?;
Ok(creds)
}

Some files were not shown because too many files have changed in this diff Show More