Compare commits

..
Author SHA1 Message Date
Dylan M. Taylor 9b5e2070ac ci: harden workflows with credential isolation, stepsecurity runner, and dependabot cooldown
- Add StepSecurity Harden-Runner to Linux, macOS, and Windows release publishing workflows
- Set persist-credentials: false on all actions/checkout steps
- Pin actions/checkout, actions/setup-node, dtolnay/rust-toolchain, actions/upload-artifact, actions/download-artifact, and tauri-apps/tauri-action to immutable commit SHAs
- Add 7-day cooldown to Dependabot for cargo and github-actions ecosystems
2026-11-10 13:28:28 -05:00
FredApps 50afb36f9c api_clients: use tokio::fs inside async functions (Sonar S7493) (#2803)
## Summary

Replaces blocking `std::fs` calls inside `async fn`s with `tokio::fs` in
four API client crates. SonarCloud reports these as **rust:S7493** ("use
an asynchronous file API instead of this blocking file operation"). This
PR resolves 13 of the 40 artcraft findings.

| Crate | File | Change |

|---------------------|-------------------------------------------------------------|------------------------------------------------------------|
| `artcraft_client` | `endpoints/media_files/*_from_file.rs` (4 files) |
`std::fs::read` → `tokio::fs::read(..).await` |
| `artcraft_client` | `recipes/download_media_file.rs` | `File::create`
+ `write_all` + `flush` → `tokio::fs::write` |
| `artcraft_router` | `generate_video/providers/kinovi/upload.rs` |
`std::fs::read` → `tokio::fs::read(..).await` |
| `midjourney_client` | `credentials/credential_manager.rs`
(`clear_credentials`) | `std::fs::remove_file` →
`tokio::fs::remove_file(..).await` |
| `openai_sora_client`|
`requests/deprecated/job_status/sora_job_status.rs` | `create_dir_all` /
`File::create` + `write_all` → `tokio::fs` |

The workspace `tokio` dependency enables only `macros`.
`artcraft_client` (which had `tokio` only as a dev-dependency) and
`artcraft_router` now declare `tokio = { workspace = true, features =
["fs"] }`. `midjourney_client` and `openai_sora_client` already use
`tokio` with `full`. `Cargo.lock` does not change. In
`save_generations_to_dir`, the blocking `dir.exists()` check before
`create_dir_all` is dropped, because `create_dir_all` already succeeds
when the directory exists.

The behaviour stays the same: same paths, same error mapping, and the
same `NotFound` handling in `clear_credentials`.

## Not included

- `crates/schema/database/sqlite_tasks/src/connection.rs:25` (S7493):
#2360 removes this `remove_file` entirely, so it's left to that PR.
- `crates/desktop/artcraft` (26 findings): handled in a separate PR.

#2365, #2373 and #2785 edit nearby lines in the `*_from_file.rs` upload
files (client builder, idempotency token). None of them touch the
`std::fs::read` lines. Expect at most a trivial rebase.

## Testing

Run locally on Linux x86_64 with stable Rust:

- `cargo check -p artcraft_client -p artcraft_router -p
midjourney_client -p openai_sora_client` passes.
- `cargo test` on the same four crates passes: 2555 + 39 + 16 + 6 unit
tests, with ignored tests left ignored. That includes
`midjourney_client`'s existing
`loads_a_saved_login_and_logout_removes_every_format`, which exercises
the changed `clear_credentials` path.
- `cargo clippy` on these crates reports nothing on the changed lines.
Clippy cannot finish on `main` itself: `enums` fails on
`clippy::deprecated_semver` and `openai_sora_client` fails on
`should_implement_trait` in `task_id.rs`, both pre-existing.

No new test is added. This is a non-blocking I/O refactor with identical
observable behaviour, so there is no test that fails before the change
and passes after it. The other changed paths do network I/O and have no
offline tests. Please rely on CI and the SonarCloud re-analysis to
confirm the S7493 findings close.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Refactor**
* File uploads and downloads, generated image saving, and
credential-file cleanup now use asynchronous file operations.
* When a local file can’t be read during a video upload, the existing
fallback to downloading from the source URL remains in place.
* Existing handling of file-read, write, and removal errors is
preserved, including ignoring missing credential files.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 18:53:57 +01:00
FredApps c3fa0453a4 desktop: use tokio::fs for Sora and Midjourney file I/O (Sonar S7493) (#2804)
## Summary

Replaces blocking `std::fs` calls inside `async fn`s in the desktop
crate with `tokio::fs`. SonarCloud reports these as **rust:S7493** ("use
an asynchronous file API instead of this blocking file operation"). This
PR resolves 17 of the 26 desktop findings.

| File (under `crates/desktop/artcraft/src/services/`) | Findings |
Change |

|----------------------------------------------------------------|----------|----------------------------------------------------------------------------|
| `sora/state/read_sora_credentials_from_disk_with_upgrade.rs` | 10 |
`read_to_string` and `OpenOptions`+`write_all`+`flush` →
`tokio::fs::{read_to_string, write}` |
| `sora/state/initialize_sora_jwt_bearer_token.rs` | 2 |
`std::fs::{read_to_string, write}` → `tokio::fs` |
| `sora/state/sora_credential_manager.rs` (`call_sentinel_refresh`) | 2
| `OpenOptions`+`write_all`+`flush` → `tokio::fs::write` |
| `midjourney/utils/download_midjourney_image.rs` | 2 |
`File::create`+`write_all` → `tokio::fs::write` |
| `midjourney/threads/upload_midjourney_batch.rs` | 1 |
`std::fs::remove_file` → `tokio::fs::remove_file(..).await` |

`tokio::fs::write` creates or truncates the file and writes everything.
That is the same as the
`OpenOptions::new().create(true).write(true).truncate(true)` +
`write_all` + `flush` sequence it replaces. The synchronous helpers in
`sora_credential_manager.rs` (`persist_*_to_disk`) are not `async`, are
not flagged, and are left unchanged. The `.exists()` guards are also
left as they are. The desktop crate already depends on `tokio` with the
`fs` feature, so no manifest or lockfile changes.

## Not included (overlap with open PRs)

The remaining 9 desktop S7493 findings are in files that open PRs are
rewriting. They are left alone to avoid conflicting changes:

- `core/utils/simple_http_download.rs`,
`sora/.../handle_successful_generations.rs`,
`worldlabs/threads/worldlabs_marble_task_polling.rs`: #2777 (fail
downloads on non-2xx responses).
- `core/threads/.../fal/handle_fal_complete.rs`: #2550, #2373.
- `core/utils/auto_download.rs`: #2785, #2686, #2393.

The api_clients findings are in #2803.

## Testing

- Each changed file parses with `rustfmt --check`. The formatting diffs
it reports already exist on `main`, and no files were reformatted.
- **I could not build the desktop crate locally.** `cargo check -p
artcraft` stops at `libdbus-sys` because the build machine lacks the
GTK, WebKit and D-Bus system libraries Tauri needs. Please rely on CI
for compilation before marking this ready.
- No new tests are added. This is a behaviour-preserving switch to
non-blocking I/O, so no test fails before the change and passes after
it, and these paths have no existing offline tests.


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Updated background handling for saved images, checkpoints, and account
credentials. Existing cleanup, credential processing, and error
reporting behavior remain unchanged.
* These maintenance updates do not change the app’s available features
or how people interact with it.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
2026-11-10 18:36:03 +01:00
FredAppsandFredApps 1b99c184ef ci: one macOS and one Windows release build per branch at a time, with job time limits (#2809)
This changes release workflows (house rule 9). It only adds limits, but
it needs a maintainer's look before merging.

## Problem

`artcraft-macos-publish.yml` and `artcraft-windows-publish.yml` have no
`concurrency` and no `timeout-minutes`. `artcraft-linux-publish.yml`
already has both.

- **Overlapping builds:** two pushes to `release` or `artcraft-release`
in quick succession start two builds of the same platform side by side.
Both create or update the same `artcraft-v<version>` draft release
through `tauri-action`. Nothing stops this today. It happens whenever a
second push lands while a 10-36 minute build is still running. (The two
runs on 2026-11-07 were 35 minutes apart and did not overlap.)
- **Hung builds:** a build that hangs runs until GitHub's 6-hour
default. The Windows run on 2026-06-12 did exactly that and was
cancelled at 360 minutes. Successful builds take far less:

| workflow | successful builds, last 15 runs (2026-04 to 2026-11) |
|---|---|
| macOS publish | 10-22 min |
| Windows publish | 17-36 min |

## Change

Two files, limits only. Triggers, steps, secrets, signing, release
settings and action versions are unchanged.

- **Concurrency:** `concurrency: { group: macos-release-${{ github.ref
}}, cancel-in-progress: false }`, and likewise `windows-release-…`. This
is the same pattern as `artcraft-linux-publish.yml` (`linux-release-${{
github.ref }}`).
- A release build that is already running is **never cancelled**, since
it may already have uploaded to the draft release.
- One GitHub behaviour to be aware of: a concurrency group holds one
running and one pending run. If a third push lands while one build runs
and another waits, GitHub replaces the waiting run with the newest. So
the next build is always of the latest commit on that branch, and an
intermediate commit may not get its own build. For a release branch I
think that is what we want. If every commit must get a build, this PR
shouldn't be merged as it is; tell me and I'll change it.
- The groups are per platform and per branch, so macOS, Windows and
Linux still build in parallel with each other.
- **Timeouts:** `timeout-minutes: 60` for macOS and `90` for Windows.
That is about 2.5-3× the slowest successful build, and Windows matches
the Linux build job's 90. A hang now stops after an hour or an hour and
a half instead of six.

Not changed: `dtolnay/rust-toolchain@stable` and
`tauri-apps/tauri-action@v0` are still unpinned tags. Pinning them would
change which action code builds a release, so it should be a separate
decision.

## Verification

- `actionlint` is clean on the three publish workflows.
- These workflows only run on pushes to `release`, `artcraft-release`,
`artcraft-macos` and `artcraft-windows`, so this PR's checks don't
exercise them. The new settings take effect on the next release push.
- I didn't trigger a release build.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

<!-- coderabbit-ai-generated:v1 -->

## Summary by CodeRabbit

* **Chores**
* Release builds for macOS and Windows now finish in progress without
being cancelled by newer runs. When multiple builds are queued for the
same branch or ref, only the newest pending build is kept.
* Publishing jobs now have maximum run times of 60 minutes for macOS and
90 minutes for Windows.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Co-authored-by: FredApps <FredApps@users.noreply.github.com>
2026-11-10 16:45:35 +01:00
AxylumandCédric Legrand 1510abf563 url_utils: satisfy Clippy pedantic lints (#2681)
## Update — 2026-11-01

The latest revision removes the systematic `#[must_use]` additions and
any product-name backticks introduced solely for optional
`clippy::pedantic` compliance. The useful behavior-preserving cleanups
remain. Earlier summary and verification text below describes the
original revision; the latest PR comment records the current checks.

---

## Summary
- resolve all 54 distinct diagnostics in `url_utils`: 17 pedantic, 36
deprecation warnings, and one Eq/Hash contract issue
- preserve the public compatibility module and nested module paths with
narrow documented expectations
- implement `Hash` against the same bare value as `PartialEq`, with a
regression test
- apply the repository two-space rustfmt rules within `url_utils` only

No public API or dependency changes.

## Verification
- `SQLX_OFFLINE=true cargo clippy -p url_utils --all-targets -- -W
clippy::pedantic -D warnings`
- `SQLX_OFFLINE=true cargo test -p url_utils` — 32 unit tests and 2
doctests passed
- `cargo fmt -p url_utils -- --check`
- Independent read-only review: no Rust findings; the untracked planning
file is excluded from the PR

---------

Co-authored-by: Cédric Legrand <cedric@legrand-tech.fr>
2026-11-10 11:32:48 +01:00
Dylan M. Taylor 1218293fe2 docs: add CodeRabbit badge to README (#2790)
Add CodeRabbit review badge to README alongside SonarCloud.
2026-11-09 10:09:12 -05:00
26 changed files with 190 additions and 156 deletions
+4
View File
@@ -14,6 +14,8 @@ updates:
- "*"
commit-message:
prefix: "chore(deps)"
cooldown:
default-days: 7
# Maintain GitHub Actions workflows
- package-ecosystem: "github-actions"
@@ -28,3 +30,5 @@ updates:
- "*"
commit-message:
prefix: "ci(deps)"
cooldown:
default-days: 7
+42 -12
View File
@@ -13,7 +13,13 @@ jobs:
runs-on: ubuntu-22.04
timeout-minutes: 10
steps:
- uses: actions/checkout@v7
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: rpm jq binutils zsync
@@ -34,7 +40,13 @@ jobs:
TAURI_FRONTEND_PATH: ${{ github.workspace }}/frontend
TAURI_APP_PATH: ${{ github.workspace }}/crates/desktop/artcraft
steps:
- uses: actions/checkout@v7
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: storytold/craft-common-actions/actions/install-apt-packages@main
with:
packages: libwebkit2gtk-4.1-dev libappindicator3-dev librsvg2-dev patchelf libssl-dev libclang-dev libasound2-dev libxdo-dev libdbus-1-dev build-essential clang file jq perl cmake nasm rpm zsync gstreamer1.0-plugins-base gstreamer1.0-plugins-good gstreamer1.0-libav
@@ -63,7 +75,7 @@ jobs:
PKG_CONFIG_LIBDIR_riscv64gc_unknown_linux_gnu=/usr/lib/riscv64-linux-gnu/pkgconfig:/usr/share/pkgconfig
ENV
echo "CMAKE_TOOLCHAIN_FILE_riscv64gc_unknown_linux_gnu=$GITHUB_WORKSPACE/packaging/linux/riscv64.cmake" >> "$GITHUB_ENV"
- uses: actions/setup-node@v7
- uses: actions/setup-node@0a44ba7841725637a54e2801669c06500c541b61 # v7.0.0
with:
node-version: '22'
cache: npm
@@ -80,7 +92,7 @@ jobs:
working-directory: frontend
- run: ./node_modules/.bin/nx run artcraft:build
working-directory: frontend
- uses: tauri-apps/tauri-action@fce9c6108b31ea247710505d3aaaa893ee6768d4 # v0
- uses: tauri-apps/tauri-action@fce9c6108b31ea247710505d3aaaa893ee6768d4 # v0.5.18
with:
projectPath: crates/desktop/artcraft
args: --target ${{ matrix.target }} --bundles ${{ matrix.bundles }} --config ${{ github.workspace }}/packaging/linux/tauri.conf.json
@@ -89,7 +101,7 @@ jobs:
ARCH: ${{ matrix.arch }}
TARGET: ${{ matrix.target }}
run: packaging/linux/package.sh "$ARCH" "$TARGET"
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/release/*
@@ -102,8 +114,14 @@ jobs:
matrix:
arch: [x86_64, aarch64]
steps:
- uses: actions/checkout@v7
- uses: actions/download-artifact@v8
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
@@ -126,7 +144,7 @@ jobs:
for package in dist/release/*.pkg.tar.zst; do
tar --zstd -xOf "$package" .PKGINFO | grep -Fx 'depend = webkit2gtk-4.1'
done
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-arch-${{ matrix.arch }}
path: dist/release/*.pkg.tar.zst
@@ -145,8 +163,14 @@ jobs:
- {arch: x86_64, runner: ubuntu-24.04}
- {arch: aarch64, runner: ubuntu-24.04-arm}
steps:
- uses: actions/checkout@v7
- uses: actions/download-artifact@v8
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: actions/download-artifact@9000827ccba6bdab643e8b6fd33ac0654aef8333 # v8.0.2
with:
name: release-linux-${{ matrix.arch }}
path: dist/linux
@@ -164,7 +188,7 @@ jobs:
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
run: packaging/linux/flatpak.sh
- uses: actions/upload-artifact@v7
- uses: actions/upload-artifact@cf430e030ddbb5b0abf93d22962f4752f3646cd9 # v7.0.2
with:
name: release-flatpak-${{ matrix.arch }}
path: |
@@ -177,7 +201,13 @@ jobs:
permissions:
contents: write
steps:
- uses: actions/checkout@v7
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: Read version
id: version
run: echo "value=$(jq -r .version crates/desktop/artcraft/tauri.conf.json)" >> "$GITHUB_OUTPUT"
+21 -4
View File
@@ -10,6 +10,13 @@ on:
# This workflow creates Universal macOS builds with signing certificate support.
# On each push to the release branches it will create or update a GitHub release, build your app, and upload the artifacts to the release.
# One release build per branch at a time. A run in progress is never cancelled, because it may
# already have created or uploaded to the draft release. If more pushes arrive while one runs,
# GitHub keeps only the newest as pending, so the next build is of the latest commit.
concurrency:
group: macos-release-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-artcraft:
permissions:
@@ -28,17 +35,27 @@ jobs:
# args: ''
runs-on: ${{ matrix.platform }}
# Successful builds took 10-22 min (2026-04 to 2026-11).
timeout-minutes: 60
steps:
- uses: actions/checkout@v7
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: setup node
uses: actions/setup-node@v7
uses: actions/setup-node@0a44ba7841725637a54e2801669c06500c541b61 # v7.0.0
with:
node-version: lts/*
- name: install Rust stable
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # v1
with:
toolchain: stable
targets: ${{ matrix.platform == 'macos-latest-xlarge' && 'aarch64-apple-darwin,x86_64-apple-darwin' || '' }}
- name: install dependencies (ubuntu only)
@@ -217,7 +234,7 @@ jobs:
- name: build and publish
uses: tauri-apps/tauri-action@v0
uses: tauri-apps/tauri-action@fce9c6108b31ea247710505d3aaaa893ee6768d4 # v0.5.18
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
APPLE_ID: ${{ secrets.APPLE_ID }}
+22 -4
View File
@@ -10,6 +10,13 @@ on:
# This workflow creates Windows builds.
# On each push to the release branches it will create or update a GitHub release, build your app, and upload the artifacts to the release.
# One release build per branch at a time. A run in progress is never cancelled, because it may
# already have created or uploaded to the draft release. If more pushes arrive while one runs,
# GitHub keeps only the newest as pending, so the next build is of the latest commit.
concurrency:
group: windows-release-${{ github.ref }}
cancel-in-progress: false
jobs:
publish-artcraft-windows:
permissions:
@@ -23,11 +30,22 @@ jobs:
args: ''
runs-on: ${{ matrix.platform }}
# Successful builds took 17-36 min (2026-04 to 2026-11); one hung run used the full 6-hour default.
timeout-minutes: 90
steps:
- uses: actions/checkout@v7
- name: Harden runner
uses: step-security/harden-runner@1d88384fd173af6ecb012dfd0264a306b711241a # v2.22.4
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- name: install Rust stable
uses: dtolnay/rust-toolchain@stable
uses: dtolnay/rust-toolchain@683b588c487a69d01f6c6ddf40f036ff682cc2c8 # v1
with:
toolchain: stable
- name: install wreq dependencies
run: |
@@ -39,7 +57,7 @@ jobs:
shell: cmd
- name: setup node
uses: actions/setup-node@v7
uses: actions/setup-node@0a44ba7841725637a54e2801669c06500c541b61 # v7.0.0
with:
node-version: lts/*
@@ -92,7 +110,7 @@ jobs:
$config | ConvertTo-Json -Depth 10 | Set-Content tauri.conf.json
- name: build and publish
uses: tauri-apps/tauri-action@v0
uses: tauri-apps/tauri-action@fce9c6108b31ea247710505d3aaaa893ee6768d4 # v0.5.18
env:
# NB(bt): This is needed by wreq. Hopefully it doesn't kill Tauri
RUSTFLAGS: "-C target-feature=+crt-static"
@@ -33,6 +33,7 @@ chrono.workspace = true
cookie.workspace = true
log.workspace = true
reqwest = { workspace = true, features = ["multipart", "gzip"] }
tokio = { workspace = true, features = ["fs"] }
url.workspace = true
# Serialization
@@ -60,7 +60,7 @@ pub async fn legacy_upload_media_file_from_file<P: AsRef<Path>>(
.build()
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_bytes = std::fs::read(args.path.as_ref())
let file_bytes = tokio::fs::read(args.path.as_ref()).await
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
@@ -69,7 +69,7 @@ pub async fn upload_image_media_file_from_file_with_idempotency_token<P: AsRef<P
.build()
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_bytes = std::fs::read(args.path.as_ref())
let file_bytes = tokio::fs::read(args.path.as_ref()).await
.map_err(|err| StorytellerError::Client(ClientError::from(err)))?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
@@ -35,7 +35,7 @@ pub async fn upload_new_engine_asset_from_file<P: AsRef<Path>>(
let client = storyteller_client_builder(api_host)
.build()?;
let file_bytes = std::fs::read(path.as_ref())?;
let file_bytes = tokio::fs::read(path.as_ref()).await?;
let file_name = path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
let mut form = Form::new()
@@ -61,7 +61,7 @@ pub async fn upload_video_media_file_from_file<P: AsRef<Path>>(
let client = storyteller_client_builder(args.api_host)
.build()?;
let file_bytes = std::fs::read(args.path.as_ref())?;
let file_bytes = tokio::fs::read(args.path.as_ref()).await?;
let file_name = args.path.as_ref().file_name()
.and_then(|n| n.to_str()).unwrap_or("file").to_string();
let mut form = Form::new()
@@ -1,5 +1,3 @@
use std::fs;
use std::io::Write;
use std::path::{Path, PathBuf};
use log::{info, warn};
@@ -68,13 +66,7 @@ pub async fn download_media_file<P: AsRef<Path>>(
let bytes = download_bytes(cdn_url).await?;
// 4. Write to disk.
let mut file = fs::File::create(&output_path)
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
file.write_all(&bytes)
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
file.flush()
tokio::fs::write(&output_path, &bytes).await
.map_err(|err| StorytellerError::Client(ClientError::IoError(err)))?;
info!("Downloaded {} bytes to {:?}", bytes.len(), output_path);
@@ -31,7 +31,7 @@ reqwest.workspace = true
serde.workspace = true
wreq.workspace = true
serde_derive.workspace = true
tokio.workspace = true
tokio = { workspace = true, features = ["fs"] }
uuid.workspace = true
[dev-dependencies]
@@ -24,7 +24,7 @@ pub(crate) async fn upload_to_kinovi_web(
.unwrap_or_else(|| "jpg".to_string());
let file_bytes = match maybe_local_path {
Some(local_path) => match std::fs::read(local_path) {
Some(local_path) => match tokio::fs::read(local_path).await {
Ok(bytes) => {
info!("Reusing predownloaded file for kinovi upload: {}", source_url);
bytes
@@ -93,7 +93,7 @@ impl MidjourneyCredentialManager {
let _guard = self.mutation.lock().await;
// Remove every format, so a later startup cannot migrate a logged-out account.
for path in self.legacy_paths.iter().chain([&self.credential_path]) {
match std::fs::remove_file(path) {
match tokio::fs::remove_file(path).await {
Ok(()) => {},
Err(err) if err.kind() == std::io::ErrorKind::NotFound => {},
Err(err) => return Err(err.into()),
@@ -11,7 +11,7 @@ use errors::AnyhowResult;
use log::{debug, error, warn};
use serde_derive::{Deserialize, Serialize};
use std::fmt::{Debug, Display};
use std::{fs::File, io::Write, path::Path};
use std::path::Path;
use url::Url;
use wreq::Client;
@@ -271,9 +271,7 @@ pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &
pub async fn save_generations_to_dir(generations: &[Generation], dir: &str) -> AnyhowResult<()> {
let dir = Path::new(dir);
if !dir.exists() {
std::fs::create_dir_all(dir)?;
}
tokio::fs::create_dir_all(dir).await?;
for generation in generations {
let response = wreq::get(generation.url.clone()).send().await?;
let image_bytes = response.bytes().await?;
@@ -282,15 +280,13 @@ pub async fn save_generations_to_dir(generations: &[Generation], dir: &str) -> A
let ext = url.path().split(".").last().unwrap_or("png");
let path = Path::new(dir).join(format!("{}.{}", generation.id, ext));
let mut file = File::create(path)?;
file.write_all(&image_bytes)?;
tokio::fs::write(path, &image_bytes).await?;
let thumbnail_url = generation.encodings.thumbnail.path.clone();
let thumbnail_response = wreq::get(thumbnail_url).send().await?;
let thumbnail_bytes = thumbnail_response.bytes().await?;
let thumbnail_path = Path::new(dir).join(format!("{}_thumbnail.{}", generation.id, ext));
let mut thumbnail_file = File::create(thumbnail_path)?;
thumbnail_file.write_all(&thumbnail_bytes)?;
tokio::fs::write(thumbnail_path, &thumbnail_bytes).await?;
}
Ok(())
@@ -119,7 +119,7 @@ pub(super) async fn upload_midjourney_batch(maybe_app_handle: Option<&AppHandle>
}
}
if let Err(err) = std::fs::remove_file(MidjourneyUploadCheckpoint::path(app_data_root, local_task)) {
if let Err(err) = tokio::fs::remove_file(MidjourneyUploadCheckpoint::path(app_data_root, local_task)).await {
log::warn!("Could not remove completed Midjourney upload checkpoint: {}", err);
}
Ok(())
@@ -2,8 +2,6 @@ use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use errors::AnyhowResult;
use midjourney_client::utils::get_image_url::get_image_url;
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use url::Url;
use midjourney_client::utils::image_downloader_client::ImageDownloaderClient;
@@ -28,8 +26,7 @@ pub async fn download_midjourney_image(
let download_filename = format!("{}_{}.{}", job_id, index, ext);
let download_path = tempdir.join(download_filename);
let mut file = File::create(&download_path)?;
file.write_all(&image_bytes)?;
tokio::fs::write(&download_path, &image_bytes).await?;
Ok(download_path)
}
@@ -14,7 +14,7 @@ pub async fn initialize_sora_jwt_bearer_token(app_data_root: &AppDataRoot) -> An
return Ok(()); // Bearer token already exists.
}
let cookie = std::fs::read_to_string(app_data_root.get_sora_cookie_file_path())?
let cookie = tokio::fs::read_to_string(app_data_root.get_sora_cookie_file_path()).await?
.trim()
.to_string();
@@ -22,7 +22,7 @@ pub async fn initialize_sora_jwt_bearer_token(app_data_root: &AppDataRoot) -> An
let bearer = generate_bearer_jwt_with_cookie(&cookie).await?;
std::fs::write(app_data_root.get_sora_bearer_token_file_path(), bearer)?;
tokio::fs::write(app_data_root.get_sora_bearer_token_file_path(), bearer).await?;
Ok(())
}
@@ -8,8 +8,7 @@ use openai_sora_client::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
use openai_sora_client::creds::sora_sentinel::SoraSentinel;
use openai_sora_client::creds::sora_sentinel_token::SoraSentinelToken;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use std::fs::{read_to_string, OpenOptions};
use std::io::Write;
use tokio::fs::{read_to_string, write};
pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDataRoot) -> AnyhowResult<SoraCredentialSet> {
let cookie_file = app_data_root.get_sora_cookie_file_path();
@@ -21,7 +20,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
return Err(anyhow!("Cookie file does not exist: {:?}", cookie_file));
}
let value = read_to_string(cookie_file)?
let value = read_to_string(cookie_file).await?
.trim()
.to_string();
@@ -32,14 +31,14 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
let mut sentinel_token = None;
if bearer_file.exists() {
let value = read_to_string(&bearer_file)?
let value = read_to_string(&bearer_file).await?
.trim()
.to_string();
bearer = Some(SoraJwtBearerToken::new(value)?);
}
if legacy_sentinel_file.exists() {
let value = read_to_string(&legacy_sentinel_file)?
let value = read_to_string(&legacy_sentinel_file).await?
.trim()
.to_string();
@@ -47,7 +46,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
}
if sentinel_token_file.exists() {
let value = read_to_string(&sentinel_token_file)?
let value = read_to_string(&sentinel_token_file).await?
.trim()
.to_string();
@@ -73,14 +72,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting bearer token value...");
let value = bearer.token_str();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&bearer_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&bearer_file, value.as_bytes()).await?;
}
}
@@ -89,14 +81,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting sentinel value...");
let value = sentinel.get_sentinel();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&legacy_sentinel_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&legacy_sentinel_file, value.as_bytes()).await?;
}
}
@@ -105,14 +90,7 @@ pub async fn read_sora_credentials_from_disk_with_upgrade(app_data_root: &AppDat
info!("Persisting sentinel token...");
let value = sentinel_token.to_persistent_storage_json()?;
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(&sentinel_token_file)?;
file.write_all(value.as_bytes())?;
file.flush()?;
write(&sentinel_token_file, value.as_bytes()).await?;
}
}
@@ -162,14 +162,7 @@ impl SoraCredentialManager {
self.holder.set_credentials(&creds)?;
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(self.app_data_root.get_sora_legacy_sentinel_file_path())?;
file.write_all(sentinel.as_bytes())?;
file.flush()?;
tokio::fs::write(self.app_data_root.get_sora_legacy_sentinel_file_path(), sentinel.as_bytes()).await?;
Ok(creds)
}
@@ -1,4 +1,3 @@
/// When downloading a remote file, it makes sense to try to match the same file extension (with reasonable safeguards).
#[deprecated(note = "Use `Extension` in `url_utils::extension::extension` instead")]
#[derive(Debug, Copy, Clone, Eq, PartialEq)]
@@ -12,9 +12,7 @@ pub fn extract_download_extension_from_url_str(url: &str) -> Option<DownloadExte
#[deprecated(note = "Use `extract_extension_from_url` in `url_utils::extension::extract_extension_from_url` instead")]
pub fn extract_download_extension_from_url(url: &Url) -> Option<DownloadExtension> {
let path = url.path();
let extension = std::path::Path::new(path)
.extension()
.and_then(|ext| ext.to_str())?;
let extension = std::path::Path::new(path).extension().and_then(|ext| ext.to_str())?;
DownloadExtension::try_from_str(extension)
}
@@ -26,14 +24,14 @@ mod tests {
#[test]
fn success_case() {
let url = "https://storyteller.ai/example.jpg";
let maybe_extension= extract_download_extension_from_url_str(url);
let maybe_extension = extract_download_extension_from_url_str(url);
assert_eq!(maybe_extension, Some(DownloadExtension::Jpg));
}
#[test]
fn difficult_url() {
let url = "https://videos.youtube.com/foo/bar/files/00000000-f19a-1275-a092-caa60b38f8d2%2Fraw?se=2025-11-21T22%3A26%3A53Z&sp=r&sv=2024-08-09&sr=b&skoid=8cfff87a-01fc-27c1-9091-32999dcd1389&sktid=aa8ccaf6-e6da-484e-ab11-9c7496c21c53&skt=2025-11-17T21%3A09%3A08Z&ske=2025-10-23T21%3A14%3A08Z&sks=b&skv=2024-08-04&sig=T2U6%2Brw9WJdJagBkCL4/GlFr72KhZGP3QPlobIO7JlU%3D&ac=oaisdsorprnorthcentralus";
let maybe_extension= extract_download_extension_from_url_str(url);
let maybe_extension = extract_download_extension_from_url_str(url);
assert!(maybe_extension.is_none());
}
}
@@ -1,3 +1,4 @@
#[expect(clippy::module_inception, reason = "Preserve the existing public module path")]
mod download_extension;
pub mod extract_download_extension_from_url;
+29 -12
View File
@@ -1,5 +1,6 @@
use std::borrow::Cow;
use std::fmt;
use std::hash::{Hash, Hasher};
/// A file extension that can be either a compile-time constant or a runtime-parsed value.
///
@@ -16,7 +17,7 @@ use std::fmt;
/// assert_eq!(ext.without_period(), "mp4");
/// assert_eq!(ext.with_period(), ".mp4");
/// ```
#[derive(Clone, Eq, Hash)]
#[derive(Clone, Eq)]
pub struct Extension {
/// The extension without the leading period, e.g. "png", "mp4".
value: Cow<'static, str>,
@@ -34,20 +35,14 @@ impl Extension {
/// const MP4: Extension = Extension::from_static("mp4", ".mp4");
/// ```
pub const fn from_static(ext: &'static str, dotted: &'static str) -> Self {
Self {
value: Cow::Borrowed(ext),
dotted: Cow::Borrowed(dotted),
}
Self { value: Cow::Borrowed(ext), dotted: Cow::Borrowed(dotted) }
}
/// Creates an `Extension` from a `&str` at runtime.
/// The input must NOT include a leading period.
pub fn new(ext: &str) -> Self {
let dotted = format!(".{}", ext);
Self {
value: Cow::Owned(ext.to_string()),
dotted: Cow::Owned(dotted),
}
let dotted = format!(".{ext}");
Self { value: Cow::Owned(ext.to_string()), dotted: Cow::Owned(dotted) }
}
/// Returns the extension without the leading period, e.g. `"png"`.
@@ -67,6 +62,12 @@ impl PartialEq for Extension {
}
}
impl Hash for Extension {
fn hash<H: Hasher>(&self, state: &mut H) {
self.value.hash(state);
}
}
impl fmt::Debug for Extension {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(f, "Extension(\"{}\")", self.value)
@@ -135,12 +136,28 @@ mod tests {
#[test]
fn display() {
assert_eq!(format!("{}", PNG), "png");
assert_eq!(format!("{PNG}"), "png");
assert_eq!(format!("{}", Extension::new("xyz")), "xyz");
}
#[test]
fn debug() {
assert_eq!(format!("{:?}", PNG), "Extension(\"png\")");
assert_eq!(format!("{PNG:?}"), "Extension(\"png\")");
}
#[test]
fn hash_matches_equality() {
let plain = Extension::from_static("png", ".png");
let inconsistent = Extension::from_static("png", ".not-a-suffix");
assert_eq!(plain, inconsistent);
assert_eq!(hash(&plain), hash(&inconsistent));
}
fn hash(extension: &Extension) -> u64 {
use std::collections::hash_map::DefaultHasher;
let mut hasher = DefaultHasher::new();
extension.hash(&mut hasher);
hasher.finish()
}
}
@@ -6,24 +6,13 @@ use url::Url;
// TODO: Lookups against these as lists will be slow
/// Known image extensions.
const KNOWN_IMAGE_EXTENSIONS: &[Extension] = &[
Extension::from_static("png", ".png"),
Extension::from_static("jpg", ".jpg"),
Extension::from_static("jpeg", ".jpeg"),
Extension::from_static("webp", ".webp"),
];
const KNOWN_IMAGE_EXTENSIONS: &[Extension] = &[Extension::from_static("png", ".png"), Extension::from_static("jpg", ".jpg"), Extension::from_static("jpeg", ".jpeg"), Extension::from_static("webp", ".webp")];
/// Known audio extensions.
const KNOWN_AUDIO_EXTENSIONS: &[Extension] = &[
Extension::from_static("wav", ".wav"),
Extension::from_static("mp3", ".mp3"),
];
const KNOWN_AUDIO_EXTENSIONS: &[Extension] = &[Extension::from_static("wav", ".wav"), Extension::from_static("mp3", ".mp3")];
/// Known video extensions.
const KNOWN_VIDEO_EXTENSIONS: &[Extension] = &[
Extension::from_static("mp4", ".mp4"),
Extension::from_static("webm", ".webm"),
];
const KNOWN_VIDEO_EXTENSIONS: &[Extension] = &[Extension::from_static("mp4", ".mp4"), Extension::from_static("webm", ".webm")];
/// Controls which extensions are accepted when extracting from a URL.
pub enum ExtractExtensions {
@@ -62,9 +51,7 @@ pub fn extract_extension_from_url_str(url: &str, accept: &ExtractExtensions) ->
/// Extracts a file extension from a parsed `Url`, filtered by `accept`.
pub fn extract_extension_from_url(url: &Url, accept: &ExtractExtensions) -> Option<Extension> {
let path = url.path();
let raw_ext = std::path::Path::new(path)
.extension()
.and_then(|ext| ext.to_str())?;
let raw_ext = std::path::Path::new(path).extension().and_then(|ext| ext.to_str())?;
let lower = raw_ext.to_lowercase();
let candidate = Extension::new(&lower);
@@ -72,27 +59,40 @@ pub fn extract_extension_from_url(url: &Url, accept: &ExtractExtensions) -> Opti
match accept {
ExtractExtensions::All => Some(candidate),
ExtractExtensions::Set(set) => {
if set.contains(&candidate) { Some(candidate) } else { None }
}
ExtractExtensions::KnownImage => {
if slice_contains(KNOWN_IMAGE_EXTENSIONS, &candidate) { Some(candidate) } else { None }
}
ExtractExtensions::KnownAudio => {
if slice_contains(KNOWN_AUDIO_EXTENSIONS, &candidate) { Some(candidate) } else { None }
}
ExtractExtensions::KnownVideo => {
if slice_contains(KNOWN_VIDEO_EXTENSIONS, &candidate) { Some(candidate) } else { None }
}
ExtractExtensions::KnownMedia => {
if slice_contains(KNOWN_IMAGE_EXTENSIONS, &candidate)
|| slice_contains(KNOWN_AUDIO_EXTENSIONS, &candidate)
|| slice_contains(KNOWN_VIDEO_EXTENSIONS, &candidate)
{
if set.contains(&candidate) {
Some(candidate)
} else {
None
}
}
},
ExtractExtensions::KnownImage => {
if slice_contains(KNOWN_IMAGE_EXTENSIONS, &candidate) {
Some(candidate)
} else {
None
}
},
ExtractExtensions::KnownAudio => {
if slice_contains(KNOWN_AUDIO_EXTENSIONS, &candidate) {
Some(candidate)
} else {
None
}
},
ExtractExtensions::KnownVideo => {
if slice_contains(KNOWN_VIDEO_EXTENSIONS, &candidate) {
Some(candidate)
} else {
None
}
},
ExtractExtensions::KnownMedia => {
if slice_contains(KNOWN_IMAGE_EXTENSIONS, &candidate) || slice_contains(KNOWN_AUDIO_EXTENSIONS, &candidate) || slice_contains(KNOWN_VIDEO_EXTENSIONS, &candidate) {
Some(candidate)
} else {
None
}
},
}
}
@@ -190,19 +190,14 @@ mod tests {
#[test]
fn set_matches_custom_list() {
let accept = ExtractExtensions::from_vec(vec![
Extension::from_static("csv", ".csv"),
Extension::from_static("tsv", ".tsv"),
]);
let accept = ExtractExtensions::from_vec(vec![Extension::from_static("csv", ".csv"), Extension::from_static("tsv", ".tsv")]);
let ext = extract_extension_from_url_str("https://example.com/data.csv", &accept);
assert_eq!(ext.unwrap().without_period(), "csv");
}
#[test]
fn set_rejects_non_member() {
let accept = ExtractExtensions::from_vec(vec![
Extension::from_static("csv", ".csv"),
]);
let accept = ExtractExtensions::from_vec(vec![Extension::from_static("csv", ".csv")]);
let ext = extract_extension_from_url_str("https://example.com/data.png", &accept);
assert!(ext.is_none());
}
@@ -215,10 +210,7 @@ mod tests {
#[test]
fn query_params_ignored() {
let ext = extract_extension_from_url_str(
"https://example.com/file.mp4?token=abc&format=hd",
&ExtractExtensions::KnownVideo,
);
let ext = extract_extension_from_url_str("https://example.com/file.mp4?token=abc&format=hd", &ExtractExtensions::KnownVideo);
assert_eq!(ext.unwrap().without_period(), "mp4");
}
@@ -230,8 +222,7 @@ mod tests {
}
fn extract_extension_all(url: &str) -> Option<String> {
extract_extension_from_url_str(url, &ExtractExtensions::All)
.map(|ext| ext.without_period().to_string())
extract_extension_from_url_str(url, &ExtractExtensions::All).map(|ext| ext.without_period().to_string())
}
#[test]
+2 -1
View File
@@ -1,2 +1,3 @@
#[expect(clippy::module_inception, reason = "Preserve the existing public module path")]
pub mod extension;
pub mod extract_extension_from_url;
pub mod extract_extension_from_url;
+2 -1
View File
@@ -1,2 +1,3 @@
pub mod extension;
pub mod download_extension;
#[expect(deprecated, reason = "Keep the public compatibility module while downstream users migrate to Extension")]
pub mod download_extension;