Compare commits

..
Author SHA1 Message Date
Tanish Manku d181688d94 Recreate ui-input and build 2025-05-27 15:19:54 -07:00
Brandon Thomas 569622bcc0 make frontend/clean_modules.sh take the nuclear option 2025-05-27 16:52:12 -04:00
Brandon Thomas 7aad79d16b timestamp when build finishes 2025-05-27 15:14:45 -04:00
Brandon Thomas d70f8ac41e fix circular deps in nx (#639) 2025-05-27 14:25:57 -04:00
ArEnSc 7097144e19 Removals (#636)
Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
Co-authored-by: Michael Chung <michaelchung@mac.home>
Co-authored-by: bflatastic <bombayapirat@gmail.com>
Co-authored-by: Tanish Manku <tanish2k09.dev@gmail.com>
Good work every one =)
2025-05-27 12:00:35 -04:00
Brandon Thomas 709af560a5 enqueue gpt-4o images too 2025-05-26 23:46:05 -04:00
Brandon Thomas 0f1cf93252 add fal image generation end-to-end (recraft, etc.) (#637) 2025-05-26 21:33:21 -04:00
bflatastic 4a10581e9a fix drag and drop logic for 3D and 2D tabs in PageEditor component 2025-05-26 23:31:47 +07:00
bflatastic 447a70b890 differentiate tabs to drop amd set up place for drag and drop on 2d 2025-05-26 23:26:13 +07:00
Brandon Thomas f4e998d945 cleanup deps (#635) 2025-05-26 01:38:27 -04:00
Brandon Thomas a954859ac2 cleanup dead dependencies 2025-05-25 23:55:15 -04:00
Brandon Thomas 2d8e835401 standardize actix versions in root Cargo.toml 2025-05-25 23:47:12 -04:00
Brandon Thomas c23380af90 codegen db 2025-05-25 23:22:10 -04:00
Brandon Thomas c34f75d071 cleanup hakari packages 2025-05-25 23:15:32 -04:00
Brandon Thomas 8a886e9033 cleanup scripts (2) 2025-05-25 22:57:01 -04:00
Brandon Thomas 3101df8211 cleanup scripts 2025-05-25 22:53:40 -04:00
Brandon Thomas a4c264ebbf cleanup packages, fixup storyteller-web 2025-05-25 22:50:45 -04:00
Brandon Thomas 3d807fdeb2 refactor database crates 2025-05-25 22:33:18 -04:00
Brandon Thomas b3a62cce64 refactor packages 2025-05-25 22:09:59 -04:00
Brandon Thomas 42ccc25e4d fix video enqueue 2025-05-25 21:31:56 -04:00
Brandon Thomas 6595518b35 add detailed build info in an about settings pane (#634) 2025-05-25 14:45:50 -04:00
ArEnSc 618cbc8f0e Feature/clean (#633)
Clean up the TS work unify and the app
2025-05-24 23:31:25 -04:00
BFlat b31a152c1a text to image tab (commented out) (#632) 2025-05-24 10:57:28 +07:00
BFlat 0e2169b902 updated topbar menu icons with tooltip (#631)
- animated on hover
- to accommodate more pages
2025-05-23 23:20:09 +07:00
Michael Chung 0ee9b7c442 fix: remove DialogTitle from DialogEditText 2025-05-23 11:33:48 -04:00
bflatastic 17495276e7 Refactor GalleryModal refresh logic and add refresh button
- Simplified the refresh logic by extracting it into a separate `refreshGallery` function.
- Added a refresh button with a tooltip to allow users to manually refresh the gallery items.
- Updated the useEffect to call the new refresh function when the modal opens, ensuring the gallery is always up-to-date.
2025-05-23 20:47:29 +07:00
bflatastic 2d78fd5a6f library always refreshed on open 2025-05-23 20:39:40 +07:00
BFlat 98e513e917 gallery dnd code refactor (#630)
- cleaner code + reusable image drop function for later 2d image drop
implementation
2025-05-23 09:21:43 +07:00
ArEnScandMichael Chung 97eb1c65ca Fix cors allow all urls for now, since there isn't really a way to ha… (#628)
…ck the app lol.

Co-authored-by: Michael Chung <michaelchung@mac.home>
2025-05-22 18:43:59 -04:00
bflatastic ba82a438dd image not dropping at correct position fix 2025-05-23 02:11:54 +07:00
bflatastic de7b990ec5 bug fix: select mode library modal not closing on X click 2025-05-23 01:35:50 +07:00
BFlat 7b90c59675 library modal signals bug fix (#627)
- fixed weird behavior with library modal not opening
- image drag and drop more accurate position (production drag and drop
still doesnt work - looking into it)
2025-05-23 01:31:49 +07:00
Brandon Thomas 11f1a5f41a sora aspect ratios (#626) 2025-05-22 00:09:59 -04:00
BFlat b92cd58f81 Enhance GalleryModal UX with Filters, Drag-and-Drop, Pagination, and UI Improvements (#625)
**✨ Features & Enhancements:**
- GalleryModal Filters: Replaced tab-based navigation with a more
scalable filter system.
- Image Fit Toggle: Allows toggling image fit display mode for better
previewing.
- Pagination & Infinite Scroll: Improved item loading performance using
soft refresh and infinite scroll logic.
- Drag-and-drop setup for gallery items
- Prevented dragging for video items
- Added support for image drop in PageEditor (3D)
- "My Library" Label Updates
2025-05-21 23:29:06 +07:00
Brandon Thomas 914c26b4c6 classify gateway timeout 2025-05-21 11:04:32 -04:00
Brandon Thomas fe8939a0f6 incomplete runwayml implementation (pausing for now) (#624) 2025-05-20 23:39:34 -04:00
Tanish Manku c809ff88fa Add save/load support for character poses (#623) 2025-05-20 20:13:04 -04:00
bflatastic 86675568a8 Update modal components for improved drag functionality and styling
- Increased the height of the drag handle in AssetModal and GalleryModal for better usability.
- Adjusted z-index properties for elements within AssetModal to ensure proper layering.
- Enhanced the modal's drag behavior to allow overflow beyond viewport edges, improving user experience when repositioning.
2025-05-21 02:32:20 +07:00
bflatastic a8927cc78d Refactor expand button in modal component for improved styling and functionality
- Updated the ExpandButton component to accept size props for better responsiveness.
- Replaced SVG icons with FontAwesome icons for a more consistent design.
- Adjusted the GalleryModal layout to include a gap between buttons for enhanced usability.
2025-05-21 02:17:17 +07:00
bflatastic 0e8dd19ecf Enhance modal functionality with draggable support and expand button
- Added draggable functionality to the modal, allowing users to reposition it.
- Introduced an expand button to toggle full-screen mode for the modal.
- Updated modal backdrop to support interaction options and improved styling.
- Adjusted AssetModal and GalleryModal components to utilize new modal features.
2025-05-21 02:06:51 +07:00
bflatastic b32a2e79b9 promptbox height bug fix 2025-05-20 22:47:38 +07:00
bflatastic a662c5809d lightbox videos fix 2025-05-20 22:31:27 +07:00
Michael Chung 7498720674 Fixing the gallery modal to show videos and images. 2025-05-20 11:14:36 -04:00
ArEnScandMichael Chung c8a4490b52 Feature/add to scene video 2 (#621)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-20 09:22:46 -04:00
Brandon Thomas c4e30e6dc7 swap out the background (#620) 2025-05-19 23:01:33 -04:00
ArEnScandMichael Chung 12070f14ad Feature/add to scene video 2 (#619)
Adding videos to scene via signal.

---------

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-19 02:19:31 -04:00
BFlat 77a103de06 gallery videos tab (#617)
- 16/9 ratio grid items
- lightbox for videos will only show download button
2025-05-19 10:58:34 +07:00
BFlat 82f5805f24 image-to-video generation page with background gallery and UI components (#616) 2025-05-19 09:51:52 +07:00
f7c9ebfaa3 Feature/video addition (#615)
Co-authored-by: Michael Chung <michaelchung@Michaels-MacBook-Pro-2.local>
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-18 20:38:12 -04:00
Brandon Thomas 76cd00cc13 add fal hunyuan image-to-3d (#614) 2025-05-18 15:37:55 -04:00
Brandon Thomas cb64f45b48 refactor artcraft crates to organize by service (module reorg) (#613) 2025-05-18 00:53:00 -04:00
Brandon Thomas 6751ab0b60 fal video integration (#612) 2025-05-17 19:47:40 -04:00
ArEnScandMichael Chung 9827a3efc5 Feature/add scene (#611)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-15 22:09:38 -04:00
Brandon Thomas 93b53d84cd remove some dead code + fix missing "dev" assets (#610) 2025-05-15 18:32:03 -04:00
Brandon Thomas a7794f6338 don't need to build for editor2d anymore 2025-05-15 17:20:56 -04:00
ArEnScandMichael Chung ef03e292c3 Add scene to 2d Lightbox modal. (#609)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-14 12:11:19 -04:00
BFlat 0e8c1fa3e7 Update landing2 mobile videos fix 2025-05-14 12:07:07 +07:00
BFlat af08834de7 artcraft landing 2 (#608) 2025-05-14 02:28:05 +07:00
Brandon Thomas e728806c04 more csp fixes 2025-05-13 14:18:29 -04:00
Brandon Thomas 3e605e7b18 add google analytics to CSP 2025-05-13 14:03:31 -04:00
Brandon Thomas d244524abc fal api integration in tauri + frontend controls + background removal (#607) 2025-05-13 13:12:49 -04:00
bflatastic 8e455e27a9 fixed: Artcraft login in onboarding shown when already logged into Artcraft 2025-05-13 11:55:44 +07:00
Brandon Thomas e0df1644a5 switch order of 2d/3d 2025-05-12 19:11:29 -04:00
Brandon Thomas 736d033ebf status alerts on 2d side 2025-05-12 18:54:42 -04:00
Brandon Thomas af3f0baa63 handle sora logged out state after app starts 2025-05-12 18:44:54 -04:00
2075 changed files with 16708 additions and 31753 deletions
@@ -48,7 +48,7 @@
"name": "maybe_creator_anonymous_visitor_token: tokens::tokens::anonymous_visitor_tracking::AnonymousVisitorTrackingToken",
"type_info": {
"type": "VarString",
"flags": "BINARY",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
Generated
+432 -2900
View File
File diff suppressed because it is too large Load Diff
+55 -32
View File
@@ -4,13 +4,6 @@ resolver = "2"
# This block defines the workspace members.
# When we add a library or binary crate, it needs an entry here.
members = [
# Types: common strongly-typed tokens, ids, payloads, etc.
"crates/_types/composite_identifiers",
"crates/_types/enums",
"crates/_types/enums_public",
"crates/_types/reusable_types", # TODO(bt, 2022-12-21): Deprecate
"crates/_types/tokens",
# CLI tools for operations, etc.
"crates/cli/devenv/dev-database-seed",
"crates/cli/devenv/dev-upload-media-file",
@@ -30,13 +23,16 @@ members = [
"crates/lib/actix_cors_configs",
"crates/lib/actix_helpers",
"crates/lib/api_clients/cloud_storage",
"crates/lib/api_clients/fal_client",
"crates/lib/api_clients/google_drive_common",
"crates/lib/api_clients/google_sign_in",
"crates/lib/api_clients/openai_sora_client",
"crates/lib/api_clients/runwayml_client",
"crates/lib/api_clients/storyteller_client",
"crates/lib/api_clients/thumbnail_generator",
"crates/lib/bootstrap",
"crates/lib/bucket_paths",
"crates/lib/build_metadata",
"crates/lib/caching/memory_caching",
"crates/lib/caching/memory_store",
"crates/lib/caching/redis_caching",
@@ -50,10 +46,10 @@ members = [
"crates/lib/deprecated/http_server_common", # TODO(bt, 2023-03-20): Move useful pieces to actix_helpers
"crates/lib/dimensions",
"crates/lib/easyenv", # NB: Vendored version of https://github.com/echelon/easyenv.rs that we'll improve.
"crates/lib/elasticsearch_schema",
"crates/lib/email_sender",
"crates/lib/errors",
"crates/lib/files/archives",
"crates/lib/files/file_to_base64_url",
"crates/lib/files/filesys",
"crates/lib/files/hashing",
"crates/lib/files/images",
@@ -63,16 +59,13 @@ members = [
"crates/lib/idempotency",
"crates/lib/image_algorithms",
"crates/lib/jobs_common",
"crates/lib/jwt_light",
"crates/lib/logging",
"crates/lib/machine_learning/ml_models",
"crates/lib/machine_learning/ml_weights_registry",
"crates/lib/machine_learning/prompt_engineering",
"crates/lib/markdown",
"crates/lib/migration",
"crates/lib/mysql_queries",
"crates/lib/password",
"crates/lib/redis_common",
"crates/lib/redis_schema",
"crates/lib/server_environment",
"crates/lib/shared_service_components",
"crates/lib/sqlx_mysql_helpers",
@@ -84,7 +77,21 @@ members = [
"crates/lib/user_input_common",
"crates/lib/web_scrapers",
#"crates/lib/api_clients/fakeyou_client",
#"crates/lib/sqlite_queries",
#"crates/lib/machine_learning/ml_models",
# Schema (database): queries, database layouts, Redis keys, etc.
"crates/schema/database/elasticsearch_schema",
"crates/schema/database/mysql_queries",
"crates/schema/database/redis_common",
"crates/schema/database/redis_schema",
#"crates/schema/database/sqlite_queries",
# Schema (public): common strongly-typed tokens, ids, payloads, etc.
"crates/schema/public/composite_identifiers",
"crates/schema/public/enums",
"crates/schema/public/enums_public",
"crates/schema/public/reusable_types", # TODO(bt, 2022-12-21): Deprecate
"crates/schema/public/tokens",
# Services: API servers, jobs, workers, etc.
"crates/service/job/analytics_job",
@@ -105,19 +112,20 @@ members = [
"crates/service/plugins/user_traits_component",
# Vendored code
"crates/vendor/candle/candle-core",
"crates/vendor/candle/candle-datasets",
"crates/vendor/candle/candle-examples",
"crates/vendor/candle/candle-flash-attn",
"crates/vendor/candle/candle-kernels",
"crates/vendor/candle/candle-metal-kernels",
"crates/vendor/candle/candle-nn",
"crates/vendor/candle/candle-onnx",
"crates/vendor/candle/candle-pyo3",
"crates/vendor/candle/candle-transformers",
"crates/vendor/candle/tensor-tools",
"crates/vendor/limitation",
"crates/vendor/tauri-plugin-http",
# NB(bt,2025-05-25): Too many failures on Mac. Disable for now.
#"crates/vendor/candle/candle-core",
#"crates/vendor/candle/candle-datasets",
#"crates/vendor/candle/candle-examples",
#"crates/vendor/candle/candle-flash-attn",
#"crates/vendor/candle/candle-kernels",
#"crates/vendor/candle/candle-metal-kernels",
#"crates/vendor/candle/candle-nn",
#"crates/vendor/candle/candle-onnx",
#"crates/vendor/candle/candle-pyo3",
#"crates/vendor/candle/candle-transformers",
#"crates/vendor/candle/tensor-tools",
# NB(bt,2023-12-12): Newrelic keeps breaking with upgrades. Kill for now.
#"crates/vendor/newrelic-telemetry-sdk-rust",
@@ -157,35 +165,50 @@ workspace-build-acceleration = { path = "crates/workspace-build-acceleration" }
# External libs
# Centrally pinned versions so that we can uniformly upgrade/downgrade versions across the workspace
# If a crate needs to use an older/newer version, the dependency should be managed in that crate's own Cargo.toml
anyhow = "1.0.97"
actix = "0.13.5"
actix-cors = "0.6.4" # TODO(bt): Carefully upgrade this. CORS broke last time.
actix-http = "3.7.0"
actix-multipart = "0.6.2"
actix-rt = "2.10.0"
actix-service = "2.0.3"
actix-utils = "3.0.1"
actix-web = "4.7.0" # TODO(bt): We're behind and need to bump versions.
actix-web-actors = "4.3.1+deprecated"
actix-web-lab = "0.20.2"
anyhow = "1.0.98"
base64 = "0.22.1"
chrono = { version = "0.4.40", features = ["serde"] }
chrono = { version = "0.4.41", features = ["serde"] }
cookie = "0.18.1"
futures = { version = "0.3.31" }
hex-literal = "1.0.0"
indexmap = "2.9.0"
log = "0.4.27"
once_cell = "1.21.3"
rand = "0.9.0"
rand = "0.9.1"
regex = "1.11.1"
reqwest = { version = "0.12.15", features = ["json", "cookies"] }
serde = { version = "1.0.219", features = ["derive"] }
serde_derive = "1.0.219"
serde_json = "1.0.140"
serde_yaml = "0.9.1"
serde_yaml = "0.9.34+deprecated"
sha3 = "0.10.8"
tauri = { version = "2.4.1", features = ["unstable", "devtools"] } # NB: "devtools" prevents App Store distribution
thiserror = "2.0.12"
toml = "0.8.20"
tokio = { version = "1.45.1", features = ["macros"] }
toml = "0.8.22"
url = { version = "2.5.4", features = ["serde"] }
# Internal libs
build_metadata = { path = "crates/lib/build_metadata" }
crockford = { path = "crates/lib/data/crockford" }
enums = { path = "crates/_types/enums" }
enums_public = { path = "crates/_types/enums_public" }
enums = { path = "crates/schema/public/enums" }
enums_public = { path = "crates/schema/public/enums_public" }
errors = { path = "crates/lib/errors" }
fal_client = { path = "crates/lib/api_clients/fal_client" }
filesys = { path = "crates/lib/files/filesys" }
idempotency = { path = "crates/lib/idempotency" }
images = { path = "crates/lib/files/images" }
jwt_light = { path = "crates/lib/jwt_light" }
memory_caching = { path = "crates/lib/caching/memory_caching" }
memory_store = { path = "crates/lib/caching/memory_store" }
mimetypes = { path = "crates/lib/files/mimetypes" }
@@ -194,4 +217,4 @@ primitives = { path = "crates/lib/data/primitives" }
shared_service_components = { path = "crates/lib/shared_service_components" }
storyteller_client = { path = "crates/lib/api_clients/storyteller_client" }
subprocess_common = { path = "crates/lib/subprocess_common" }
tokens = { path = "crates/_types/tokens" }
tokens = { path = "crates/schema/public/tokens" }
@@ -16,23 +16,23 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
bucket_paths = { path = "../../../lib/bucket_paths" }
cloud_storage = { path = "../../../lib/api_clients/cloud_storage" }
config = { path = "../../../lib/deprecated/config" }
easyenv = { path = "../../../lib/easyenv" }
enums = { path = "../../../_types/enums" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
filesys = { path = "../../../lib/files/filesys" }
hashing = { path = "../../../lib/files/hashing" }
idempotency = { path = "../../../lib/idempotency" }
mimetypes = { path = "../../../lib/files/mimetypes" }
mysql_queries = { path = "../../../lib/mysql_queries" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
tokens = { path = "../../../schema/public/tokens" }
# External
log = "0.4.14"
@@ -16,20 +16,20 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
bucket_paths = { path = "../../../lib/bucket_paths" }
cloud_storage = { path = "../../../lib/api_clients/cloud_storage" }
config = { path = "../../../lib/deprecated/config" }
easyenv = { path = "../../../lib/easyenv" }
enums = { path = "../../../_types/enums" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
filesys = { path = "../../../lib/files/filesys" }
mimetypes = { path = "../../../lib/files/mimetypes" }
mysql_queries = { path = "../../../lib/mysql_queries" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
tokens = { path = "../../../schema/public/tokens" }
# External
clap = "4.4.8"
+2 -3
View File
@@ -16,14 +16,13 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
errors = { path = "../../../lib/errors" }
web_scrapers = { path = "../../../lib/web_scrapers" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
tokio = { version = "1.38.0", features = ["macros"] }
+4 -4
View File
@@ -17,15 +17,15 @@ normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
workspace-build-acceleration = { workspace = true }
# Internal
datetimes = { path = "../../../lib/data/datetimes" }
easyenv = { path = "../../../lib/easyenv" }
enums = { path = "../../../_types/enums" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
mysql_queries = { path = "../../../lib/mysql_queries" }
tokens = { path = "../../../_types/tokens" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
tokens = { path = "../../../schema/public/tokens" }
# External
anyhow = "1.0.86"
+3 -7
View File
@@ -16,20 +16,16 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
easyenv = { path = "../../../lib/easyenv" }
errors = { path = "../../../lib/errors" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
anyhow = "1.0.86"
# TODO/NB: Commenting out clap4 as it prevents windows builds
# NB: clap4 = { package = "clap", version = "4.0" }
dotenv = "0.15.0"
log = "0.4.14"
# Serialization
serde = "1.0.125"
serde_derive = "1.0.125"
+6 -6
View File
@@ -16,17 +16,17 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
bucket_paths = { path = "../../../lib/bucket_paths" }
easyenv = { path = "../../../lib/easyenv" }
enums = { path = "../../../_types/enums" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
mysql_queries = { path = "../../../lib/mysql_queries" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
tokens = { path = "../../../schema/public/tokens" }
# External
clap = "4.4.8"
@@ -16,20 +16,20 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
bucket_paths = { path = "../../../lib/bucket_paths" }
config = { path = "../../../lib/deprecated/config" }
easyenv = { path = "../../../lib/easyenv" }
elasticsearch_schema = { path = "../../../lib/elasticsearch_schema" }
enums = { path = "../../../_types/enums" }
elasticsearch_schema = { path = "../../../schema/database/elasticsearch_schema" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
mysql_queries = { path = "../../../lib/mysql_queries" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
primitives = { path = "../../../lib/data/primitives" }
storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
tokens = { path = "../../../schema/public/tokens" }
# External
clap = "4.4.8"
@@ -16,14 +16,14 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile yytimes)
workspace-build-acceleration = { workspace = true }
# Internal
config = { path = "../../../lib/deprecated/config" }
easyenv = { path = "../../../lib/easyenv" }
errors = { path = "../../../lib/errors" }
# Workspace hack (faster compile yytimes)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
log = "0.4.20"
once_cell = "1.18.0"
@@ -16,13 +16,13 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
errors = { path = "../../../lib/errors" }
logging = { path = "../../../lib/logging" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = { version = "4.4.8", features = ["derive"] }
walkdir = "2.4.0"
@@ -16,21 +16,21 @@ path = "src/main.rs"
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
bucket_paths = { path = "../../../lib/bucket_paths" }
cloud_storage = { path = "../../../lib/api_clients/cloud_storage" }
config = { path = "../../../lib/deprecated/config" }
easyenv = { path = "../../../lib/easyenv" }
enums = { path = "../../../_types/enums" }
enums = { path = "../../../schema/public/enums" }
errors = { path = "../../../lib/errors" }
filesys = { path = "../../../lib/files/filesys" }
hashing = { path = "../../../lib/files/hashing" }
mysql_queries = { path = "../../../lib/mysql_queries" }
mysql_queries = { path = "../../../schema/database/mysql_queries" }
storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
tokens = { path = "../../../schema/public/tokens" }
# External
clap = "4.4.8"
+4 -7
View File
@@ -23,15 +23,15 @@ tauri-build = { version = "2.1.1", features = [] }
workspace-build-acceleration = { workspace = true }
# Internal
crockford = { workspace = true }
build_metadata = { workspace = true }
errors = { workspace = true }
fal_client = { workspace = true }
filesys = { workspace = true }
memory_caching = { workspace = true }
memory_store = { workspace = true }
mimetypes = { workspace = true }
openai_sora_client = { workspace = true }
primitives = { workspace = true }
storyteller_client = { workspace = true }
subprocess_common = { workspace = true }
tokens = { workspace = true }
# External
@@ -39,7 +39,6 @@ anyhow = { workspace = true }
base64 = { workspace = true }
bytes = "1.10"
chrono = { workspace = true }
const_format = "0.2.34"
directories = "6.0.0"
dirs = "6.0.0"
futures = "0.3"
@@ -52,7 +51,6 @@ reqwest = { workspace = true, features = ["multipart", "stream"], default-featur
serde = { workspace = true }
serde_derive = { workspace = true }
serde_json = { workspace = true }
serde_yaml = { workspace = true }
tauri = { workspace = true }
tauri-plugin-log = "2.3.1"
tauri-plugin-upload = "2.2.1"
@@ -60,6 +58,7 @@ tempdir = "0.3.7"
tempfile = "3.18.0"
tokio = { version = "1.42", features = ["rt", "rt-multi-thread", "fs"] }
tokio-util = { version = "0.7", features = ["codec"] }
url = { workspace = true }
# NB: We need to vendor this in so that we can access the cookies inside via the private API
tauri-plugin-http = { path = "../../vendor/tauri-plugin-http" }
@@ -70,9 +69,7 @@ discord-rich-presence = { git = "https://github.com/vionya/discord-rich-presence
tauri-plugin-dialog = "2"
[dev-dependencies]
libloading = "0.8.6"
serial_test = "3.2.0"
assertor = "0.0.3"
[profile.dev]
incremental = true # Compile your binary in smaller steps.
@@ -10,19 +10,12 @@
"log:default",
"upload:allow-download",
"upload:default",
"http:allow-fetch",
{
"identifier": "http:default",
"allow": [
{
"url": "https://*.storyteller.ai"
},
{
"url": "https://*.fakeyou.com"
}
],
"deny": [
{
"url": "https://private.tauri.app"
"url": "https://*"
}
]
},
@@ -1,5 +1,5 @@
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use errors::AnyhowResult;
use log::{error, info};
use serde_derive::Serialize;
@@ -1,6 +1,6 @@
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use log::{error, info};
@@ -6,6 +6,9 @@ use std::str::FromStr;
pub type CommandResult<SuccessPayload, ErrType, ErrPayload> =
Result<CommandSuccessResponseWrapper<SuccessPayload>, CommandErrorResponseWrapper<ErrType, ErrPayload>>;
/// Easy to use infallible response.
pub type InfallibleResponse<SuccessPayload> = CommandSuccessResponseWrapper<SuccessPayload>;
/// No inner payloads for this type. Just strings as messages.
pub type SimpleCommandResult = CommandResult<(), (), ()>;
@@ -73,7 +76,7 @@ impl CommandSuccessResponseWrapper<()> {
/// `Serialize`. Rust's `"specialization"`, which would let us account for this, is an unstable, nightly-only
/// feature. So instead we manually mark all types other than `()` with `SerializeMarker` as a hack to
/// distinguish the two.
trait SerializeMarker : Serialize {}
pub trait SerializeMarker : Serialize {}
// NB: () is Serialize! So we can't broadly implement just `T: Serialize`.
impl From<()> for CommandSuccessResponseWrapper<()> {
@@ -0,0 +1,196 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::commands::enqueue::image::handle_fal::handle_fal;
use crate::core::commands::enqueue::image::handle_sora::handle_sora;
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct EnqueueTextToImageRequest {
/// Text prompt for the image generation. Required.
pub prompt: Option<String>,
/// The model to use.
pub model: Option<EnqueueTextToImageModel>,
}
#[derive(Deserialize, Debug, Copy, Clone)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueTextToImageModel {
#[serde(rename = "flux_pro_ultra")]
FluxProUltra,
#[serde(rename = "gpt_image_1")]
GptImage1,
#[serde(rename = "recraft_3")]
Recraft3,
}
#[derive(Serialize)]
pub struct EnqueueTextToImageSuccessResponse {
}
impl SerializeMarker for EnqueueTextToImageSuccessResponse {}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueTextToImageErrorType {
/// Caller didn't specify a model
ModelNotSpecified,
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
/// Fal had an API error
FalError,
}
#[tauri::command]
pub async fn enqueue_text_to_image_command(
request: EnqueueTextToImageRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
sora_creds_manager: State<'_, SoraCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> CommandResult<EnqueueTextToImageSuccessResponse, EnqueueTextToImageErrorType, ()> {
info!("enqueue_text_to_image called");
let result = handle_request(
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
&sora_creds_manager,
&sora_task_queue,
).await;
match result {
Err(err) => {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = EnqueueTextToImageErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
match err {
InternalImageError::NoModelSpecified => {
status = CommandErrorStatus::BadRequest;
error_type = EnqueueTextToImageErrorType::ModelNotSpecified;
error_message = "No model specified for image generation";
}
InternalImageError::NeedsFalApiKey => {
status = CommandErrorStatus::Unauthorized;
error_type = EnqueueTextToImageErrorType::NeedsFalApiKey;
error_message = "You need to set a FAL api key";
},
_ => {}, // Fall-through
}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
Ok(result) => {
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(EnqueueTextToImageSuccessResponse {
}.into())
}
}
}
pub async fn handle_request(
request: EnqueueTextToImageRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
sora_creds_manager: &SoraCredentialManager,
sora_task_queue: &SoraTaskQueue,
) -> Result<(), InternalImageError> {
match request.model {
None => {
return Err(InternalImageError::NoModelSpecified);
}
Some(EnqueueTextToImageModel::FluxProUltra) | Some(EnqueueTextToImageModel::Recraft3) => {
handle_fal(request, fal_creds_manager, fal_task_queue).await?;
},
Some(EnqueueTextToImageModel::GptImage1) => {
handle_sora(request, sora_creds_manager, sora_task_queue).await?;
}
};
Ok(())
}
@@ -0,0 +1,67 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use log::{error, info, warn};
pub async fn handle_fal(
request: EnqueueTextToImageRequest,
fal_creds_manager: &FalCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalImageError> {
let api_key = fal_creds_manager.get_key_required()
.map_err(|err| {
error!("EnqueueTextToImage FAL api key required: {:?}", err);
InternalImageError::NeedsFalApiKey
})?;
let prompt = request.prompt.as_deref().unwrap_or("");
let result = match request.model {
None => {
return Err(InternalImageError::NoModelSpecified);
}
Some(EnqueueTextToImageModel::GptImage1) => {
return Err(InternalImageError::AnyhowError(anyhow!("wrong logic: fal is handling sora images")));
}
Some(EnqueueTextToImageModel::FluxProUltra) => {
info!("enqueue Flux Pro Ultra text-to-image with prompt: {}", prompt);
enqueue_flux_pro_ultra_text_to_image(FluxProUltraTextToImageArgs {
prompt,
api_key: &api_key,
}).await
}
Some(EnqueueTextToImageModel::Recraft3) => {
info!("enqueue Recraft v3 text-to-image with prompt: {}", prompt);
enqueue_recraft3_text_to_image(Recraft3TextToImageArgs {
prompt,
api_key: &api_key,
}).await
}
};
match result {
Ok(enqueued) => {
info!("Successfully enqueued text to image");
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InternalImageError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
}
}
Err(err) => {
error!("Failed to enqueue text to image: {:?}", err);
return Err(InternalImageError::FalError(err));
}
}
Ok(())
}
@@ -0,0 +1,77 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use log::{error, info, warn};
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::recipes::simple_image_gen_with_session_auto_renew::{simple_image_gen_with_session_auto_renew, SimpleImageGenAutoRenewRequest};
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages, SoraImageGenResponse};
use openai_sora_client::sora_error::SoraError;
use std::time::Duration;
const SORA_SIMPLE_IMAGE_GEN_TIMEOUT : Duration = Duration::from_millis(1000 * 30); // 30 seconds
pub async fn handle_sora(
request: EnqueueTextToImageRequest,
sora_creds_manager: &SoraCredentialManager,
sora_task_queue: &SoraTaskQueue,
) -> Result<(), InternalImageError> {
let mut creds = sora_creds_manager
.get_credentials_required()
.map_err(|err| {
error!("EnqueueTextToImage Sora creds required: {:?}", err);
InternalImageError::NeedsFalApiKey
})?;
let credential_updated = maybe_upgrade_or_renew_session(&mut creds)
.await
.map_err(|err| {
error!("Failed to upgrade or renew session: {:?}", err);
err
})?;
if credential_updated {
info!("Storing updated credentials");
sora_creds_manager.set_credentials(&creds)?;
}
let prompt = request.prompt.as_deref().unwrap_or("");
let result =
simple_image_gen_with_session_auto_renew(SimpleImageGenAutoRenewRequest {
prompt: prompt.to_string(),
num_images: NumImages::One,
image_size: ImageSize::Square,
credentials: &creds,
request_timeout: Some(SORA_SIMPLE_IMAGE_GEN_TIMEOUT),
}).await;
let (response, maybe_new_creds) =
match result {
Ok((response, maybe_new_creds)) => (response, maybe_new_creds),
Err(err) => {
return Err(InternalImageError::SoraError(err));
}
};
if let Some(new_creds) = maybe_new_creds {
info!("Storing updated credentials.");
sora_creds_manager.set_credentials(&new_creds)?;
}
info!("New Sora Task ID: {:?} ", response.task_id);
sora_task_queue.insert(&response.task_id)?;
Ok(())
}
@@ -0,0 +1,54 @@
use base64::DecodeError;
use errors::AnyhowError;
use fal_client::error::fal_error_plus::FalErrorPlus;
use openai_sora_client::sora_error::SoraError;
use storyteller_client::error::api_error::ApiError;
#[derive(Debug)]
pub enum InternalImageError {
NoModelSpecified,
NeedsFalApiKey,
NeedsSoraCredentials,
FalError(FalErrorPlus),
SoraError(SoraError),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
DecodeError(DecodeError),
IoError(std::io::Error),
}
impl From<AnyhowError> for InternalImageError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InternalImageError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<SoraError> for InternalImageError {
fn from(value: SoraError) -> Self {
Self::SoraError(value)
}
}
impl From<ApiError> for InternalImageError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
}
}
impl From<DecodeError> for InternalImageError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InternalImageError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
@@ -0,0 +1,4 @@
pub mod enqueue_text_to_image_command;
pub mod handle_fal;
pub mod handle_sora;
pub mod internal_image_error;
@@ -0,0 +1 @@
pub mod image;
@@ -0,0 +1,39 @@
use crate::core::commands::command_response_wrapper::{CommandResult, InfallibleResponse, SerializeMarker};
use crate::core::state::os_platform::OsPlatform;
use crate::services::fal::commands::fal_background_removal_command::FalBackgroundRemovalSuccessResponse;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use chrono::{DateTime, Utc};
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::ImageReader;
use log::info;
use serde_derive::Serialize;
use std::io::Cursor;
#[derive(Debug, Serialize)]
pub struct BuildInfoResponse {
pub build_timestamp: DateTime<Utc>,
pub git_commit_id: Option<String>,
pub git_commit_short_id: Option<String>,
pub git_commit_timestamp: Option<DateTime<Utc>>,
}
impl SerializeMarker for BuildInfoResponse {}
#[tauri::command]
pub fn get_build_info_command() -> InfallibleResponse<BuildInfoResponse> {
info!("get_build_info_command called...");
let build_timestamp = build_metadata::build_timestamp();
let git_commit_id = build_metadata::git_commit_id();
let git_commit_short_id = build_metadata::git_commit_short_id();
let git_commit_timestamp = build_metadata::git_commit_timestamp();
BuildInfoResponse {
build_timestamp,
git_commit_id: git_commit_id.map(|s| s.to_string()),
git_commit_short_id: git_commit_short_id.map(|s| s.to_string()),
git_commit_timestamp,
}.into()
}
@@ -1,5 +1,6 @@
pub mod app_preferences;
pub mod command_response_wrapper;
pub mod enqueue;
pub mod flip_image;
pub mod get_build_info_command;
pub mod platform_info_command;
pub mod sora;
@@ -1,4 +1,4 @@
use crate::state::os_platform::OsPlatform;
use crate::core::state::os_platform::OsPlatform;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
@@ -1,4 +1,3 @@
pub mod notification_event;
pub mod sendable_event_error;
pub mod sendable_event_trait;
pub mod sora;
pub mod sendable_event_trait;
@@ -1,5 +1,5 @@
use log::info;
use crate::events::sendable_event_error::SendableEventError;
use crate::core::events::sendable_event_error::SendableEventError;
use serde::Serialize;
use tauri::{AppHandle, Emitter};
+6
View File
@@ -0,0 +1,6 @@
pub mod commands;
pub mod events;
pub mod state;
pub mod threads;
pub mod utils;
pub mod windows;
@@ -1,6 +1,6 @@
use crate::state::app_preferences::app_preferences_serializable::AppPreferencesSerializable;
use crate::state::app_preferences::preferred_download_directory::{PreferredDownloadDirectory, SystemDownloadDirectory};
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::app_preferences::app_preferences_serializable::AppPreferencesSerializable;
use crate::core::state::app_preferences::preferred_download_directory::{PreferredDownloadDirectory, SystemDownloadDirectory};
use crate::core::state::data_dir::app_data_root::AppDataRoot;
#[derive(Clone)]
pub struct AppPreferences {
@@ -1,5 +1,5 @@
use crate::state::app_preferences::app_preferences::AppPreferences;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::app_preferences::app_preferences::AppPreferences;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use memory_store::clone_cell::CloneCell;
pub type AppPreferencesManager = CloneCell<AppPreferences>;
@@ -1,6 +1,6 @@
use crate::state::app_preferences::app_preferences::AppPreferences;
use crate::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::app_preferences::app_preferences::AppPreferences;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
@@ -58,8 +58,8 @@ impl Display for SystemDownloadDirectory {
#[cfg(test)]
mod tests {
use crate::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::state::app_preferences::preferred_download_directory::SystemDownloadDirectory;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::app_preferences::preferred_download_directory::SystemDownloadDirectory;
use std::str::FromStr;
mod json {
@@ -1,4 +1,4 @@
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use chrono::{DateTime, Local};
use std::path::{Path, PathBuf};
@@ -1,4 +1,4 @@
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use std::path::{Path, PathBuf};
#[derive(Clone)]
@@ -40,4 +40,8 @@ impl AppCredentialsDir {
pub fn get_storyteller_session_cookie_file_path(&self) -> PathBuf {
self.path.join("artcraft_session.txt")
}
pub fn get_fal_api_key_file_path(&self) -> PathBuf {
self.path.join("fal_api_key.txt")
}
}
@@ -1,12 +1,12 @@
use crate::state::data_dir::app_assets_dir::AppAssetsDir;
use crate::state::data_dir::app_credentials_dir::AppCredentialsDir;
use crate::state::data_dir::app_downloads_dir::AppDownloadsDir;
use crate::state::data_dir::app_settings_dir::AppSettingsDir;
use crate::state::data_dir::app_weights_dir::AppWeightsDir;
use crate::state::data_dir::temporary_dir::TemporaryDir;
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::state::expanduser::expanduser;
use crate::state::os_platform::OsPlatform;
use crate::core::state::data_dir::app_assets_dir::AppAssetsDir;
use crate::core::state::data_dir::app_credentials_dir::AppCredentialsDir;
use crate::core::state::data_dir::app_downloads_dir::AppDownloadsDir;
use crate::core::state::data_dir::app_settings_dir::AppSettingsDir;
use crate::core::state::data_dir::app_weights_dir::AppWeightsDir;
use crate::core::state::data_dir::temporary_dir::TemporaryDir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::expanduser::expanduser;
use crate::core::state::os_platform::OsPlatform;
use anyhow::anyhow;
use directories::UserDirs;
use std::path::{Path, PathBuf};
@@ -1,4 +1,4 @@
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use std::path::{Path, PathBuf};
#[derive(Clone)]
@@ -1,5 +1,5 @@
use crate::state::app_preferences::app_preferences::AppPreferences;
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::app_preferences::app_preferences::AppPreferences;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use errors::AnyhowResult;
use serde::Serialize;
use std::fs::OpenOptions;
@@ -1,4 +1,4 @@
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use std::path::{Path, PathBuf};
#[derive(Clone)]
@@ -1,4 +1,4 @@
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use std::path::{Path, PathBuf};
use tempdir::TempDir;
use tempfile::{Builder, NamedTempFile};
@@ -35,4 +35,17 @@ impl TemporaryDir {
.tempfile_in(&self.path)?;
Ok(tempfile)
}
pub fn new_named_temp_file_with_extension(&self, extension: &str) -> Result<NamedTempFile, std::io::Error> {
let extension = if extension.starts_with(".") {
extension.to_string()
} else {
format!(".{}", extension)
};
let tempfile = Builder::new()
.prefix("temp_")
.suffix(&extension)
.tempfile_in(&self.path)?;
Ok(tempfile)
}
}
@@ -43,7 +43,7 @@ pub fn home_dir() -> io::Result<PathBuf> {
#[cfg(test)]
mod tests {
use crate::state::expanduser::expanduser;
use crate::core::state::expanduser::expanduser;
use std::env;
use std::path::PathBuf;
@@ -1,5 +1,5 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::windows::main_window::constants::MAIN_WINDOW_NAME;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::windows::main_window::constants::MAIN_WINDOW_NAME;
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
use std::fs::OpenOptions;
@@ -1,6 +1,6 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use crate::windows::main_window::constants::MAIN_WINDOW_NAME;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::best_window_size_heuristic::best_window_size_heuristic;
use crate::core::windows::main_window::constants::MAIN_WINDOW_NAME;
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
use std::fs::OpenOptions;
@@ -4,6 +4,4 @@ pub mod data_dir;
pub mod expanduser;
pub mod main_window_position;
pub mod main_window_size;
pub mod os_platform;
pub mod sora;
pub mod storyteller;
pub mod os_platform;
@@ -1,11 +1,11 @@
use crate::state::app_startup_time::AppStartupTime;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::threads::main_window_thread::persist_storyteller_cookies_task::persist_storyteller_cookies_task;
use crate::threads::main_window_thread::persist_window_position_task::persist_window_position_task;
use crate::threads::main_window_thread::persist_window_resize_task::persist_window_resize_task;
use crate::core::state::app_startup_time::AppStartupTime;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::main_window_position::MainWindowPosition;
use crate::core::state::main_window_size::MainWindowSize;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use crate::core::threads::main_window_thread::persist_storyteller_cookies_task::persist_storyteller_cookies_task;
use crate::core::threads::main_window_thread::persist_window_position_task::persist_window_position_task;
use crate::core::threads::main_window_thread::persist_window_resize_task::persist_window_resize_task;
use errors::AnyhowResult;
use log::{error, info};
use memory_store::clone_slot::CloneSlot;
@@ -1,6 +1,6 @@
use crate::state::app_startup_time::AppStartupTime;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::core::state::app_startup_time::AppStartupTime;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use chrono::TimeDelta;
use errors::AnyhowResult;
@@ -1,7 +1,7 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::main_window_position::MainWindowPosition;
use crate::core::state::main_window_size::MainWindowSize;
use crate::core::utils::best_window_size_heuristic::best_window_size_heuristic;
use errors::AnyhowResult;
use log::info;
use memory_store::clone_slot::CloneSlot;
@@ -1,6 +1,6 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_size::MainWindowSize;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::main_window_size::MainWindowSize;
use crate::core::utils::best_window_size_heuristic::best_window_size_heuristic;
use errors::AnyhowResult;
use log::info;
use memory_store::clone_slot::CloneSlot;
@@ -0,0 +1,2 @@
pub mod discord_presence_thread;
pub mod main_window_thread;
@@ -0,0 +1,29 @@
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use errors::AnyhowResult;
use storyteller_client::media_files::get_media_file::get_media_file;
use storyteller_client::utils::api_host::ApiHost;
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
// TODO: Better more concrete error handling
pub async fn download_media_file_to_temp_dir(app_data_root: &AppDataRoot, token: &MediaFileToken) -> AnyhowResult<NamedTempFile> {
let response = get_media_file(&ApiHost::Storyteller, token).await?;
let media_file_url = &response.media_file.media_links.cdn_url;
let extension_with_dot = get_url_file_extension(media_file_url)
.map(|ext| format!(".{}", ext))
.unwrap_or_else(|| ".png".to_string()); // TODO: Better default extension passed by caller as heuristic.
//let filename = format!("{}{}", response.media_file.token.as_str(), extension_with_dot);
//let filename = app_data_root.downloads_dir().path().join(&filename);
let mut file = app_data_root.temp_dir().new_named_temp_file_with_extension(&extension_with_dot)?;
simple_http_download_to_tempfile(&media_file_url, &mut file).await?;
Ok(file) // NB: Must return TempFile to not drop / delete it
}
@@ -0,0 +1,20 @@
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use errors::AnyhowResult;
use std::io::Write;
use tempfile::NamedTempFile;
use url::Url;
pub async fn download_url_to_temp_dir(url: &str, app_data_root: &AppDataRoot) -> AnyhowResult<NamedTempFile> {
let url = Url::parse(&url)?;
let response = reqwest::get(url.clone()).await?;
let response_bytes = response.bytes().await?;
let ext = url.path().split(".").last().unwrap_or("png");
let mut temp_file = app_data_root.temp_dir().new_named_temp_file_with_extension(ext)?;
temp_file.write_all(&response_bytes)?;
Ok(temp_file)
}
@@ -1,7 +1,7 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use crate::utils::large_file_transfer::error::Error;
use crate::utils::large_file_transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use crate::core::utils::large_file_transfer::error::Error;
use crate::core::utils::large_file_transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use futures::stream::FuturesUnordered;
use futures::StreamExt;
use log::info;
@@ -1,7 +1,7 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use crate::utils::large_file_transfer::error::Error;
use crate::utils::large_file_transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use crate::core::utils::large_file_transfer::error::Error;
use crate::core::utils::large_file_transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use futures::stream::FuturesUnordered;
use futures::StreamExt;
use reqwest::header::CONTENT_LENGTH;
@@ -1,9 +1,12 @@
pub mod best_window_size_heuristic;
pub mod clear_all_webview_cookies;
pub mod download_media_file_to_temp_dir;
pub mod download_url_to_temp_dir;
pub mod get_url_file_extension;
pub mod image;
pub mod large_file_transfer;
pub mod object_store;
pub mod save_base64_image_to_temp_dir;
pub mod simple_http_download;
pub mod sora;
pub mod simple_http_download_to_tempfile;
pub mod webview_unsafe;
@@ -0,0 +1,25 @@
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use errors::AnyhowResult;
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use std::io::Write;
use tempfile::NamedTempFile;
// TODO: Retain error types (io/decode)
pub async fn save_base64_image_to_temp_dir(app_data_root: &AppDataRoot, base64_image: String) -> AnyhowResult<NamedTempFile> {
let bytes = BASE64_STANDARD.decode(base64_image)?;
let extension = MimetypeInfo::get_for_bytes(&bytes)
.map(|info| info.file_extension())
.flatten()
.map(|ext| ext.extension_with_period().to_string())
.unwrap_or_else(|| ".png".to_string());
let mut file = app_data_root.temp_dir().new_named_temp_file_with_extension(&extension)?;
file.write_all(bytes.as_ref())?;
Ok(file) // NB: Must return TempFile to not drop / delete it
}
@@ -0,0 +1,26 @@
use errors::AnyhowResult;
use reqwest::Client;
use reqwest::Url;
use std::io::Write;
use tempfile::NamedTempFile;
const USER_AGENT: &str = "storyteller-client/1.0";
/// Downloads a (binary) file to a filesystem path. Good for images, etc. Not great for large files.
pub async fn simple_http_download_to_tempfile(url: &Url, temp_file: &mut NamedTempFile) -> AnyhowResult<()> {
let client = Client::builder()
.gzip(true)
.build()?;
let response = client.get(url.clone()) // NB: No IntoUrl for &Url.
.header("User-Agent", USER_AGENT)
.header("Accept", "*/*")
.send()
.await?;
let bytes = response.bytes().await?;
temp_file.write_all(&bytes)?;
Ok(())
}
@@ -0,0 +1 @@
pub mod main_window;
+58 -27
View File
@@ -1,32 +1,38 @@
pub mod commands;
pub mod events;
pub mod state;
pub mod threads;
pub mod utils;
pub mod windows;
pub mod core;
pub mod services;
use tauri::Manager;
use crate::commands::app_preferences::get_app_preferences_command::get_app_preferences_command;
use crate::commands::app_preferences::update_app_preference_command::update_app_preferences_command;
use crate::commands::flip_image::flip_image;
use crate::commands::platform_info_command::platform_info_command;
use crate::commands::sora::check_sora_session_command::check_sora_session_command;
use crate::commands::sora::open_sora_login_command::open_sora_login_command;
use crate::commands::sora::sora_image_generation_command::sora_image_generation_command;
use crate::commands::sora::sora_image_remix_command::sora_image_remix_command;
use crate::commands::sora::sora_logout_command::sora_logout_command;
use crate::state::app_preferences::app_preferences_manager::load_app_preferences_or_default;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::state::sora::sora_task_queue::SoraTaskQueue;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::threads::discord_presence_thread::discord_presence_thread;
use crate::threads::main_window_thread::main_window_thread::main_window_thread;
use crate::threads::sora_task_polling_thread::sora_task_polling_thread;
use crate::utils::webview_unsafe::webview_unsafe_for_app;
use crate::core::commands::app_preferences::get_app_preferences_command::get_app_preferences_command;
use crate::core::commands::app_preferences::update_app_preference_command::update_app_preferences_command;
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::enqueue_text_to_image_command;
use crate::core::commands::flip_image::flip_image;
use crate::core::commands::get_build_info_command::get_build_info_command;
use crate::core::commands::platform_info_command::platform_info_command;
use crate::core::state::app_preferences::app_preferences_manager::load_app_preferences_or_default;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::main_window_position::MainWindowPosition;
use crate::core::state::main_window_size::MainWindowSize;
use crate::core::threads::discord_presence_thread::discord_presence_thread;
use crate::core::threads::main_window_thread::main_window_thread::main_window_thread;
use crate::core::utils::webview_unsafe::webview_unsafe_for_app;
use crate::services::fal::commands::fal_background_removal_command::fal_background_removal_command;
use crate::services::fal::commands::fal_hunyuan_image_to_3d_command::fal_hunyuan_image_to_3d_command;
use crate::services::fal::commands::fal_kling_image_to_video_command::fal_kling_image_to_video_command;
use crate::services::fal::commands::get_fal_api_key_command::get_fal_api_key_command;
use crate::services::fal::commands::set_fal_api_key_command::set_fal_api_key_command;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::fal::threads::fal_task_polling_thread::fal_task_polling_thread;
use crate::services::sora::commands::check_sora_session_command::check_sora_session_command;
use crate::services::sora::commands::open_sora_login_command::open_sora_login_command;
use crate::services::sora::commands::sora_image_generation_command::sora_image_generation_command;
use crate::services::sora::commands::sora_image_remix_command::sora_image_remix_command;
use crate::services::sora::commands::sora_logout_command::sora_logout_command;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::sora::threads::sora_task_polling_thread::sora_task_polling_thread;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use tauri_plugin_dialog;
use tauri_plugin_http;
@@ -47,15 +53,23 @@ pub fn run() {
println!("Attempting to read existing artcraft credentials...");
let storyteller_creds_manager = StorytellerCredentialManager::initialize_from_disk_infallible(&app_data_root);
let storyteller_creds_manager_2 = storyteller_creds_manager.clone();
let storyteller_creds_manager_3 = storyteller_creds_manager.clone();
println!("Attempting to read existing credentials...");
println!("Attempting to read existing sora credentials...");
let sora_creds_manager = SoraCredentialManager::initialize_from_disk_infallible(&app_data_root);
let sora_creds_manager_2 = sora_creds_manager.clone();
println!("Attempting to read existing fal credentials...");
let fal_creds_manager = FalCredentialManager::initialize_from_disk_infallible(&app_data_root);
let fal_creds_manager_2 = fal_creds_manager.clone();
// Other state
let sora_task_queue = SoraTaskQueue::new();
let sora_task_queue_2 = sora_task_queue.clone();
let fal_task_queue = FalTaskQueue::new();
let fal_task_queue_2 = fal_task_queue.clone();
println!("Initializing backend runtime...");
tauri::Builder::default()
@@ -89,13 +103,19 @@ pub fn run() {
eprintln!("Error setting webview unsafe: {:?}", err);
}
// TODO(bt): Clean this up. We can just clone at the callsite. Also clean initialization
let app_2 = app.clone();
let app_3 = app.clone();
let app_4 = app.clone();
let app_5 = app.clone();
let app_data_root_3 = app_data_root_2.clone();
let app_data_root_4 = app_data_root_2.clone();
let app_data_root_5 = app_data_root_2.clone();
let sora_creds_manager_3 = sora_creds_manager_2.clone();
let storyteller_creds_manager_3 = storyteller_creds_manager.clone();
let storyteller_creds_manager_4 = storyteller_creds_manager.clone();
let fal_creds_manager_3 = fal_creds_manager_2.clone();
let fal_task_queue_3 = fal_task_queue_2.clone();
match MainWindowSize::from_filesystem_configs(&app_data_root_3) {
Ok(None) => {}
@@ -128,19 +148,30 @@ pub fn run() {
tauri::async_runtime::spawn(main_window_thread(app_3, app_data_root_3, storyteller_creds_manager_2));
tauri::async_runtime::spawn(sora_task_polling_thread(app_4, app_data_root_4, sora_creds_manager_3, storyteller_creds_manager_3, sora_task_queue_2));
tauri::async_runtime::spawn(discord_presence_thread());
tauri::async_runtime::spawn(fal_task_polling_thread(app_5, app_data_root_5, fal_creds_manager_3, storyteller_creds_manager_4, fal_task_queue_3));
Ok(())
})
.manage(app_data_root)
.manage(app_preferences)
.manage(fal_creds_manager)
.manage(fal_task_queue)
.manage(sora_creds_manager)
.manage(sora_task_queue)
.manage(storyteller_creds_manager_3)
.invoke_handler(tauri::generate_handler![
check_sora_session_command,
enqueue_text_to_image_command,
fal_background_removal_command,
fal_hunyuan_image_to_3d_command,
fal_kling_image_to_video_command,
flip_image,
get_app_preferences_command,
get_build_info_command,
get_fal_api_key_command,
open_sora_login_command,
platform_info_command,
set_fal_api_key_command,
sora_image_generation_command,
sora_image_remix_command,
sora_logout_command,
@@ -0,0 +1,257 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct FalBackgroundRemovalRequest {
/// Image media file; the image to remove the background from.
pub image_media_token: Option<MediaFileToken>,
/// Base64-encoded image
pub base64_image: Option<String>,
}
#[derive(Serialize)]
pub struct FalBackgroundRemovalSuccessResponse {
/// Result media token
pub media_token: MediaFileToken,
/// Result URL
pub cdn_url: Url,
/// Base64 bytes
pub base64_bytes: String,
}
impl SerializeMarker for FalBackgroundRemovalSuccessResponse {}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum BackgroundRemovalErrorType {
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
}
#[tauri::command]
pub async fn fal_background_removal_command(
request: FalBackgroundRemovalRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
) -> CommandResult<FalBackgroundRemovalSuccessResponse, BackgroundRemovalErrorType, ()> {
info!("fal_background_removal_command called; image media token: {:?}", request.image_media_token);
let has_credentials = fal_creds_manager
.has_apparent_api_token()
.unwrap_or(true); // NB: Failures would be lock issues
if !has_credentials {
warn!("No API key found");
return Err(CommandErrorResponseWrapper {
status: CommandErrorStatus::Unauthorized,
error_message: Some("You need to set a FAL api key".to_string()),
error_type: Some(BackgroundRemovalErrorType::NeedsFalApiKey),
error_details: None,
});
}
let result = remove_background(
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager
).await;
match result {
Err(err) => {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = BackgroundRemovalErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
Ok(result) => {
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(FalBackgroundRemovalSuccessResponse {
media_token: result.0.media_file.token,
cdn_url: result.0.media_file.media_links.cdn_url,
base64_bytes: result.1,
}.into())
}
}
}
#[derive(Debug)]
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
DecodeError(DecodeError),
IoError(std::io::Error),
}
impl From<AnyhowError> for InnerError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InnerError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
}
}
impl From<DecodeError> for InnerError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InnerError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
pub async fn remove_background(
request: FalBackgroundRemovalRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
) -> Result<(GetMediaFileSuccessResponse, String), InnerError> {
let api_key = fal_creds_manager.get_key_required()?;
let creds = storyteller_creds_manager.get_credentials_required()?;
let mut temp_download;
if let Some(media_token) = request.image_media_token {
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else if let Some(base64_bytes) = request.base64_image {
temp_download = save_base64_image_to_temp_dir(&app_data_root, base64_bytes).await?;
} else {
return Err(InnerError::AnyhowError(anyhow!("No image media token or base64 image provided")));
}
info!("Calling FAL image background removal...");
let filename = temp_download.path().to_path_buf();
let result = remove_background_rembg(filename, &api_key).await?;
let extension_with_dot = get_url_file_extension(&result.image_url)
.map(|ext| format!(".{}", ext))
.unwrap_or_else(|| ".png".to_string());
let mut result_file = app_data_root.temp_dir().new_named_temp_file_with_extension(&extension_with_dot)?;
info!("Downloading file result file...");
simple_http_download_to_tempfile(&result.image_url, &mut result_file).await?;
let result_filename = result_file.path().to_path_buf();
// NB: We couldn't pass the existing file handle as I think the file handle pointer is already at the EOF
let bytes = file_read_bytes(&result_filename)?;
let base64_bytes = BASE64_STANDARD.encode(bytes.as_bytes());
info!("Uploading image media file...");
let upload_result = upload_image_media_file_from_file(
&ApiHost::Storyteller,
Some(&creds),
result_filename
).await?;
// TODO: Don't re-request to simply build MediaLinks (or CDN URL). Get those from the upload API in one turn.
info!("Re-requesting media file...");
let response = get_media_file(&ApiHost::Storyteller, &upload_result.media_file_token).await?;
info!("Uploaded media file: {:?}", response.media_file.token);
Ok((response, base64_bytes))
}
@@ -0,0 +1,219 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct FalHunyuanImageTo3dRequest {
/// Image media file; the image to remove the background from.
pub image_media_token: Option<MediaFileToken>,
/// Base64-encoded image
pub base64_image: Option<String>,
}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum HunyuanErrorType {
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
}
#[tauri::command]
pub async fn fal_hunyuan_image_to_3d_command(
request: FalHunyuanImageTo3dRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
) -> CommandResult<(), HunyuanErrorType, ()> {
info!("fal_hunyuan_image_to_3d_command called; image media token: {:?}", request.image_media_token);
let has_credentials = fal_creds_manager
.has_apparent_api_token()
.unwrap_or(true); // NB: Failures would be lock issues
if !has_credentials {
warn!("No API key found");
return Err(CommandErrorResponseWrapper {
status: CommandErrorStatus::Unauthorized,
error_message: Some("You need to set a FAL api key".to_string()),
error_type: Some(HunyuanErrorType::NeedsFalApiKey),
error_details: None,
});
}
let result = image_to_3d(
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
).await;
if let Err(err) = result {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = HunyuanErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
return Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(().into())
}
#[derive(Debug)]
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
DecodeError(DecodeError),
IoError(std::io::Error),
}
impl From<AnyhowError> for InnerError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InnerError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
}
}
impl From<DecodeError> for InnerError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InnerError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
pub async fn image_to_3d(
request: FalHunyuanImageTo3dRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InnerError> {
let api_key = fal_creds_manager.get_key_required()?;
let creds = storyteller_creds_manager.get_credentials_required()?;
let mut temp_download;
if let Some(media_token) = request.image_media_token {
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else if let Some(base64_bytes) = request.base64_image {
temp_download = save_base64_image_to_temp_dir(&app_data_root, base64_bytes).await?;
} else {
return Err(InnerError::AnyhowError(anyhow!("No image media token or base64 image provided")));
}
info!("Calling FAL image to 3d ...");
let filename = temp_download.path().to_path_buf();
let enqueued = enqueue_hunyuan2_image_to_3d(Hunyuan2Args {
image_path: filename,
api_key: &api_key,
}).await?;
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InnerError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
}
Ok(())
}
@@ -0,0 +1,226 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::video_gen::enqueue_kling_16_image_to_video::{enqueue_kling_16_image_to_video, Kling16Args, Kling16Duration};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct FalKlingImageToVideoRequest {
/// Image media file; the image to remove the background from.
/// This must be supplied **OR** the `base64_image`.
pub image_media_token: Option<MediaFileToken>,
/// Base64-encoded image
/// This must be supplied **OR** the `image_media_token`.
pub base64_image: Option<String>,
/// Optional: Text prompt.
pub prompt: Option<String>,
}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum KlingImageToVideoErrorType {
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
}
#[tauri::command]
pub async fn fal_kling_image_to_video_command(
request: FalKlingImageToVideoRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
) -> CommandResult<(), KlingImageToVideoErrorType, ()> {
info!("fal_kling_image_to_video_command called; image media token: {:?}", request.image_media_token);
let has_credentials = fal_creds_manager
.has_apparent_api_token()
.unwrap_or(true); // NB: Failures would be lock issues
if !has_credentials {
warn!("No API key found");
return Err(CommandErrorResponseWrapper {
status: CommandErrorStatus::Unauthorized,
error_message: Some("You need to set a FAL api key".to_string()),
error_type: Some(KlingImageToVideoErrorType::NeedsFalApiKey),
error_details: None,
});
}
let result = image_to_video(
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
).await;
if let Err(err) = result {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = KlingImageToVideoErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
return Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(().into())
}
#[derive(Debug)]
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
DecodeError(DecodeError),
IoError(std::io::Error),
}
impl From<AnyhowError> for InnerError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InnerError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
}
}
impl From<DecodeError> for InnerError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InnerError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
pub async fn image_to_video(
request: FalKlingImageToVideoRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InnerError> {
let api_key = fal_creds_manager.get_key_required()?;
let creds = storyteller_creds_manager.get_credentials_required()?;
let mut temp_download;
if let Some(media_token) = request.image_media_token {
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else if let Some(base64_bytes) = request.base64_image {
temp_download = save_base64_image_to_temp_dir(&app_data_root, base64_bytes).await?;
} else {
return Err(InnerError::AnyhowError(anyhow!("No image media token or base64 image provided")));
}
info!("Calling FAL image to video ...");
let filename = temp_download.path().to_path_buf();
let enqueued = enqueue_kling_16_image_to_video(Kling16Args {
image_path: filename,
api_key: &api_key,
duration: Kling16Duration::Default,
prompt: request.prompt.as_deref().unwrap_or(""),
}).await?;
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InnerError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
}
Ok(())
}
@@ -0,0 +1,46 @@
use crate::core::commands::command_response_wrapper::{CommandResult, SerializeMarker};
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
use log::{error, info};
use openai_sora_client::sora_error::SoraError::AnyhowError;
use primitives::traits::trim_or_emptyable::TrimOrEmptyable;
use serde_derive::{Deserialize, Serialize};
use tauri::{AppHandle, State};
#[derive(Serialize)]
pub struct GetFalApiKeyResponse {
pub key: Option<String>,
}
impl SerializeMarker for GetFalApiKeyResponse {}
#[tauri::command]
pub async fn get_fal_api_key_command(
creds_manager: State<'_, FalCredentialManager>,
) -> CommandResult<GetFalApiKeyResponse, (), ()> {
let key = get_key(&creds_manager)
.await
.map_err(|err| {
error!("Error getting API key: {:?}", err);
"error getting key"
})?;
Ok(key.into())
}
async fn get_key(
creds_manager: &FalCredentialManager,
) -> AnyhowResult<GetFalApiKeyResponse> {
let maybe_key = creds_manager.get_key()?
.map(|key| key.0)
.map(|key| key.trim().to_string())
.filter(|key| !key.is_empty());
Ok(GetFalApiKeyResponse { key: maybe_key })
}
@@ -0,0 +1,5 @@
pub mod fal_background_removal_command;
pub mod fal_hunyuan_image_to_3d_command;
pub mod fal_kling_image_to_video_command;
pub mod get_fal_api_key_command;
pub mod set_fal_api_key_command;
@@ -0,0 +1,59 @@
use crate::core::commands::command_response_wrapper::CommandResult;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
use log::{error, info};
use openai_sora_client::sora_error::SoraError::AnyhowError;
use serde_derive::{Deserialize, Serialize};
use tauri::{AppHandle, State};
/// For now, we'll only update a single preference at a time.
#[derive(Deserialize)]
pub struct SetFalApiKeyRequest{
pub key: Option<String>,
}
#[tauri::command]
pub async fn set_fal_api_key_command(
request: SetFalApiKeyRequest,
creds_manager: State<'_, FalCredentialManager>,
) -> CommandResult<(), (), ()> {
info!("update_app_preferences_command called");
set_key(request, &creds_manager)
.await
.map_err(|err| {
error!("Error getting app preferences: {:?}", err);
"error setting key"
})?;
Ok(().into())
}
async fn set_key(
request: SetFalApiKeyRequest,
creds_manager: &FalCredentialManager,
) -> AnyhowResult<()> {
let key = request.key
.map(|key| key.trim().to_string())
.filter(|key| !key.is_empty())
.map(|key| FalApiKey::from_str(&key));
match key {
None => {
creds_manager.clear_key()?;
creds_manager.purge_api_key_from_disk()?;
}
Some(key) => {
creds_manager.set_key(&key)?;
creds_manager.persist_to_disk()?;
}
}
Ok(())
}
@@ -0,0 +1,3 @@
pub mod commands;
pub mod state;
pub mod threads;
@@ -0,0 +1,98 @@
use crate::services::fal::state::read_fal_api_key_from_disk::read_fal_api_key_from_disk;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
use memory_store::clone_slot::CloneSlot;
use std::fs::OpenOptions;
use std::io::Write;
use std::path::Path;
use std::{fs, io};
#[derive(Clone)]
pub struct FalCredentialManager {
key: CloneSlot<FalApiKey>,
app_data_root: AppDataRoot,
}
impl FalCredentialManager {
pub fn new(app_data_root: &AppDataRoot) -> Self {
Self {
key: CloneSlot::empty(),
app_data_root: app_data_root.clone(),
}
}
pub fn initialize_from_disk_infallible(app_data_root: &AppDataRoot) -> Self {
let mut holder = CloneSlot::empty();
match read_fal_api_key_from_disk(app_data_root) {
Err(err) => {
log::warn!("Failed to read FAL API key from disk: {:?}", err);
}
Ok(api_key) => {
holder.set_clone(&api_key).expect("Failed to set FAL API key");
}
}
Self {
key: holder,
app_data_root: app_data_root.clone(),
}
}
pub fn set_key(&self, key: &FalApiKey) -> AnyhowResult<()> {
self.key.set_clone(key)
}
pub fn get_key(&self) -> AnyhowResult<Option<FalApiKey>> {
self.key.get_clone()
}
pub fn get_key_required(&self) -> AnyhowResult<FalApiKey> {
self.key.get_clone_required()
}
pub fn clear_key(&self) -> AnyhowResult<()> {
self.key.clear()
}
/// Does not check the token for validity
pub fn has_apparent_api_token(&self) -> AnyhowResult<bool> {
self.key.is_some()
}
pub fn purge_api_key_from_disk(&self) -> Result<(), io::Error> {
let filename = self.app_data_root.credentials_dir().get_fal_api_key_file_path();
if filename.exists() && filename.is_file() {
fs::remove_file(filename)?;
}
Ok(())
}
pub fn persist_to_disk(&self) -> AnyhowResult<()> {
let key = match self.get_key() {
Ok(Some(key)) => key,
Ok(None) => {
self.purge_api_key_from_disk()?;
return Ok(());
},
Err(err) => {
return Err(anyhow!("Failed to get FAL key: {:?}", err));
}
};
let filename = self.app_data_root.credentials_dir().get_fal_api_key_file_path();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(filename)?;
file.write_all(key.0.as_bytes())?;
file.flush()?;
Ok(())
}
}
@@ -0,0 +1,164 @@
use chrono::{DateTime, Utc};
use errors::AnyhowResult;
use fal_client::model::enqueued_request::EnqueuedRequest;
use fal_client::model::fal_request_id::FalRequestId;
use indexmap::IndexMap;
use std::collections::HashSet;
use std::sync::{Arc, RwLock};
#[derive(Clone)]
pub struct FalTaskDetails {
pub enqueue_time: DateTime<Utc>,
pub enqueued_request: EnqueuedRequest,
}
impl FalTaskDetails {
pub fn for_request(request: EnqueuedRequest) -> Self {
Self {
enqueue_time: Utc::now(),
enqueued_request: request,
}
}
}
#[derive(Clone)]
pub struct FalTaskQueue {
// Insertion-order map of task IDs to task details
queue: Arc<RwLock<IndexMap<FalRequestId, FalTaskDetails>>>
}
impl FalTaskQueue {
pub fn new() -> Self {
Self {
queue: Arc::new(RwLock::new(IndexMap::new()))
}
}
/// Determine if queue is empty
/// This is an O(1) operation.
pub fn is_empty(&self) -> AnyhowResult<bool> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => Ok(lock.is_empty())
}
}
/// Get length of set
/// This is an O(1) operation.
pub fn len(&self) -> AnyhowResult<usize> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => Ok(lock.len())
}
}
/// Get the task at an index.
/// This is an O(1) operation.
pub fn get_index(&self, i: usize) -> AnyhowResult<Option<FalTaskDetails>> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => {
if i >= lock.len() {
Ok(None)
} else {
match lock.get_index(i) {
None => Ok(None),
Some((_key, value)) => {
Ok(Some(value.clone()))
}
}
}
}
}
}
/// Determine if the key is in the queue
/// This is an O(1) operation.
pub fn contains_key(&self, request_id: &FalRequestId) -> AnyhowResult<bool> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => Ok(lock.contains_key(request_id))
}
}
/// Get the first enqueued task
/// This is an O(1) operation.
pub fn first(&self) -> AnyhowResult<Option<FalTaskDetails>> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => match lock.first() {
None => Ok(None),
Some((request_id, task_details)) => Ok(Some(task_details.clone()))
}
}
}
/// Get the last enqueued task
/// This is an O(1) operation.
pub fn last(&self) -> AnyhowResult<Option<FalTaskDetails>> {
match self.queue.read() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire read lock: {:?}", err))
}
Ok(lock) => match lock.last() {
None => Ok(None),
Some((request_id, task_details)) => Ok(Some(task_details.clone()))
}
}
}
/// Insert a new task. If a task already exists, we update its value.
/// This is an O(1) operation.
pub fn insert(&self, request: &EnqueuedRequest) -> AnyhowResult<Option<FalTaskDetails>> {
match self.queue.write() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire write lock: {:?}", err))
}
Ok(mut lock) => {
// NB: Probably fine to compute TaskDetails here for now. We may eventually expose this if we include additional info.
let task_details = FalTaskDetails::for_request(request.clone());
Ok(lock.insert(request.request_id.clone(), task_details))
}
}
}
/// Remove a task by ID.
/// This is an O(n) operation.
pub fn remove(&self, request_id: &FalRequestId) -> AnyhowResult<Option<FalTaskDetails>> {
match self.queue.write() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire write lock: {:?}", err))
}
Ok(mut lock) => {
Ok(lock.shift_remove(request_id))
}
}
}
/// Remove a list of tasks by ID.
/// This is an O(n) operation.
pub fn remove_list(&self, request_ids: &[&FalRequestId]) -> AnyhowResult<()> {
match self.queue.write() {
Err(err) => {
Err(anyhow::anyhow!("Failed to acquire write lock: {:?}", err))
}
Ok(mut lock) => {
// TODO: Cleaner way to do this?
let mut set : HashSet<&FalRequestId, _> = HashSet::with_capacity(request_ids.len());
set.extend(request_ids);
lock.retain(|k, v| !set.contains(k));
Ok(())
}
}
}
}
@@ -0,0 +1,3 @@
pub mod fal_credential_manager;
pub mod fal_task_queue;
pub mod read_fal_api_key_from_disk;
@@ -0,0 +1,19 @@
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
use std::fs::read_to_string;
pub fn read_fal_api_key_from_disk(app_data_root: &AppDataRoot) -> AnyhowResult<FalApiKey> {
let api_key_file = app_data_root.credentials_dir().get_fal_api_key_file_path();
if !api_key_file.exists() {
return Err(anyhow!("API key file does not exist: {:?}", api_key_file));
}
let value = read_to_string(api_key_file)?
.trim()
.to_string();
Ok(FalApiKey::new(value))
}
@@ -0,0 +1,234 @@
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_url_to_temp_dir::download_url_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_generation_complete_event::SoraImageGenerationCompleteEvent;
use crate::services::sora::events::sora_image_generation_failed_event::SoraImageGenerationFailedEvent;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use errors::AnyhowResult;
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::export::queue::Status;
use fal_client::model::fal_request_id::FalRequestId;
use fal_client::utils::queue_status_checker::QueueStatusChecker;
use log::{error, info, warn};
use mimetypes::mimetype_for_file::get_mimetype_for_file;
use mimetypes::mimetype_info::file_extension::FileExtension;
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::requests::image_gen::image_gen_status::{Generation, TaskId, TaskStatus};
use serde_derive::Serialize;
use std::fs::File;
use std::io::Write;
use std::path::PathBuf;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::media_files::upload_new_engine_asset_from_file::upload_new_engine_asset_from_file;
use storyteller_client::media_files::upload_video_media_file_from_file::{upload_video_media_file_from_file, UploadVideoMediaFileSuccessResponse};
use storyteller_client::recipes::upload_media_file_from_file::upload_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter};
use tempdir::TempDir;
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
use url::Url;
pub async fn fal_task_polling_thread(
app_handle: AppHandle,
app_data_root: AppDataRoot,
fal_creds_manager: FalCredentialManager,
storyteller_creds_manager: StorytellerCredentialManager,
fal_task_queue: FalTaskQueue,
) -> ! {
loop {
let res = polling_loop(
&app_handle,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
&app_data_root,
).await;
if let Err(err) = res {
error!("An error occurred: {:?}", err);
}
tokio::time::sleep(std::time::Duration::from_millis(30_000)).await;
}
}
async fn polling_loop(
app_handle: &AppHandle,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
app_data_root: &AppDataRoot,
) -> AnyhowResult<()> {
loop {
if fal_task_queue.is_empty()? {
// No need to poll if we don't have pending tasks.
tokio::time::sleep(std::time::Duration::from_millis(1_000)).await;
continue;
}
info!("Task queue has {} pending tasks.", fal_task_queue.len()?);
let key = fal_creds_manager.get_key_required()?;
let storyteller_creds = storyteller_creds_manager.get_credentials_required()?;
let queue_status_checker = QueueStatusChecker::new(key);
let mut succeeded_tasks = Vec::new();
//let mut failed_tasks = Vec::new(); // TODO
let len = fal_task_queue.len()?;
for i in 0..len {
let task = match fal_task_queue.get_index(i)? {
Some(task) => task,
None => {
warn!("Failed to get task ID at index {}", i);
break;
}
};
let task_id = task.enqueued_request.request_id.clone();
info!("Checking task status for task ID: {:?}", &task_id);
let result = queue_status_checker.check_status(&task.enqueued_request).await;
let url = match result {
Err(err) => {
warn!("Error checking job status: {:?}", err);
tokio::time::sleep(std::time::Duration::from_millis(2_000)).await;
// TODO: Some failures will be permanent job failures and the jobs will need to be removed.
continue;
}
Ok(status) => {
info!("Status: {:?}", status);
match status.status {
Status::Completed => {
let result = queue_status_checker.get_download_url(&task.enqueued_request).await;
let url = match result {
Ok(url) => url,
Err(err) => {
warn!("Failed to get download url: {:?}", err);
// TODO: Some failures will be permanent job failures.
continue;
}
};
url
}
_ => {
continue;
},
}
}
};
// TODO: Clean up
let download_file = download_url_to_temp_dir(&url, &app_data_root).await?;
let result = upload_media_file_from_file(
&ApiHost::Storyteller,
Some(&storyteller_creds),
download_file
).await;
match result {
Ok(success) => {
info!("Uploaded to API backend: {:?}", success.media_file_token);
//let event = SoraImageGenerationCompleteEvent {
// media_file_token: success.media_file_token,
//};
//event.send(&app_handle)?;
succeeded_tasks.push(task_id);
}
Err(err) => {
warn!("Failed to upload media file: {:?}", err);
continue;
}
}
}
// // TODO: The cursoring logic likely needs to improve.
// for task in response.task_responses {
// let status = TaskStatus::from_str(&task.status);
//
// match status {
// TaskStatus::Succeeded => {
// succeeded_tasks.push(task);
// }
// TaskStatus::Failed => {
// failed_tasks.push(task);
// }
// TaskStatus::Queued => {}
// TaskStatus::Running => {}
// TaskStatus::Unknown(_) => {}
// }
// }
//
// for task in failed_tasks.iter() {
// if fal_task_queue.contains_key(&task.id)? {
// let event = SoraImageGenerationFailedEvent {
// prompt: task.prompt.clone(),
// };
// event.send(&app_handle)?;
// }
// }
//
// let failed_task_ids: Vec<&TaskId> = failed_tasks
// .iter()
// .map(|task| &task.id)
// .collect();
//
// fal_task_queue.remove_list(&failed_task_ids)?;
//
// let creds = storyteller_creds_manager.get_credentials_required()?;
// let api_host = ApiHost::Storyteller;
//
// for task in succeeded_tasks.iter() {
// if !fal_task_queue.contains_key(&task.id)? {
// continue;
// }
// info!("Task succeeded: {:?}", task.id);
//
// for (i, generation) in task.generations.iter().enumerate() {
// info!("Downloading generated file...");
// let download_path = download_generation(generation, &app_data_root).await?;
//
// info!("Uploading to backend...");
// let result = upload_image_media_file_from_file(&api_host, Some(&creds), download_path).await?;
//
// info!("Uploaded to API backend: {:?}", result.media_file_token);
//
// let event = SoraImageGenerationCompleteEvent {
// media_file_token: result.media_file_token,
// };
//
// event.send(&app_handle)?;
// }
// }
//
let succeeded_task_ids : Vec<&FalRequestId> = succeeded_tasks
.iter()
.map(|task| task)
.collect::<Vec<_>>();
fal_task_queue.remove_list(&succeeded_task_ids)?;
tokio::time::sleep(std::time::Duration::from_millis(6_000)).await;
}
}
@@ -0,0 +1 @@
pub mod fal_task_polling_thread;
@@ -0,0 +1,3 @@
pub mod fal;
pub mod sora;
pub mod storyteller;
@@ -1,10 +1,9 @@
use std::ops::Add;
use crate::core::commands::app_preferences::get_app_preferences_command::AppPreferencesPayload;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::sora::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use chrono::{DateTime, TimeDelta, Utc};
use crate::commands::app_preferences::get_app_preferences_command::AppPreferencesPayload;
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use errors::AnyhowResult;
use log::{error, info};
use once_cell::sync::Lazy;
@@ -12,6 +11,7 @@ use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::list_media::list_media::list_media;
use serde_derive::Serialize;
use std::ops::Add;
use tauri::{AppHandle, State};
const CACHE_PERIOD : TimeDelta = TimeDelta::milliseconds(1000 * 60 * 5); // 5 minutes

Some files were not shown because too many files have changed in this diff Show More