Compare commits

...
Author SHA1 Message Date
CrossPr0duct b3bd4e7032 Fixed undo of stroke. 2025-06-16 13:59:10 -04:00
CrossPr0duct 1a15e222d4 Ok fixed issue with image sizing. doubled the size. 2025-06-16 13:12:06 -04:00
CrossPr0duct 6c97d6add2 Fixed bug with ContextMenu appearing below the prompt box 2025-06-16 13:04:02 -04:00
BFlat c56cc8700c gif tooltips for 2d and 3d (#674) 2025-06-16 14:49:02 +07:00
Brandon Thomas caaff569a6 fal client - lots of api wrappers (#673) 2025-06-16 01:08:27 -04:00
BFlat 3b0ed7a8ba feature: create 3d model modal (#668)
- Create 3d model from image works now (accessed via wand button next to
the + asset button).
- Modal pops up, or if you dont have a FAL api set in settings it'll pop
up an action reminder to set it first.
- You can choose image from library (opens gallery select mode) or
upload an image from computer.
- Pressing create enqueues the image to 3d and closes the modal.
Spinning loading toast on the top right says Generating... until it
finishes.
- When done, currently it just pops up in the library and you can drag
it onto the scene yourself.

**Current bugs/missing features:**
1. No 3D object image thumbnail in gallery
2. If you open up the gallery and try add the newly generated 3d object
to the scene, it will be stuck at "loading" text because of a CORS error
(not sure why). Pressing the button to refresh the gallery before
dragging it in seems to fix that though.
2025-06-14 20:44:03 -04:00
Brandon Thomas ea1905cfcf tauri bg removal fix (#672) 2025-06-14 01:25:26 -04:00
ArEnScandCrossPr0duct d1f28877ca Feature/add background remove (#670)
Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-06-13 19:07:41 -04:00
BFlat 1b8d8cbd97 login error messages now show properly (#671) 2025-06-13 17:33:36 +07:00
bflatastic 8be5013260 re-enable login modal 2025-06-13 16:29:09 +07:00
bflatastic 3c9b81c3ea bug fix: 3d promptbox events 2025-06-12 22:35:27 +07:00
BFlat 9d1b7d22ee bug fix: ensure no keyboard events leak through to the canvas (#669) 2025-06-12 22:30:43 +07:00
ArEnScandCrossPr0duct b046f6ff2a Feature/Zlayer and opacity issues (#667)
This adds all the z-layer work and context menus back for composition.
Also fixes the alpha issues and removes menus that are not needed for
now.

---------

Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-06-11 13:43:29 -04:00
BFlat df0b98ec63 drag and drop gallery 3d objects to 3d scene feature (#666)
- can now drag and drop 3d models onto the editor scene from the gallery
modal
2025-06-10 12:48:44 +07:00
BFlat f214577af1 2D canvas gallery image drop feature (#665)
- now you can drag and drop onto the 2D canvas from the gallery modal
2025-06-10 12:24:28 +07:00
ArEnScandCrossPr0duct 3efe9db4ca Create Menus and Bug Fixes (#664)
Removed:
Save and Load Scene for later.
Added undo and redo as button.
Hide Generate Image
Added a menu ( still needs logic code )
Fixed Opacity on brush.

---------

Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-06-09 17:36:16 -04:00
Brandon Thomas 8208206e5a missing 2025-06-09 12:20:21 -04:00
Tanish Manku 4f33e74e3a Make 3D state save and load on tab change (#661) 2025-06-06 22:50:47 -07:00
Brandon Thomas f75b210dcd end-to-end fal in artcraft backend (#663) 2025-06-06 21:37:06 -04:00
Brandon ThomasandMichael Chung 7a07f196d1 Fix Safari issue (#662)
Co-authored-by: Michael Chung <michaelchung@mac.home>
2025-06-06 20:26:30 -04:00
a609282d8a Feature/page draw (#659)
Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
Co-authored-by: BFlat <bombayapirat@gmail.com>
2025-06-06 12:03:09 -04:00
BFlat 0a32e06cd5 reusable model selector (#660)
- reusable model selector component
- uses zustand with separated states for each tab

Image model list:
- GPT Image 1 (GPT-4o)
- Flux Pro Ultra
- Recraft 3
- Flux.1 Kontext

Video model list:
- Kling 2.0
- Runway
2025-06-06 17:31:36 +07:00
Brandon Thomas f1563feb91 codegen db 2025-06-05 09:24:06 -04:00
Brandon Thomas cc73ffab5d mark fal jobs complete (#658) 2025-06-05 09:19:38 -04:00
Brandon Thomas d374d9339c codegen db 2025-06-05 00:20:34 -04:00
Brandon Thomas 53681eac3f hunyuan2 image-to-3d (#657) 2025-06-05 00:15:13 -04:00
Brandon Thomas f22369ea4a update artcraft website with download links 2025-06-04 22:01:22 -04:00
Brandon Thomas f7060c0278 classify cloudflare errors 2025-06-04 20:33:27 -04:00
Brandon Thomas 2af8fbd9e1 npm install verbose flags 2025-06-04 20:10:05 -04:00
BFlat a4eb74292f login and popup modals (#652)
- moved artcraft login to first step before sora login
- made sora login skippable
- added pop up to block generation if sora session is not found
2025-06-04 19:47:27 -04:00
Tanish Manku 78b8c56066 Make 2D, 3D, image, video conditionally render (#654) 2025-06-04 14:47:39 -07:00
Brandon Thomas 221aa347bf forgot to commit these 2025-06-04 09:31:34 -04:00
Brandon Thomas 14a27ee803 fal endpoints: kling video 1.6 (#656) 2025-06-04 09:27:26 -04:00
Brandon Thomas 36e35dab0b fal on the backend (#653) 2025-06-03 14:59:28 -04:00
Brandon Thomas cb868c7275 add migrations and materialized final schema 2025-06-02 18:00:04 +05:30
Brandon Thomas b756f81421 vendor in fal-rs so we can hack in webhook support (then upstream it) (#650) 2025-06-02 01:43:18 -04:00
Brandon Thomas 0a1ad3d814 less flux-y generated images as background 2025-06-01 19:13:09 -04:00
Brandon Thomas 9df8c981cd fix api docs 2025-06-01 16:16:50 -04:00
Brandon Thomas e7622318c4 deprecate old endpoints (don't remove yet) (#649) 2025-06-01 15:51:37 -04:00
Brandon Thomas 352c7517cc openart client (maybe we won't do this just yet) (#648) 2025-05-30 19:39:41 -04:00
bflatastic 010a2f043a Remove old unused SettingsModal component 2025-05-30 16:09:25 +07:00
bflatastic cbb32f8275 Fix vertical line color in AssetModal and SettingsModal 2025-05-30 16:08:45 +07:00
bflatastic 0aecc97ede Update TopBar component: correct label for video tab from "Prompt to Video" to "Image to Video" 2025-05-30 15:44:43 +07:00
bflatastic 72a1bb07e0 Update command in windows_frontend_dev.ps1 to use 'npx' for executing nx dev 2025-05-30 13:05:06 +07:00
Brandon Thomas 6a01e4bba0 we don't need to spam the logs with the challenge html 2025-05-30 00:22:01 -04:00
Brandon Thomas 90d741323c check cloudflare challenges 2025-05-30 00:03:48 -04:00
Brandon Thomas 066d1c8bfb emit version to logging, add old sora cookie name 2025-05-29 21:38:07 -04:00
Brandon Thomas 3536736f53 better logging of sora session issues 2025-05-29 20:42:33 -04:00
Brandon Thomas 76d1f65164 more better alerts (#647) 2025-05-29 05:28:16 -04:00
Brandon Thomas ab72551ee7 better events (#646) 2025-05-29 05:03:32 -04:00
Brandon Thomas 9cf87cda5b start phasing fields out from our OpenAI client. These are a liability
It's smarter to just dump the fields we don't care about. That is less
of a chance that we got things wrong or things become incongruent at
runtime in the field.
2025-05-28 20:26:07 -04:00
Brandon Thomas 5163f357af fix nullable type in sora api 2025-05-28 12:15:08 -04:00
BFlat 42dfa9263d artcraft site landing update (#645) 2025-05-28 15:14:35 +07:00
Brandon Thomas f4d9385668 Fix package.lock (#644) 2025-05-28 01:20:09 -04:00
Brandon Thomas 189d6a6df8 refactor response types 2025-05-27 23:37:34 -04:00
Brandon Thomas 89cdc8492f tabs refactor (#643) 2025-05-27 22:54:04 -04:00
BFlat 46d2944823 background gallery for image tab (#638)
- reduced image size
- a more performant css only animation
2025-05-27 22:25:27 -04:00
ArEnScandMichael Chung 216e091257 Fixing build issue with button-link. (#642)
Co-authored-by: Michael Chung <michaelchung@mac.home>
2025-05-27 19:27:24 -04:00
Brandon Thomas f8bfa3ab2b fix nx builds (#641) 2025-05-27 18:58:47 -04:00
1229 changed files with 133293 additions and 14502 deletions
@@ -0,0 +1,12 @@
{
"db_name": "MySQL",
"query": "\nINSERT INTO generic_inference_jobs\nSET\n token = ?,\n uuid_idempotency_token = ?,\n\n job_type = ?,\n\n maybe_external_third_party = ?,\n maybe_external_third_party_id = ?,\n\n product_category = ?,\n inference_category = ?,\n\n maybe_model_type = NULL,\n maybe_model_token = NULL,\n\n maybe_input_source_token = NULL,\n maybe_input_source_token_type = NULL,\n\n maybe_download_url = NULL,\n maybe_cover_image_media_file_token = NULL,\n\n maybe_raw_inference_text = NULL,\n\n maybe_inference_args = ?,\n\n maybe_creator_user_token = ?,\n maybe_creator_anonymous_visitor_token = ?,\n creator_ip_address = ?,\n creator_set_visibility = ?,\n\n priority_level = 0,\n is_keepalive_required = FALSE,\n max_duration_seconds = 0,\n\n is_debug_request = FALSE,\n maybe_routing_tag = NULL,\n\n status = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 13
},
"nullable": []
},
"hash": "0fa824822cd94024dce21243f41e69e4a3e8685c4de676834a7c0ee14982c976"
}
@@ -0,0 +1,113 @@
{
"db_name": "MySQL",
"query": "\nSELECT\n jobs.token as `job_token: tokens::tokens::generic_inference_jobs::InferenceJobToken`,\n\n jobs.status as `status: enums::common::job_status_plus::JobStatusPlus`,\n \n jobs.maybe_external_third_party as `external_third_party: enums::by_table::generic_inference_jobs::inference_job_external_third_party::InferenceJobExternalThirdParty`,\n jobs.maybe_external_third_party_id as `external_third_party_id`,\n\n jobs.maybe_creator_user_token as `maybe_creator_user_token: tokens::tokens::users::UserToken`,\n jobs.maybe_creator_anonymous_visitor_token as `maybe_creator_anonymous_visitor_token: tokens::tokens::anonymous_visitor_tracking::AnonymousVisitorTrackingToken`,\n jobs.creator_ip_address,\n\n jobs.created_at,\n jobs.updated_at\n\nFROM generic_inference_jobs as jobs\n\nWHERE jobs.maybe_external_third_party = ?\nAND jobs.maybe_external_third_party_id = ?\n ",
"describe": {
"columns": [
{
"ordinal": 0,
"name": "job_token: tokens::tokens::generic_inference_jobs::InferenceJobToken",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | UNIQUE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 1,
"name": "status: enums::common::job_status_plus::JobStatusPlus",
"type_info": {
"type": "String",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | ENUM",
"char_set": 224,
"max_size": 76
}
},
{
"ordinal": 2,
"name": "external_third_party: enums::by_table::generic_inference_jobs::inference_job_external_third_party::InferenceJobExternalThirdParty",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 64
}
},
{
"ordinal": 3,
"name": "external_third_party_id",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 256
}
},
{
"ordinal": 4,
"name": "maybe_creator_user_token: tokens::tokens::users::UserToken",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 5,
"name": "maybe_creator_anonymous_visitor_token: tokens::tokens::anonymous_visitor_tracking::AnonymousVisitorTrackingToken",
"type_info": {
"type": "VarString",
"flags": "MULTIPLE_KEY | BINARY",
"char_set": 224,
"max_size": 128
}
},
{
"ordinal": 6,
"name": "creator_ip_address",
"type_info": {
"type": "VarString",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | NO_DEFAULT_VALUE",
"char_set": 224,
"max_size": 160
}
},
{
"ordinal": 7,
"name": "created_at",
"type_info": {
"type": "Timestamp",
"flags": "NOT_NULL | MULTIPLE_KEY | BINARY | TIMESTAMP",
"char_set": 63,
"max_size": 19
}
},
{
"ordinal": 8,
"name": "updated_at",
"type_info": {
"type": "Timestamp",
"flags": "NOT_NULL | BINARY | TIMESTAMP | ON_UPDATE_NOW",
"char_set": 63,
"max_size": 19
}
}
],
"parameters": {
"Right": 2
},
"nullable": [
false,
false,
true,
true,
true,
true,
false,
false,
false
]
},
"hash": "64dfacc1f0552d539f009fea91e902a2fe296c06ad539808596bbf2c8fc9eaa0"
}
@@ -1,12 +1,12 @@
{
"db_name": "MySQL",
"query": "\n INSERT INTO media_files\n SET\n token = ?,\n\n media_class = ?,\n media_type = ?,\n\n is_user_upload = ?,\n is_intermediate_system_file = ?,\n\n origin_category = ?, \n origin_product_category = ?, \n maybe_origin_model_type = ?, \n maybe_origin_model_token = ?, \n maybe_origin_filename = ?,\n\n is_batch_generated = ?,\n maybe_batch_token = ?,\n\n maybe_mime_type = ?,\n file_size_bytes = ?,\n maybe_duration_millis = ?,\n maybe_audio_encoding = ?, \n maybe_video_encoding = ?, \n maybe_frame_width = ?, \n maybe_frame_height = ?,\n maybe_prompt_token = ?,\n checksum_sha2 = ?,\n\n maybe_title = ?,\n maybe_text_transcript = ?,\n\n maybe_scene_source_media_file_token = ?,\n\n public_bucket_directory_hash = ?, \n maybe_public_bucket_prefix = ?, \n maybe_public_bucket_extension = ?, \n\n maybe_creator_user_token = ?, \n maybe_creator_anonymous_visitor_token = ?, \n\n creator_ip_address = ?, \n creator_set_visibility = ?, \n\n maybe_creator_file_synthetic_id = ?, \n maybe_creator_category_synthetic_id = ?,\n\n extra_file_modification_info = ?,\n\n maybe_mod_user_token = ?, \n is_generated_on_prem = ?, \n generated_by_worker = ?, \n generated_by_cluster = ?\n ",
"query": "\n INSERT INTO media_files\n SET\n token = ?,\n\n media_class = ?,\n media_type = ?,\n\n is_user_upload = ?,\n is_intermediate_system_file = ?,\n\n origin_category = ?, \n origin_product_category = ?, \n maybe_origin_model_type = ?, \n maybe_origin_model_token = ?, \n maybe_origin_filename = ?,\n\n is_batch_generated = ?,\n maybe_batch_token = ?,\n\n maybe_mime_type = ?,\n file_size_bytes = ?,\n maybe_duration_millis = ?,\n maybe_audio_encoding = ?, \n maybe_video_encoding = ?, \n maybe_frame_width = ?, \n maybe_frame_height = ?,\n maybe_prompt_token = ?,\n checksum_sha2 = ?,\n \n maybe_engine_category = ?,\n\n maybe_title = ?,\n maybe_text_transcript = ?,\n\n maybe_scene_source_media_file_token = ?,\n\n public_bucket_directory_hash = ?, \n maybe_public_bucket_prefix = ?, \n maybe_public_bucket_extension = ?, \n\n maybe_creator_user_token = ?, \n maybe_creator_anonymous_visitor_token = ?, \n\n creator_ip_address = ?, \n creator_set_visibility = ?, \n\n maybe_creator_file_synthetic_id = ?, \n maybe_creator_category_synthetic_id = ?,\n\n extra_file_modification_info = ?,\n\n maybe_mod_user_token = ?, \n is_generated_on_prem = ?, \n generated_by_worker = ?, \n generated_by_cluster = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 38
"Right": 39
},
"nullable": []
},
"hash": "c58f414b5e4d0f0420d1894e2f40311e5613919f49f3ac91f81f5fc2bef1bea8"
"hash": "b4470e607c005421dd8504b9f1d695080a581936ac6ce00b6f437b2764e741be"
}
@@ -0,0 +1,12 @@
{
"db_name": "MySQL",
"query": "\nUPDATE generic_inference_jobs\nSET\n status = ?,\n on_success_result_entity_type = ?,\n on_success_result_entity_token = ?,\n failure_reason = NULL,\n internal_debugging_failure_reason = NULL,\n success_execution_millis = NULL,\n success_inference_execution_millis = NULL,\n retry_at = NULL,\n successfully_completed_at = NOW()\nWHERE token = ?\n ",
"describe": {
"columns": [],
"parameters": {
"Right": 4
},
"nullable": []
},
"hash": "f68a8917e639418a0bb952bef606c96299001e143c745ee30df99b72d1235d35"
}
+13 -4
View File
@@ -1,6 +1,15 @@
{
"files.exclude": {
"**/*~": true,
"frontend_old/**": true
}
"files.exclude": {
"**/*~": true,
".sqlx/**": true,
"_elasticsearch/**": true,
"_sql/**": true,
"frontend_old/**": true,
"includes/**": true,
"target/**": true,
"test_data/**": true,
"tmp/**": true,
"virtualenv/**": true,
"windows_dlls/**": true,
}
}
Generated
+216 -60
View File
@@ -12,7 +12,7 @@ dependencies = [
"actix-rt",
"actix_derive",
"bitflags 2.5.0",
"bytes 1.10.0",
"bytes 1.10.1",
"crossbeam-channel",
"futures-core",
"futures-sink",
@@ -34,7 +34,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5f7b0a21988c1bf877cf4759ef5ddaac04c1c9fe808c9142ecb78ba97d97a28a"
dependencies = [
"bitflags 2.5.0",
"bytes 1.10.0",
"bytes 1.10.1",
"futures-core",
"futures-sink",
"memchr",
@@ -73,7 +73,7 @@ dependencies = [
"base64 0.22.1",
"bitflags 2.5.0",
"brotli 6.0.0",
"bytes 1.10.0",
"bytes 1.10.1",
"bytestring",
"derive_more 0.99.17",
"encoding_rs",
@@ -117,7 +117,7 @@ dependencies = [
"actix-multipart-derive",
"actix-utils",
"actix-web",
"bytes 1.10.0",
"bytes 1.10.1",
"derive_more 0.99.17",
"futures-core",
"futures-util",
@@ -226,7 +226,7 @@ dependencies = [
"actix-utils",
"actix-web-codegen",
"ahash 0.8.11",
"bytes 1.10.0",
"bytes 1.10.1",
"bytestring",
"cfg-if 1.0.0",
"cookie 0.16.2",
@@ -261,7 +261,7 @@ dependencies = [
"actix-codec",
"actix-http",
"actix-web",
"bytes 1.10.0",
"bytes 1.10.1",
"bytestring",
"futures-core",
"pin-project-lite",
@@ -296,7 +296,7 @@ dependencies = [
"ahash 0.8.11",
"arc-swap",
"async-trait",
"bytes 1.10.0",
"bytes 1.10.1",
"bytestring",
"csv",
"derive_more 0.99.17",
@@ -648,9 +648,10 @@ name = "artcraft"
version = "0.1.0"
dependencies = [
"anyhow",
"artcraft_api_defs",
"base64 0.22.1",
"build_metadata",
"bytes 1.10.0",
"bytes 1.10.1",
"chrono",
"directories",
"dirs",
@@ -659,6 +660,7 @@ dependencies = [
"fal_client",
"filesys",
"futures 0.3.31",
"idempotency",
"image",
"indexmap 2.9.0",
"log",
@@ -686,6 +688,20 @@ dependencies = [
"tokio",
"tokio-util",
"url",
"web_base64",
"workspace-build-acceleration",
]
[[package]]
name = "artcraft_api_defs"
version = "0.0.1"
dependencies = [
"enums",
"serde",
"serde_derive",
"serde_json",
"tokens",
"utoipa",
"workspace-build-acceleration",
]
@@ -1057,7 +1073,7 @@ dependencies = [
"async-trait",
"axum-core",
"bitflags 1.3.2",
"bytes 1.10.0",
"bytes 1.10.1",
"futures-util",
"http 0.2.11",
"http-body 0.4.6",
@@ -1083,7 +1099,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "759fa577a247914fd3f7f76d62972792636412fbfd634cd452f6a385a74d2d2c"
dependencies = [
"async-trait",
"bytes 1.10.0",
"bytes 1.10.1",
"futures-util",
"http 0.2.11",
"http-body 0.4.6",
@@ -1595,9 +1611,9 @@ dependencies = [
[[package]]
name = "bytes"
version = "1.10.0"
version = "1.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f61dac84819c6588b558454b194026eb1f09c293b9036ae9b159e74e73ab6cf9"
checksum = "d71b6127be86fdcfddb610f7182ac57211d4b18a3e9c82eb2d17662f2227ad6a"
dependencies = [
"serde",
]
@@ -1608,7 +1624,7 @@ version = "1.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "74d80203ea6b29df88012294f62733de21cfeab47f17b41af3a38bc30a03ee72"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
]
[[package]]
@@ -1666,6 +1682,17 @@ dependencies = [
"serde",
]
[[package]]
name = "cargo-manifest"
version = "0.19.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a1d8af896b707212cd0e99c112a78c9497dd32994192a463ed2f7419d29bd8c6"
dependencies = [
"serde",
"thiserror 2.0.12",
"toml 0.8.22",
]
[[package]]
name = "cargo-platform"
version = "0.1.9"
@@ -1947,7 +1974,7 @@ version = "4.6.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "35ed6e9d84f0b51a7f52daf1c7d71dd136fd7a3f41a8462b8cdb8c78d920fad4"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"memchr",
]
@@ -3165,7 +3192,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "40d9bd57d914cc66ce878f098f63ed7b5d5b64c30644a5adb950b008f874a6c6"
dependencies = [
"base64 0.11.0",
"bytes 1.10.0",
"bytes 1.10.1",
"dyn-clone",
"lazy_static",
"percent-encoding",
@@ -3537,26 +3564,25 @@ dependencies = [
[[package]]
name = "fal"
version = "0.3.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8fd7dd8565317310f05569296166e2cbdd2bc870173b2b24b77ed681166268d0"
dependencies = [
"base64 0.22.1",
"eventsource-stream",
"fal-derive",
"futures 0.3.31",
"image",
"log",
"percent-encoding-rfc3986",
"reqwest 0.12.15",
"serde",
"serde_json",
"smart-default 0.7.1",
"thiserror 1.0.63",
"tokio",
]
[[package]]
name = "fal-derive"
version = "0.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "87f377f2f18a6bf107f907aa687df6bda2874bd38b25b1531f2a6fb38a9ca643"
dependencies = [
"proc-macro2",
"quote",
@@ -3572,6 +3598,7 @@ dependencies = [
"fal",
"file_to_base64_url",
"futures 0.3.31",
"log",
"reqwest 0.12.15",
"serde",
"testing",
@@ -4037,6 +4064,21 @@ dependencies = [
"x11",
]
[[package]]
name = "generate-endpoints"
version = "0.1.0"
dependencies = [
"cargo-manifest",
"percent-encoding",
"reqwest 0.12.15",
"serde",
"serde_json",
"tokio",
"toml 0.8.22",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "generic-array"
version = "0.14.7"
@@ -4316,7 +4358,7 @@ version = "0.3.26"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "81fe527a889e1532da5c525686d96d4c2e74cdd345badf8dfef9f6b39dd5f5e8"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"fnv",
"futures-core",
"futures-sink",
@@ -4336,7 +4378,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fa82e28a107a8cc405f0839610bdc9b15f1e25ec7d696aa5cf173edbcb1486ab"
dependencies = [
"atomic-waker",
"bytes 1.10.0",
"bytes 1.10.1",
"fnv",
"futures-core",
"futures-sink",
@@ -4546,7 +4588,7 @@ version = "0.2.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8947b1a6fad4393052c7ba1f4cd97bed3e953a95c79c92ad9b051a04611d9fbb"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"fnv",
"itoa 1.0.11",
]
@@ -4557,7 +4599,7 @@ version = "1.1.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "21b9ddb458710bc376481b842f5da65cdf31522de232c1ca8146abce2a358258"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"fnv",
"itoa 1.0.11",
]
@@ -4568,7 +4610,7 @@ version = "0.4.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ceab25649e9960c0311ea418d17bee82c0dcec1bd053b5f9a66e265a693bed2"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"http 0.2.11",
"pin-project-lite",
]
@@ -4579,7 +4621,7 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1cac85db508abc24a2e48553ba12a996e87244a0395ce011e62b37158745d643"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"http 1.1.0",
]
@@ -4589,7 +4631,7 @@ version = "0.1.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "793429d76616a256bcb62c2a2ec2bed781c8307e797e2598c50010f2bee2544f"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures-util",
"http 1.1.0",
"http-body 1.0.0",
@@ -4658,7 +4700,7 @@ version = "0.14.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bf96e135eb83a2a8ddf766e426a841d8ddd7449d5f00d34ea02b41d2f19eef80"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures-channel",
"futures-core",
"futures-util",
@@ -4682,7 +4724,7 @@ version = "1.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cc2b571658e38e0c01b1fdca3bbbe93c00d3d71693ff2770043f8c29bc7d6f80"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures-channel",
"futures-util",
"h2 0.4.5",
@@ -4733,7 +4775,7 @@ version = "0.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6183ddfa99b85da61a140bea0efc93fdf56ceaa041b37d553518030827f9905"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"hyper 0.14.28",
"native-tls",
"tokio",
@@ -4746,7 +4788,7 @@ version = "0.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "70206fc6890eaca9fde8a0bf71caa2ddfc9fe045ac9e5c70df101a7dbde866e0"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"http-body-util",
"hyper 1.6.0",
"hyper-util",
@@ -4762,7 +4804,7 @@ version = "0.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df2dcfbe0677734ab2f3ffa7fa7bfd4706bfdc1ef393f2ee30184aed67e631b4"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures-channel",
"futures-util",
"http 1.1.0",
@@ -6201,7 +6243,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c9ef834d5ed55e494a2ae350220314dc4aacd1c43a9498b00e320e0ea352a5c3"
dependencies = [
"byteorder",
"bytes 1.10.0",
"bytes 1.10.1",
"num-rational",
"serde",
"serde_json",
@@ -6384,6 +6426,16 @@ version = "0.3.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0676bb32a98c1a483ce53e500a81ad9c3d5b3f7c920c28c24e9cb0980d0b5bc8"
[[package]]
name = "nu-ansi-term"
version = "0.46.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "77a8165726e8236064dbb45459242600304b42a5ea24ee2948e18e023bf7ba84"
dependencies = [
"overload",
"winapi 0.3.9",
]
[[package]]
name = "num"
version = "0.4.3"
@@ -6834,6 +6886,35 @@ dependencies = [
"workspace-build-acceleration",
]
[[package]]
name = "openart_client"
version = "0.0.1"
dependencies = [
"anyhow",
"base64 0.22.1",
"chrono",
"cookie 0.18.1",
"crockford",
"errors",
"filesys",
"hex-literal",
"idempotency",
"log",
"once_cell",
"rand 0.9.1",
"reqwest 0.12.15",
"serde",
"serde_derive",
"serde_json",
"sha3",
"subprocess_common",
"testing",
"thiserror 2.0.12",
"tokio",
"toml 0.8.22",
"workspace-build-acceleration",
]
[[package]]
name = "openssl"
version = "0.10.71"
@@ -6989,6 +7070,12 @@ dependencies = [
"pin-project-lite",
]
[[package]]
name = "overload"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b15813163c1d831bf4a13c3610c05c0d03b39feb07f7e09fa234dac9b15aaf39"
[[package]]
name = "p256"
version = "0.13.2"
@@ -7162,6 +7249,12 @@ version = "2.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e3148f5046208a5d56bcfc03053e3ca6334e51da8dfb19b6cdc8b306fae3283e"
[[package]]
name = "percent-encoding-rfc3986"
version = "0.1.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3637c05577168127568a64e9dc5a6887da720efef07b3d9472d45f63ab191166"
[[package]]
name = "phf"
version = "0.8.0"
@@ -7603,7 +7696,7 @@ version = "0.12.6"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "deb1435c188b76130da55f17a466d252ff7b1418b2ad3e037d127b94e3411f29"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"prost-derive",
]
@@ -7755,7 +7848,7 @@ version = "0.11.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8c7c5fdde3cdae7203427dc4f0a68fe0ed09833edc525a03456b153b79828684"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"pin-project-lite",
"quinn-proto",
"quinn-udp",
@@ -7773,7 +7866,7 @@ version = "0.11.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fadfaed2cd7f389d0161bb73eeb07b7b78f8691047a6f3e73caaeae55310a4a6"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"rand 0.8.5",
"ring",
"rustc-hash",
@@ -8068,7 +8161,7 @@ version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6435842fc2fea44b528719eb8c32203bbc1bb2f5b619fbe0c0a3d8350fd8d2a8"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures 0.3.31",
"pin-project-lite",
]
@@ -8246,7 +8339,7 @@ checksum = "dd67538700a17451e7cba03ac727fb961abb7607553461627b97de0b89cf4a62"
dependencies = [
"async-compression",
"base64 0.21.7",
"bytes 1.10.0",
"bytes 1.10.1",
"encoding_rs",
"futures-core",
"futures-util",
@@ -8288,7 +8381,7 @@ checksum = "d19c46a6fdd48bc4dab94b6103fccc55d34c67cc0ad04653aad4ea2a07cd7bbb"
dependencies = [
"async-compression",
"base64 0.22.1",
"bytes 1.10.0",
"bytes 1.10.1",
"cookie 0.18.1",
"cookie_store",
"encoding_rs",
@@ -8412,7 +8505,7 @@ checksum = "9008cd6385b9e161d8229e1f6549dd23c3d022f132a2ea37ac3a10ac4935779b"
dependencies = [
"bitvec",
"bytecheck",
"bytes 1.10.0",
"bytes 1.10.1",
"hashbrown 0.12.3",
"ptr_meta",
"rend",
@@ -8551,7 +8644,7 @@ dependencies = [
"aws-creds",
"aws-region",
"base64 0.22.1",
"bytes 1.10.0",
"bytes 1.10.1",
"cfg-if 1.0.0",
"futures 0.3.31",
"hex",
@@ -8583,7 +8676,7 @@ checksum = "b082d80e3e3cc52b2ed634388d436fe1f4de6af5786cc2de9ba9737527bdf555"
dependencies = [
"arrayvec",
"borsh",
"bytes 1.10.0",
"bytes 1.10.1",
"num-traits",
"rand 0.8.5",
"rkyv",
@@ -8743,7 +8836,7 @@ version = "0.11.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65bc94b580d0f5a6b7a2d604e597513d3c673154b52ddeccd1d5c32360d945ee"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"rxml_validation",
]
@@ -9313,6 +9406,15 @@ dependencies = [
"keccak",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6"
dependencies = [
"lazy_static",
]
[[package]]
name = "shared_service_components"
version = "0.0.1"
@@ -9565,7 +9667,7 @@ dependencies = [
"atoi",
"bigdecimal",
"byteorder",
"bytes 1.10.0",
"bytes 1.10.1",
"chrono",
"crc",
"crossbeam-queue",
@@ -9652,7 +9754,7 @@ dependencies = [
"bigdecimal",
"bitflags 2.5.0",
"byteorder",
"bytes 1.10.0",
"bytes 1.10.1",
"chrono",
"crc",
"digest 0.10.7",
@@ -9799,12 +9901,14 @@ dependencies = [
"actix_cors_configs",
"actix_helpers",
"anyhow",
"artcraft_api_defs",
"async-stripe",
"async-trait",
"base64 0.22.1",
"billing_component",
"bootstrap",
"bucket_paths",
"bytes 1.10.1",
"chrono",
"cloud_storage",
"collections",
@@ -9822,6 +9926,7 @@ dependencies = [
"enums",
"enums_public",
"errors",
"fal_client",
"filesys",
"futures 0.1.31",
"futures 0.3.31",
@@ -9893,6 +9998,7 @@ name = "storyteller_client"
version = "0.0.1"
dependencies = [
"anyhow",
"artcraft_api_defs",
"backoff",
"chrono",
"cookie 0.18.1",
@@ -10384,7 +10490,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4d08db1ff9e011e04014e737ec022610d756c0eae0b3b3a9037bccaf3003173a"
dependencies = [
"anyhow",
"bytes 1.10.0",
"bytes 1.10.1",
"dirs",
"dunce",
"embed_plist",
@@ -10552,7 +10658,7 @@ dependencies = [
name = "tauri-plugin-http"
version = "2.4.3"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"cookie_store",
"data-url",
"http 1.1.0",
@@ -10719,11 +10825,10 @@ dependencies = [
[[package]]
name = "tempfile"
version = "3.18.0"
version = "3.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2c317e0a526ee6120d8dabad239c8dadca62b24b6f168914bbbc8e2fb1f0e567"
checksum = "e8a64e3985349f2441a1a9ef0b853f869006c3855f2cda6862a94d26ebb9d6a1"
dependencies = [
"cfg-if 1.0.0",
"fastrand 2.3.0",
"getrandom 0.3.1",
"once_cell",
@@ -10802,6 +10907,16 @@ dependencies = [
"syn 2.0.87",
]
[[package]]
name = "thread_local"
version = "1.1.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8b9ef9bad013ada3808854ceac7b46812a6465ba368859a37e2100283d2d719c"
dependencies = [
"cfg-if 1.0.0",
"once_cell",
]
[[package]]
name = "thumbnail_generator"
version = "0.1.0"
@@ -10926,7 +11041,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75ef51a33ef1da925cea3e4eb122833cb377c61439ca401b770f54902b806779"
dependencies = [
"backtrace",
"bytes 1.10.0",
"bytes 1.10.1",
"libc",
"mio 1.0.2",
"parking_lot 0.12.2",
@@ -11102,7 +11217,7 @@ version = "0.7.12"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61e7c3654c13bcd040d4a03abee2c75b1d14a37b423cf5a813ceae1cc903ec6a"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"futures-core",
"futures-sink",
"pin-project-lite",
@@ -11127,6 +11242,7 @@ version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "05ae329d1f08c4d17a59bed7ff5b5a769d062e64a62d34a3261b219e62cd5aae"
dependencies = [
"indexmap 2.9.0",
"serde",
"serde_spanned",
"toml_datetime",
@@ -11207,7 +11323,7 @@ dependencies = [
"async-trait",
"axum",
"base64 0.21.7",
"bytes 1.10.0",
"bytes 1.10.1",
"h2 0.3.26",
"http 0.2.11",
"http-body 0.4.6",
@@ -11273,9 +11389,9 @@ checksum = "8df9b6e13f2d32c91b9bd719c00d1958837bc7dec474d94952798cc8e69eeec3"
[[package]]
name = "tracing"
version = "0.1.40"
version = "0.1.41"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c3523ab5a71916ccf420eebdf5521fcef02141234bbc0b8a49f2fdc4544364ef"
checksum = "784e0ac535deb450455cbfa28a6f0df145ea1bb7ae51b821cf5e7927fdcfbdd0"
dependencies = [
"log",
"pin-project-lite",
@@ -11285,9 +11401,9 @@ dependencies = [
[[package]]
name = "tracing-attributes"
version = "0.1.27"
version = "0.1.28"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34704c8d6ebcbc939824180af020566b01a7c01f80641264eba0999f6c2b6be7"
checksum = "395ae124c09f9e6918a2310af6038fba074bcf474ac352496d5910dd59a2226d"
dependencies = [
"proc-macro2",
"quote",
@@ -11296,11 +11412,37 @@ dependencies = [
[[package]]
name = "tracing-core"
version = "0.1.32"
version = "0.1.33"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c06d3da6113f116aaee68e4d601191614c9053067f9ab7f6edbcb161237daa54"
checksum = "e672c95779cf947c5311f83787af4fa8fffd12fb27e4993211a84bdfd9610f9c"
dependencies = [
"once_cell",
"valuable",
]
[[package]]
name = "tracing-log"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3"
dependencies = [
"log",
"once_cell",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.3.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8189decb5ac0fa7bc8b96b7cb9b2701d60d48805aca84a238004d665fcc4008"
dependencies = [
"nu-ansi-term",
"sharded-slab",
"smallvec 1.13.2",
"thread_local",
"tracing-core",
"tracing-log",
]
[[package]]
@@ -11712,6 +11854,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "valuable"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "value-bag"
version = "1.10.0"
@@ -12064,11 +12212,19 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "web_base64"
version = "0.0.1"
dependencies = [
"base64 0.22.1",
"workspace-build-acceleration",
]
[[package]]
name = "web_scrapers"
version = "0.0.1"
dependencies = [
"bytes 1.10.0",
"bytes 1.10.1",
"chrono",
"enums",
"errors",
@@ -12744,7 +12900,7 @@ dependencies = [
"block-buffer 0.9.0",
"bytemuck",
"byteorder",
"bytes 1.10.0",
"bytes 1.10.1",
"cc",
"chrono",
"clap",
+12
View File
@@ -27,6 +27,7 @@ members = [
"crates/lib/api_clients/google_drive_common",
"crates/lib/api_clients/google_sign_in",
"crates/lib/api_clients/openai_sora_client",
"crates/lib/api_clients/openart_client",
"crates/lib/api_clients/runwayml_client",
"crates/lib/api_clients/storyteller_client",
"crates/lib/api_clients/thumbnail_generator",
@@ -42,6 +43,7 @@ members = [
"crates/lib/data/crockford",
"crates/lib/data/datetimes",
"crates/lib/data/primitives",
"crates/lib/data/web_base64",
"crates/lib/deprecated/config", # TODO(bt, 2023-02-13): Deprecate or repurpose
"crates/lib/deprecated/http_server_common", # TODO(bt, 2023-03-20): Move useful pieces to actix_helpers
"crates/lib/dimensions",
@@ -87,6 +89,7 @@ members = [
#"crates/schema/database/sqlite_queries",
# Schema (public): common strongly-typed tokens, ids, payloads, etc.
"crates/schema/public/api/artcraft_api_defs",
"crates/schema/public/composite_identifiers",
"crates/schema/public/enums",
"crates/schema/public/enums_public",
@@ -112,6 +115,9 @@ members = [
"crates/service/plugins/user_traits_component",
# Vendored code
"crates/vendor/fal-rs",
"crates/vendor/fal-rs/fal-derive",
"crates/vendor/fal-rs/generate",
"crates/vendor/limitation",
"crates/vendor/tauri-plugin-http",
# NB(bt,2025-05-25): Too many failures on Mac. Disable for now.
@@ -177,6 +183,7 @@ actix-web-actors = "4.3.1+deprecated"
actix-web-lab = "0.20.2"
anyhow = "1.0.98"
base64 = "0.22.1"
bytes = "1.10.1"
chrono = { version = "0.4.41", features = ["serde"] }
cookie = "0.18.1"
futures = { version = "0.3.31" }
@@ -193,18 +200,22 @@ serde_json = "1.0.140"
serde_yaml = "0.9.34+deprecated"
sha3 = "0.10.8"
tauri = { version = "2.4.1", features = ["unstable", "devtools"] } # NB: "devtools" prevents App Store distribution
tempdir = "0.3.7"
tempfile = "3.20.0"
thiserror = "2.0.12"
tokio = { version = "1.45.1", features = ["macros"] }
toml = "0.8.22"
url = { version = "2.5.4", features = ["serde"] }
# Internal libs
artcraft_api_defs = { path = "crates/schema/public/api/artcraft_api_defs" }
build_metadata = { path = "crates/lib/build_metadata" }
crockford = { path = "crates/lib/data/crockford" }
enums = { path = "crates/schema/public/enums" }
enums_public = { path = "crates/schema/public/enums_public" }
errors = { path = "crates/lib/errors" }
fal_client = { path = "crates/lib/api_clients/fal_client" }
file_to_base64_url = { path = "crates/lib/files/file_to_base64_url" }
filesys = { path = "crates/lib/files/filesys" }
idempotency = { path = "crates/lib/idempotency" }
images = { path = "crates/lib/files/images" }
@@ -218,3 +229,4 @@ shared_service_components = { path = "crates/lib/shared_service_components" }
storyteller_client = { path = "crates/lib/api_clients/storyteller_client" }
subprocess_common = { path = "crates/lib/subprocess_common" }
tokens = { path = "crates/schema/public/tokens" }
web_base64= { path = "crates/lib/data/web_base64" }
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE generic_inference_jobs DROP COLUMN maybe_external_third_party_id;
ALTER TABLE generic_inference_jobs DROP COLUMN maybe_external_third_party;
@@ -0,0 +1,9 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE generic_inference_jobs
ADD COLUMN maybe_external_third_party VARCHAR(16) DEFAULT NULL;
ALTER TABLE generic_inference_jobs
ADD COLUMN maybe_external_third_party_id VARCHAR(64) DEFAULT NULL;
@@ -0,0 +1,9 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
DROP INDEX `index_maybe_external_third_party_id`
ON `generic_inference_jobs`;
DROP INDEX `index_maybe_external_third_party_and_id`
ON `generic_inference_jobs`;
@@ -0,0 +1,11 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE generic_inference_jobs
ADD UNIQUE INDEX index_maybe_external_third_party_and_id (maybe_external_third_party, maybe_external_third_party_id),
ALGORITHM=INPLACE, LOCK=NONE;
ALTER TABLE generic_inference_jobs
ADD INDEX index_maybe_external_third_party_id (maybe_external_third_party_id),
ALGORITHM=INPLACE, LOCK=NONE;
@@ -39,8 +39,20 @@ CREATE TABLE generic_inference_jobs (
-- Examples (may not be up to date):
-- * text_to_speech
-- * voice_conversion
-- * image_generation
-- * video_generation
-- * background_removal
inference_category VARCHAR(32) NOT NULL,
-- If the job is externally enqueued, this is the third-party service
-- responsible for the job. Typically, we won't run an inference job for
-- jobs being run on a third party.
maybe_external_third_party VARCHAR(16) DEFAULT NULL,
-- If the job is externally enqueued, this is the ID the third-party
-- service uses.
maybe_external_third_party_id VARCHAR(64) DEFAULT NULL,
-- There is an index on this column.
-- NB: See notes on inference_category. This is becoming a problematic field. We're using
-- this to load the container spin-up arguments and dependencies, but the former is being
@@ -287,11 +299,13 @@ CREATE TABLE generic_inference_jobs (
PRIMARY KEY (id),
UNIQUE KEY (token),
UNIQUE KEY (uuid_idempotency_token),
UNIQUE KEY (maybe_external_third_party, maybe_external_third_party_id),
KEY index_job_type (job_type),
KEY index_product_category (product_category),
KEY index_inference_category (inference_category),
KEY index_maybe_model_type_and_maybe_model_token (maybe_model_type, maybe_model_token),
KEY index_maybe_model_type (maybe_model_type),
KEY index_maybe_external_third_party_id (maybe_external_third_party_id),
KEY fk_maybe_model_token (maybe_model_token),
KEY fk_maybe_input_source_token (maybe_input_source_token),
KEY fk_maybe_input_source_token_and_type (maybe_input_source_token, maybe_input_source_token_type),
+6 -3
View File
@@ -23,21 +23,24 @@ tauri-build = { version = "2.1.1", features = [] }
workspace-build-acceleration = { workspace = true }
# Internal
artcraft_api_defs = { workspace = true }
build_metadata = { workspace = true }
errors = { workspace = true }
fal_client = { workspace = true }
filesys = { workspace = true }
idempotency = { workspace = true }
memory_store = { workspace = true }
mimetypes = { workspace = true }
openai_sora_client = { workspace = true }
primitives = { workspace = true }
storyteller_client = { workspace = true }
tokens = { workspace = true }
web_base64 = { workspace = true }
# External
anyhow = { workspace = true }
base64 = { workspace = true }
bytes = "1.10"
bytes = "1.10.1"
chrono = { workspace = true }
directories = "6.0.0"
dirs = "6.0.0"
@@ -54,8 +57,8 @@ serde_json = { workspace = true }
tauri = { workspace = true }
tauri-plugin-log = "2.3.1"
tauri-plugin-upload = "2.2.1"
tempdir = "0.3.7"
tempfile = "3.18.0"
tempdir = { workspace = true }
tempfile = { workspace = true }
tokio = { version = "1.42", features = ["rt", "rt-multi-thread", "fs"] }
tokio-util = { version = "0.7", features = ["codec"] }
url = { workspace = true }
@@ -0,0 +1,48 @@
use std::error::Error;
use std::fmt::{Display, Formatter};
use storyteller_client::error::storyteller_error::StorytellerError;
#[derive(Debug)]
pub enum ArtcraftError {
AnyhowError(anyhow::Error),
DecodeError(base64::DecodeError),
IoError(std::io::Error),
StorytellerError(StorytellerError),
}
impl Error for ArtcraftError {}
impl Display for ArtcraftError {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
Self::AnyhowError(e) => write!(f, "AnyhowError: {:?}", e),
Self::DecodeError(e) => write!(f, "DecodeError: {:?}", e),
Self::IoError(e) => write!(f, "IoError: {:?}", e),
Self::StorytellerError(e) => write!(f, "StorytellerError: {:?}", e),
}
}
}
impl From<anyhow::Error> for ArtcraftError {
fn from(value: anyhow::Error) -> Self {
Self::AnyhowError(value)
}
}
impl From<base64::DecodeError> for ArtcraftError {
fn from(value: base64::DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for ArtcraftError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
impl From<StorytellerError> for ArtcraftError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -1,7 +1,9 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::commands::enqueue::image::handle_fal::handle_fal;
use crate::core::commands::enqueue::image::handle_sora::handle_sora;
use crate::core::commands::enqueue::image::handle_image_fal::handle_image_fal;
use crate::core::commands::enqueue::image::handle_image_sora::handle_image_sora;
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
@@ -12,8 +14,6 @@ use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -24,9 +24,8 @@ use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
@@ -56,7 +55,8 @@ use storyteller_client::media_files::upload_image_media_file_from_file::upload_i
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
use crate::core::commands::enqueue::image::handle_image_artcraft::handle_image_artcraft;
use crate::core::commands::enqueue::object::handle_object_artcraft::handle_object_artcraft;
#[derive(Deserialize)]
pub struct EnqueueTextToImageRequest {
@@ -95,11 +95,14 @@ pub enum EnqueueTextToImageErrorType {
NeedsFalApiKey,
/// Fal had an API error
FalError,
/// Needs to be logged into Artcraft
NeedsStorytellerCredentials,
}
#[tauri::command]
pub async fn enqueue_text_to_image_command(
app: AppHandle,
request: EnqueueTextToImageRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
@@ -107,11 +110,12 @@ pub async fn enqueue_text_to_image_command(
fal_task_queue: State<'_, FalTaskQueue>,
sora_creds_manager: State<'_, SoraCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> CommandResult<EnqueueTextToImageSuccessResponse, EnqueueTextToImageErrorType, ()> {
) -> Response<EnqueueTextToImageSuccessResponse, EnqueueTextToImageErrorType, ()> {
info!("enqueue_text_to_image called");
let result = handle_request(
&app,
request,
&app_data_root,
&fal_creds_manager,
@@ -125,12 +129,6 @@ pub async fn enqueue_text_to_image_command(
Err(err) => {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = EnqueueTextToImageErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
@@ -156,21 +154,15 @@ pub async fn enqueue_text_to_image_command(
error_details: None,
})
}
Ok(result) => {
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(EnqueueTextToImageSuccessResponse {
}.into())
Ok(()) => {
Ok(EnqueueTextToImageSuccessResponse {}.into())
}
}
}
pub async fn handle_request(
app: &AppHandle,
request: EnqueueTextToImageRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
@@ -184,13 +176,20 @@ pub async fn handle_request(
None => {
return Err(InternalImageError::NoModelSpecified);
}
Some(EnqueueTextToImageModel::FluxProUltra) | Some(EnqueueTextToImageModel::Recraft3) => {
handle_fal(request, fal_creds_manager, fal_task_queue).await?;
},
Some(EnqueueTextToImageModel::GptImage1) => {
handle_sora(request, sora_creds_manager, sora_task_queue).await?;
handle_image_sora(&app, request, sora_creds_manager, sora_task_queue).await?;
return Ok(());
}
_ => {
// Fall-through
}
};
if fal_creds_manager.has_apparent_api_token()? {
handle_image_fal(&app, request, fal_creds_manager, fal_task_queue).await?;
} else {
handle_image_artcraft(request, &app, app_data_root, storyteller_creds_manager).await?;
}
Ok(())
}
@@ -0,0 +1,95 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use artcraft_api_defs::generate::image::generate_flux_pro_11_ultra_text_to_image::GenerateFluxPro11UltraTextToImageRequest;
use artcraft_api_defs::generate::object::generate_hunyuan_2_image_to_3d::GenerateHunyuan2ImageTo3dRequest;
use artcraft_api_defs::generate::video::generate_kling_1_6_pro_image_to_video::GenerateKling16ProImageToVideoRequest;
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use idempotency::uuid::generate_random_uuid;
use log::{error, info, warn};
use storyteller_client::generate::image::generate_flux_pro_11_ultra_text_to_image::generate_flux_pro_11_ultra_text_to_image;
use storyteller_client::generate::object::generate_hunyuan2_image_to_3d::generate_hunyuan2_image_to_3d;
use storyteller_client::generate::video::generate_kling_16_pro_image_to_video::generate_kling_16_pro_image_to_video;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, State};
pub async fn handle_image_artcraft(
request: EnqueueTextToImageRequest,
app: &AppHandle,
app_data_root: &AppDataRoot,
storyteller_creds_manager: &StorytellerCredentialManager,
) -> Result<(), InternalImageError> {
let creds = match storyteller_creds_manager.get_credentials()? {
Some(creds) => creds,
None => {
error!("No Artcraft credentials are set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_artcraft_credentials(GenerationAction::GenerateImage);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalImageError::NeedsStorytellerCredentials);
},
};
info!("Calling Artcraft text to image...");
let uuid_idempotency_token = generate_random_uuid();
let mut selected_model = None;
let job_token = match request.model {
None => {
return Err(InternalImageError::NoModelSpecified);
}
Some(EnqueueTextToImageModel::GptImage1) => {
return Err(InternalImageError::AnyhowError(anyhow!("wrong logic: artcraft is handling sora images")));
}
Some(EnqueueTextToImageModel::Recraft3) => {
return Err(InternalImageError::AnyhowError(anyhow!("not yet implemented in Artcraft")));
}
Some(EnqueueTextToImageModel::FluxProUltra) => {
info!("enqueue Flux Pro Ultra");
selected_model = Some(GenerationModel::FluxPro11Ultra);
let request = GenerateFluxPro11UltraTextToImageRequest {
uuid_idempotency_token,
prompt: request.prompt,
};
let result = generate_flux_pro_11_ultra_text_to_image(
&ApiHost::Storyteller,
Some(&creds),
request,
).await;
match result {
Ok(enqueued) => {
info!("Successfully enqueued Artcraft flux pro ultra text to image generation");
enqueued.inference_job_token
}
Err(err) => {
error!("Failed to use Artcraft flux pro ultra text to image generation: {:?}", err);
return Err(InternalImageError::StorytellerError(err));
}
}
}
};
Ok(())
}
@@ -1,5 +1,9 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
@@ -7,16 +11,34 @@ use crate::services::sora::state::sora_credential_manager::SoraCredentialManager
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use log::{error, info, warn};
use tauri::AppHandle;
pub async fn handle_fal(
pub async fn handle_image_fal(
app: &AppHandle,
request: EnqueueTextToImageRequest,
fal_creds_manager: &FalCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalImageError> {
let api_key = match fal_creds_manager.get_key()? {
Some(key) => key,
None => {
error!("No FAL API key is set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_fal_api_key(GenerationAction::GenerateImage);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalImageError::NeedsFalApiKey);
},
};
let api_key = fal_creds_manager.get_key_required()
.map_err(|err| {
error!("EnqueueTextToImage FAL api key required: {:?}", err);
@@ -34,7 +56,7 @@ pub async fn handle_fal(
}
Some(EnqueueTextToImageModel::FluxProUltra) => {
info!("enqueue Flux Pro Ultra text-to-image with prompt: {}", prompt);
enqueue_flux_pro_ultra_text_to_image(FluxProUltraTextToImageArgs {
enqueue_flux_pro_11_ultra_text_to_image(FluxPro11UltraTextToImageArgs {
prompt,
api_key: &api_key,
}).await
@@ -52,6 +74,16 @@ pub async fn handle_fal(
Ok(enqueued) => {
info!("Successfully enqueued text to image");
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InternalImageError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
@@ -59,6 +91,18 @@ pub async fn handle_fal(
}
Err(err) => {
error!("Failed to enqueue text to image: {:?}", err);
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Fal,
model: None,
reason: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalImageError::FalError(err));
}
}
@@ -1,5 +1,9 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::image::internal_image_error::InternalImageError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
@@ -7,8 +11,8 @@ use crate::services::sora::state::sora_credential_manager::SoraCredentialManager
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use fal_client::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use log::{error, info, warn};
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
@@ -17,10 +21,12 @@ use openai_sora_client::recipes::simple_image_gen_with_session_auto_renew::{simp
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages, SoraImageGenResponse};
use openai_sora_client::sora_error::SoraError;
use std::time::Duration;
use tauri::AppHandle;
const SORA_SIMPLE_IMAGE_GEN_TIMEOUT : Duration = Duration::from_millis(1000 * 30); // 30 seconds
pub async fn handle_sora(
pub async fn handle_image_sora(
app: &AppHandle,
request: EnqueueTextToImageRequest,
sora_creds_manager: &SoraCredentialManager,
sora_task_queue: &SoraTaskQueue,
@@ -60,6 +66,17 @@ pub async fn handle_sora(
match result {
Ok((response, maybe_new_creds)) => (response, maybe_new_creds),
Err(err) => {
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
reason: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalImageError::SoraError(err));
}
};
@@ -73,5 +90,15 @@ pub async fn handle_sora(
sora_task_queue.insert(&response.task_id)?;
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
Ok(())
}
@@ -1,20 +1,23 @@
use crate::core::artcraft_error::ArtcraftError;
use base64::DecodeError;
use errors::AnyhowError;
use fal_client::error::fal_error_plus::FalErrorPlus;
use openai_sora_client::sora_error::SoraError;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::storyteller_error::StorytellerError;
#[derive(Debug)]
pub enum InternalImageError {
NoModelSpecified,
NeedsFalApiKey,
NeedsSoraCredentials,
NeedsStorytellerCredentials,
FalError(FalErrorPlus),
SoraError(SoraError),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InternalImageError {
@@ -35,9 +38,9 @@ impl From<SoraError> for InternalImageError {
}
}
impl From<ApiError> for InternalImageError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
impl From<StorytellerError> for InternalImageError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -52,3 +55,15 @@ impl From<std::io::Error> for InternalImageError {
Self::IoError(value)
}
}
impl From<ArtcraftError> for InternalImageError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
ArtcraftError::StorytellerError(e) => Self::StorytellerError(e),
_ => Self::ArtcraftError(value),
}
}
}
@@ -1,4 +1,5 @@
pub mod enqueue_text_to_image_command;
pub mod handle_fal;
pub mod handle_sora;
pub mod handle_image_artcraft;
pub mod handle_image_fal;
pub mod handle_image_sora;
pub mod internal_image_error;
@@ -1 +1,3 @@
pub mod image;
pub mod image;
pub mod object;
pub mod video;
@@ -0,0 +1,177 @@
use crate::core::commands::enqueue::object::handle_object_artcraft::handle_object_artcraft;
use crate::core::commands::enqueue::object::handle_object_fal::handle_object_fal;
use crate::core::commands::enqueue::object::internal_object_error::InternalObjectError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct EnqueueImageTo3dObjectRequest {
/// Image media file; the image to remove the background from.
/// TODO: In the future we may support base64 images, URLs, or file paths here.
pub image_media_token: Option<MediaFileToken>,
/// The model to use.
pub model: Option<EnqueueImageTo3dObjectModel>,
}
#[derive(Deserialize, Debug, Copy, Clone)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueImageTo3dObjectModel {
#[serde(rename = "hunyuan_3d_2")]
Hunyuan3d2,
}
#[derive(Serialize)]
pub struct EnqueueImageTo3dObjectSuccessResponse {
}
impl SerializeMarker for EnqueueImageTo3dObjectSuccessResponse {}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueImageTo3dObjectErrorType {
/// Caller didn't specify a model
ModelNotSpecified,
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
/// Fal had an API error
FalError,
/// Needs to be logged into Artcraft
NeedsStorytellerCredentials,
}
#[tauri::command]
pub async fn enqueue_image_to_3d_object_command(
app: AppHandle,
request: EnqueueImageTo3dObjectRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> Response<EnqueueImageTo3dObjectSuccessResponse, EnqueueImageTo3dObjectErrorType, ()> {
info!("enqueue_image_to_3d_object_command called");
let result = handle_request(
&app,
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
).await;
match result {
Err(err) => {
error!("error: {:?}", err);
let mut status = CommandErrorStatus::ServerError;
let mut error_type = EnqueueImageTo3dObjectErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
match err {
InternalObjectError::NoModelSpecified => {
status = CommandErrorStatus::BadRequest;
error_type = EnqueueImageTo3dObjectErrorType::ModelNotSpecified;
error_message = "No model specified for image generation";
}
InternalObjectError::NeedsFalApiKey => {
status = CommandErrorStatus::Unauthorized;
error_type = EnqueueImageTo3dObjectErrorType::NeedsFalApiKey;
error_message = "You need to set a FAL api key";
},
InternalObjectError::NeedsStorytellerCredentials => {
status = CommandErrorStatus::Unauthorized;
error_type = EnqueueImageTo3dObjectErrorType::NeedsStorytellerCredentials;
error_message = "You need to be logged into Artcraft.";
}
_ => {}, // Fall-through
}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
Ok(()) => {
Ok(EnqueueImageTo3dObjectSuccessResponse {}.into())
}
}
}
pub async fn handle_request(
app: &AppHandle,
request: EnqueueImageTo3dObjectRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalObjectError> {
if fal_creds_manager.has_apparent_api_token()? {
handle_object_fal(&app, app_data_root, request, fal_creds_manager, fal_task_queue).await?;
} else {
handle_object_artcraft(request, &app, app_data_root, storyteller_creds_manager).await?;
}
Ok(())
}
@@ -0,0 +1,129 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::object::enqueue_image_to_3d_object_command::{EnqueueImageTo3dObjectModel, EnqueueImageTo3dObjectRequest};
use crate::core::commands::enqueue::object::internal_object_error::InternalObjectError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use artcraft_api_defs::generate::object::generate_hunyuan_2_image_to_3d::GenerateHunyuan2ImageTo3dRequest;
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use idempotency::uuid::generate_random_uuid;
use log::{error, info, warn};
use storyteller_client::generate::object::generate_hunyuan2_image_to_3d::generate_hunyuan2_image_to_3d;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, State};
pub async fn handle_object_artcraft(
request: EnqueueImageTo3dObjectRequest,
app: &AppHandle,
app_data_root: &AppDataRoot,
storyteller_creds_manager: &StorytellerCredentialManager,
) -> Result<(), InternalObjectError> {
let creds = match storyteller_creds_manager.get_credentials()? {
Some(creds) => creds,
None => {
error!("No Artcraft credentials are set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_artcraft_credentials(GenerationAction::ImageTo3d);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalObjectError::NeedsStorytellerCredentials);
},
};
//let api_key = fal_creds_manager.get_key_required()
// .map_err(|err| {
// error!("EnqueueTextToImage FAL api key required: {:?}", err);
// InternalObjectError::NeedsFalApiKey
// })?;
info!("Calling Artcraft image to 3d ...");
let uuid_idempotency_token = generate_random_uuid();
let job_token = match request.model {
None => {
return Err(InternalObjectError::NoModelSpecified);
}
Some(EnqueueImageTo3dObjectModel::Hunyuan3d2) => {
info!("enqueue Hunyuan 3D 2.0");
let request = GenerateHunyuan2ImageTo3dRequest {
uuid_idempotency_token,
media_file_token: request.image_media_token,
};
let result = generate_hunyuan2_image_to_3d(
&ApiHost::Storyteller,
Some(&creds),
request,
).await;
match result {
Ok(enqueued) => {
info!("Successfully enqueued Artcraft Hunyuan 3D 2.0");
enqueued.inference_job_token
}
Err(err) => {
error!("Failed to use Artcraft Hunyuan 3D 2.0: {:?}", err);
return Err(InternalObjectError::StorytellerError(err));
}
}
}
};
info!("Successfully enqueued 3D generation");
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::ImageTo3d,
service: GenerationServiceProvider::Artcraft,
model: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
//if let Err(err) = fal_task_queue.insert(&enqueued) {
// error!("Failed to enqueue task: {:?}", err);
// return Err(InternalObjectError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
//}
// match result {
// Ok(enqueued) => {
// }
// Err(err) => {
// error!("Failed to enqueue image to 3d: {:?}", err);
//
// let event = GenerationEnqueueFailureEvent {
// action: GenerationAction::ImageTo3d,
// service: GenerationServiceProvider::Fal,
// model: None,
// reason: None,
// };
//
// if let Err(err) = event.send(app) {
// error!("Failed to emit event: {:?}", err); // Fail open.
// }
//
// return Err(InternalObjectError::FalError(err));
// }
// }
Ok(())
}
@@ -0,0 +1,117 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::object::enqueue_image_to_3d_object_command::{EnqueueImageTo3dObjectModel, EnqueueImageTo3dObjectRequest};
use crate::core::commands::enqueue::object::internal_object_error::InternalObjectError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use log::{error, info, warn};
use tauri::AppHandle;
pub async fn handle_object_fal(
app: &AppHandle,
app_data_root: &AppDataRoot,
request: EnqueueImageTo3dObjectRequest,
fal_creds_manager: &FalCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalObjectError> {
let api_key = match fal_creds_manager.get_key()? {
Some(key) => key,
None => {
error!("No FAL API key is set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_fal_api_key(GenerationAction::ImageTo3d);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalObjectError::NeedsFalApiKey);
},
};
//let api_key = fal_creds_manager.get_key_required()
// .map_err(|err| {
// error!("EnqueueTextToImage FAL api key required: {:?}", err);
// InternalObjectError::NeedsFalApiKey
// })?;
let mut temp_download;
if let Some(media_token) = request.image_media_token {
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else {
return Err(InternalObjectError::AnyhowError(anyhow!("No image media token provided")));
}
info!("Calling FAL image to 3d ...");
let filename = temp_download.path().to_path_buf();
let result = match request.model {
None => {
return Err(InternalObjectError::NoModelSpecified);
}
Some(EnqueueImageTo3dObjectModel::Hunyuan3d2) => {
info!("enqueue Hunyuan 3D 2.0");
enqueue_hunyuan2_image_to_3d(Hunyuan2Args {
image_path: filename,
api_key: &api_key,
}).await
}
};
match result {
Ok(enqueued) => {
info!("Successfully enqueued text to image");
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::ImageTo3d,
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InternalObjectError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
}
}
Err(err) => {
error!("Failed to enqueue image to 3d: {:?}", err);
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::ImageTo3d,
service: GenerationServiceProvider::Fal,
model: None,
reason: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalObjectError::FalError(err));
}
}
Ok(())
}
@@ -0,0 +1,61 @@
use crate::core::artcraft_error::ArtcraftError;
use base64::DecodeError;
use errors::AnyhowError;
use fal_client::error::fal_error_plus::FalErrorPlus;
use storyteller_client::error::storyteller_error::StorytellerError;
#[derive(Debug)]
pub enum InternalObjectError {
NoModelSpecified,
NeedsFalApiKey,
NeedsSoraCredentials,
NeedsStorytellerCredentials,
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InternalObjectError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InternalObjectError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<StorytellerError> for InternalObjectError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
impl From<DecodeError> for InternalObjectError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InternalObjectError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
impl From<ArtcraftError> for InternalObjectError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
ArtcraftError::StorytellerError(e) => Self::StorytellerError(e),
_ => Self::ArtcraftError(value),
}
}
}
@@ -0,0 +1,4 @@
pub mod enqueue_image_to_3d_object_command;
pub mod handle_object_artcraft;
pub mod handle_object_fal;
pub mod internal_object_error;
@@ -0,0 +1,179 @@
use crate::core::commands::enqueue::object::handle_object_artcraft::handle_object_artcraft;
use crate::core::commands::enqueue::object::handle_object_fal::handle_object_fal;
use crate::core::commands::enqueue::video::handle_video_artcraft::handle_video_artcraft;
use crate::core::commands::enqueue::video::handle_video_fal::handle_video_fal;
use crate::core::commands::enqueue::video::internal_video_error::InternalVideoError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
use log::{debug, error, info, warn};
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::creds::credential_migration::CredentialMigrationRef;
use openai_sora_client::recipes::image_remix_with_session_auto_renew::{image_remix_with_session_auto_renew, ImageRemixAutoRenewRequest};
use openai_sora_client::recipes::image_upload_from_file_with_session_auto_renew::{image_upload_from_file_with_session_auto_renew, ImageUploadFromFileAutoRenewRequest};
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::{Cursor, Read, Write};
use std::ops::Add;
use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tempfile::NamedTempFile;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Deserialize)]
pub struct EnqueueImageToVideoRequest {
/// Image media file; the image to remove the background from.
/// TODO: In the future we may support base64 images, URLs, or file paths here.
pub image_media_token: Option<MediaFileToken>,
/// The model to use.
pub model: Option<EnqueueImageToVideoModel>,
}
#[derive(Deserialize, Debug, Copy, Clone)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueImageToVideoModel {
#[serde(rename = "kling1_6")]
Kling16,
}
#[derive(Serialize)]
pub struct EnqueueImageToVideoSuccessResponse {
}
impl SerializeMarker for EnqueueImageToVideoSuccessResponse {}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum EnqueueImageToVideoErrorType {
/// Caller didn't specify a model
ModelNotSpecified,
/// Generic server error
ServerError,
/// No Fal API key available
NeedsFalApiKey,
/// Fal had an API error
FalError,
/// Needs to be logged into Artcraft
NeedsStorytellerCredentials,
}
#[tauri::command]
pub async fn enqueue_image_to_video_command(
app: AppHandle,
request: EnqueueImageToVideoRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> Response<EnqueueImageToVideoSuccessResponse, EnqueueImageToVideoErrorType, ()> {
info!("enqueue_image_to_video_command called");
let result = handle_request(
&app,
request,
&app_data_root,
&fal_creds_manager,
&storyteller_creds_manager,
&fal_task_queue,
).await;
match result {
Err(err) => {
error!("error: {:?}", err);
let mut status = CommandErrorStatus::ServerError;
let mut error_type = EnqueueImageToVideoErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
match err {
InternalVideoError::NoModelSpecified => {
status = CommandErrorStatus::BadRequest;
error_type = EnqueueImageToVideoErrorType::ModelNotSpecified;
error_message = "No model specified for image generation";
}
InternalVideoError::NeedsFalApiKey => {
status = CommandErrorStatus::Unauthorized;
error_type = EnqueueImageToVideoErrorType::NeedsFalApiKey;
error_message = "You need to set a FAL api key";
},
InternalVideoError::NeedsStorytellerCredentials => {
status = CommandErrorStatus::Unauthorized;
error_type = EnqueueImageToVideoErrorType::NeedsStorytellerCredentials;
error_message = "You need to be logged into Artcraft.";
}
_ => {}, // Fall-through
}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
Ok(()) => {
Ok(EnqueueImageToVideoSuccessResponse {}.into())
}
}
}
pub async fn handle_request(
app: &AppHandle,
request: EnqueueImageToVideoRequest,
app_data_root: &AppDataRoot,
fal_creds_manager: &FalCredentialManager,
storyteller_creds_manager: &StorytellerCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalVideoError> {
if fal_creds_manager.has_apparent_api_token()? {
handle_video_fal(&app, app_data_root, request, fal_creds_manager, fal_task_queue).await?;
} else {
handle_video_artcraft(request, &app, app_data_root, storyteller_creds_manager).await?;
}
Ok(())
}
@@ -0,0 +1,138 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::object::enqueue_image_to_3d_object_command::{EnqueueImageTo3dObjectModel, EnqueueImageTo3dObjectRequest};
use crate::core::commands::enqueue::video::enqueue_image_to_video_command::{EnqueueImageToVideoModel, EnqueueImageToVideoRequest};
use crate::core::commands::enqueue::video::internal_video_error::InternalVideoError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use artcraft_api_defs::generate::object::generate_hunyuan_2_image_to_3d::GenerateHunyuan2ImageTo3dRequest;
use artcraft_api_defs::generate::video::generate_kling_1_6_pro_image_to_video::{GenerateKling16ProAspectRatio, GenerateKling16ProImageToVideoRequest};
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::requests::queue::video_gen::enqueue_kling_16_pro_image_to_video::Kling16ProAspectRatio;
use idempotency::uuid::generate_random_uuid;
use log::{error, info, warn};
use storyteller_client::generate::object::generate_hunyuan2_image_to_3d::generate_hunyuan2_image_to_3d;
use storyteller_client::generate::video::generate_kling_16_pro_image_to_video::generate_kling_16_pro_image_to_video;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, State};
pub async fn handle_video_artcraft(
request: EnqueueImageToVideoRequest,
app: &AppHandle,
app_data_root: &AppDataRoot,
storyteller_creds_manager: &StorytellerCredentialManager,
) -> Result<(), InternalVideoError> {
let creds = match storyteller_creds_manager.get_credentials()? {
Some(creds) => creds,
None => {
error!("No Artcraft credentials are set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_artcraft_credentials(GenerationAction::GenerateVideo);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalVideoError::NeedsStorytellerCredentials);
},
};
//let api_key = fal_creds_manager.get_key_required()
// .map_err(|err| {
// error!("EnqueueTextToImage FAL api key required: {:?}", err);
// InternalVideoError::NeedsFalApiKey
// })?;
info!("Calling Artcraft image to video...");
let uuid_idempotency_token = generate_random_uuid();
let mut selected_model = None;
let job_token = match request.model {
None => {
return Err(InternalVideoError::NoModelSpecified);
}
Some(EnqueueImageToVideoModel::Kling16) => {
info!("enqueue Kling 1.6");
selected_model = Some(GenerationModel::Kling1_6);
let request = GenerateKling16ProImageToVideoRequest {
uuid_idempotency_token,
media_file_token: request.image_media_token,
aspect_ratio: Some(GenerateKling16ProAspectRatio::WideSixteenNine),
prompt: None,
};
let result = generate_kling_16_pro_image_to_video(
&ApiHost::Storyteller,
Some(&creds),
request,
).await;
match result {
Ok(enqueued) => {
info!("Successfully enqueued Artcraft Kling 1.6 video generation");
enqueued.inference_job_token
}
Err(err) => {
error!("Failed to use Artcraft Kling 1.6 video generation: {:?}", err);
return Err(InternalVideoError::StorytellerError(err));
}
}
}
};
info!("Successfully enqueued video generation");
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateVideo,
service: GenerationServiceProvider::Artcraft,
model: selected_model,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
//if let Err(err) = fal_task_queue.insert(&enqueued) {
// error!("Failed to enqueue task: {:?}", err);
// return Err(InternalVideoError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
//}
// match result {
// Ok(enqueued) => {
// }
// Err(err) => {
// error!("Failed to enqueue image to 3d: {:?}", err);
//
// let event = GenerationEnqueueFailureEvent {
// action: GenerationAction::ImageTo3d,
// service: GenerationServiceProvider::Fal,
// model: None,
// reason: None,
// };
//
// if let Err(err) = event.send(app) {
// error!("Failed to emit event: {:?}", err); // Fail open.
// }
//
// return Err(InternalVideoError::FalError(err));
// }
// }
Ok(())
}
@@ -0,0 +1,124 @@
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::{EnqueueTextToImageModel, EnqueueTextToImageRequest};
use crate::core::commands::enqueue::video::enqueue_image_to_video_command::{EnqueueImageToVideoModel, EnqueueImageToVideoRequest};
use crate::core::commands::enqueue::video::internal_video_error::InternalVideoError;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::download_media_file_to_temp_dir::download_media_file_to_temp_dir;
use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::state::sora_task_queue::SoraTaskQueue;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use anyhow::anyhow;
use fal_client::creds::fal_api_key::FalApiKey;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use fal_client::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use fal_client::requests::queue::video_gen::enqueue_kling_16_pro_image_to_video::{enqueue_kling_16_pro_image_to_video, Kling16ProArgs, Kling16ProAspectRatio, Kling16ProDuration};
use log::{error, info, warn};
use tauri::AppHandle;
pub async fn handle_video_fal(
app: &AppHandle,
app_data_root: &AppDataRoot,
request: EnqueueImageToVideoRequest,
fal_creds_manager: &FalCredentialManager,
fal_task_queue: &FalTaskQueue,
) -> Result<(), InternalVideoError> {
let api_key = match fal_creds_manager.get_key()? {
Some(key) => key,
None => {
error!("No FAL API key is set. Can't perform action.");
let event =
GenerationEnqueueFailureEvent::no_fal_api_key(GenerationAction::GenerateVideo);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalVideoError::NeedsFalApiKey);
},
};
//let api_key = fal_creds_manager.get_key_required()
// .map_err(|err| {
// error!("EnqueueTextToImage FAL api key required: {:?}", err);
// InternalVideoError::NeedsFalApiKey
// })?;
let mut temp_download;
if let Some(media_token) = request.image_media_token {
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else {
return Err(InternalVideoError::AnyhowError(anyhow!("No image media token provided")));
}
info!("Calling FAL image to 3d ...");
let filename = temp_download.path().to_path_buf();
let mut selected_model = None;
let result = match request.model {
None => {
return Err(InternalVideoError::NoModelSpecified);
}
Some(EnqueueImageToVideoModel::Kling16) => {
info!("enqueue Kling 1.6 image to video");
selected_model = Some(GenerationModel::Kling1_6);
enqueue_kling_16_pro_image_to_video(Kling16ProArgs {
image_path: filename,
api_key: &api_key,
duration: Kling16ProDuration::Default,
aspect_ratio: Kling16ProAspectRatio::WideSixteenNine,
prompt: "",
}).await
}
};
match result {
Ok(enqueued) => {
info!("Successfully enqueued Kling 1.6 image to video");
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateVideo,
service: GenerationServiceProvider::Fal,
model: selected_model,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
if let Err(err) = fal_task_queue.insert(&enqueued) {
error!("Failed to enqueue task: {:?}", err);
return Err(InternalVideoError::AnyhowError(anyhow!("Failed to enqueue task: {:?}", err)));
}
}
Err(err) => {
error!("Failed to enqueue image to video: {:?}", err);
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateVideo,
service: GenerationServiceProvider::Fal,
model: selected_model,
reason: None,
};
if let Err(err) = event.send(app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(InternalVideoError::FalError(err));
}
}
Ok(())
}
@@ -0,0 +1,61 @@
use crate::core::artcraft_error::ArtcraftError;
use base64::DecodeError;
use errors::AnyhowError;
use fal_client::error::fal_error_plus::FalErrorPlus;
use storyteller_client::error::storyteller_error::StorytellerError;
#[derive(Debug)]
pub enum InternalVideoError {
NoModelSpecified,
NeedsFalApiKey,
NeedsSoraCredentials,
NeedsStorytellerCredentials,
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InternalVideoError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<FalErrorPlus> for InternalVideoError {
fn from(value: FalErrorPlus) -> Self {
Self::FalError(value)
}
}
impl From<StorytellerError> for InternalVideoError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
impl From<DecodeError> for InternalVideoError {
fn from(value: DecodeError) -> Self {
Self::DecodeError(value)
}
}
impl From<std::io::Error> for InternalVideoError {
fn from(value: std::io::Error) -> Self {
Self::IoError(value)
}
}
impl From<ArtcraftError> for InternalVideoError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
ArtcraftError::StorytellerError(e) => Self::StorytellerError(e),
_ => Self::ArtcraftError(value),
}
}
}
@@ -0,0 +1,4 @@
pub mod enqueue_image_to_video_command;
pub mod handle_video_artcraft;
pub mod handle_video_fal;
pub mod internal_video_error;
@@ -1,4 +1,5 @@
use crate::core::commands::command_response_wrapper::{CommandResult, InfallibleResponse, SerializeMarker};
use crate::core::commands::response::shorthand::InfallibleResponse;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::state::os_platform::OsPlatform;
use crate::services::fal::commands::fal_background_removal_command::FalBackgroundRemovalSuccessResponse;
use base64::prelude::BASE64_STANDARD;
@@ -1,6 +1,6 @@
pub mod app_preferences;
pub mod command_response_wrapper;
pub mod enqueue;
pub mod flip_image;
pub mod get_build_info_command;
pub mod platform_info_command;
pub mod response;
@@ -1,3 +1,5 @@
use crate::core::commands::response::shorthand::InfallibleResponse;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::state::os_platform::OsPlatform;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
@@ -13,6 +15,8 @@ pub struct PlatformInfoResponse {
pub webview_runtime: WebviewRuntime,
}
impl SerializeMarker for PlatformInfoResponse {}
#[derive(Copy, Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum DetectedOs {
@@ -36,7 +40,7 @@ pub enum WebviewRuntime {
}
#[tauri::command]
pub fn platform_info_command() -> Result<PlatformInfoResponse, String> {
pub fn platform_info_command() -> InfallibleResponse<PlatformInfoResponse> {
info!("platform_info_command called...");
let os_platform = match OsPlatform::maybe_get() {
@@ -55,9 +59,9 @@ pub fn platform_info_command() -> Result<PlatformInfoResponse, String> {
DetectedOs::Unknown => WebviewRuntime::Unknown,
};
Ok(PlatformInfoResponse {
PlatformInfoResponse {
os_platform,
webview_runtime,
})
}.into()
}
@@ -0,0 +1,42 @@
use serde::Serialize;
/// Statuses for errorful commands.
#[derive(Serialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum CommandErrorStatus {
BadRequest, // 400
Unauthorized, // 401
TooManyRequests, // 429
ServerError, // 500
}
#[derive(Serialize, Debug)]
pub struct CommandErrorResponseWrapper<ErrType: Serialize, ErrPayload: Serialize> {
/// Both "success" and "error" types have a `status` field for the frontend.
/// We constrain the value types for successes and failures.
pub status: CommandErrorStatus,
#[serde(skip_serializing_if = "Option::is_none")]
pub error_message: Option<String>,
/// Represents a "type" of error, preferably an enum without payloads.
/// Preferably this serializes to a string.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_type: Option<ErrType>,
/// Represents a full error details object or payload.
/// This could be a structure, message, or whatever.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_details: Option<ErrPayload>,
}
impl From<&str> for CommandErrorResponseWrapper<(), ()> {
fn from(value: &str) -> Self {
CommandErrorResponseWrapper {
status: CommandErrorStatus::BadRequest, // NB: Default to 500 error.
error_message: Some(value.to_string()),
error_type: None,
error_details: None,
}
}
}
@@ -0,0 +1,3 @@
pub mod failure_response_wrapper;
pub mod shorthand;
pub mod success_response_wrapper;
@@ -0,0 +1,32 @@
use crate::core::commands::response::failure_response_wrapper::CommandErrorResponseWrapper;
use crate::core::commands::response::success_response_wrapper::CommandSuccessResponseWrapper;
// TODO(needs command response refactor):
// - get_app_preferences_command
// - update_app_preferences_command
// - flip_image
// - check_sora_session_command
// - open_sora_login_command
// - sora_image_generation_command
// TODO(already updated, need to update frontend):
// - platform_info_command
// TODO(simplify error types):
// - enqueue_text_to_image_command
// - fal_background_removal_command
// - fal_hunyuan_image_to_3d_command
// - fal_kling_image_to_video_command
// - get_fal_api_key_command
// - set_fal_api_key_command
// - sora_image_remix_command
/// Easy to use infallible response.
pub type InfallibleResponse<SuccessPayload> = CommandSuccessResponseWrapper<SuccessPayload>;
/// Easy to use Result<T, E> type.
pub type Response<SuccessPayload, ErrType, ErrPayload> =
Result<CommandSuccessResponseWrapper<SuccessPayload>, CommandErrorResponseWrapper<ErrType, ErrPayload>>;
/// No inner payloads for this type. Just strings as messages.
pub type SimpleResponse = Response<(), (), ()>;
@@ -1,16 +1,4 @@
use serde::Serialize;
use std::error::Error;
use std::str::FromStr;
/// Easy to use Result<T, E> type.
pub type CommandResult<SuccessPayload, ErrType, ErrPayload> =
Result<CommandSuccessResponseWrapper<SuccessPayload>, CommandErrorResponseWrapper<ErrType, ErrPayload>>;
/// Easy to use infallible response.
pub type InfallibleResponse<SuccessPayload> = CommandSuccessResponseWrapper<SuccessPayload>;
/// No inner payloads for this type. Just strings as messages.
pub type SimpleCommandResult = CommandResult<(), (), ()>;
/// Statuses for successful commands.
#[derive(Serialize, Clone, Debug)]
@@ -19,49 +7,30 @@ pub enum CommandSuccessStatus {
Success, // 200
}
/// Statuses for errorful commands.
#[derive(Serialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum CommandErrorStatus {
BadRequest, // 400
Unauthorized, // 401
TooManyRequests, // 429
ServerError, // 500
}
#[derive(Serialize, Debug)]
pub struct CommandSuccessResponseWrapper<T: Serialize> {
/// Both "success" and "error" types have a `status` field for the frontend.
/// We constrain the value types for successes and failures.
pub status: CommandSuccessStatus,
#[serde(skip_serializing_if = "Option::is_none")]
pub success_message: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub payload: Option<T>,
}
#[derive(Serialize, Debug)]
pub struct CommandErrorResponseWrapper<ErrType: Serialize, ErrPayload: Serialize> {
/// Both "success" and "error" types have a `status` field for the frontend.
pub status: CommandErrorStatus,
#[serde(skip_serializing_if = "Option::is_none")]
pub error_message: Option<String>,
/// Represents a "type" of error, preferably an enum without payloads.
/// Preferably this serializes to a string.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_type: Option<ErrType>,
/// Represents a full error details object or payload.
/// This could be a structure, message, or whatever.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_details: Option<ErrPayload>,
}
/// NB: This is a hack because of `()` (a.k.a. the "unit" type) and `Serialize` trait coherence and overlap.
/// We cannot currently implement a trait for `()` and generic `Serialize` because `()` itself implements
/// `Serialize`. Rust's `"specialization"`, which would let us account for this, is an unstable, nightly-only
/// feature. So instead we manually mark all types other than `()` with `SerializeMarker` as a hack to
/// distinguish the two.
pub trait SerializeMarker : Serialize {}
impl CommandSuccessResponseWrapper<()> {
pub fn empty_success() -> Self {
Self::empty()
}
pub fn empty() -> Self {
CommandSuccessResponseWrapper {
status: CommandSuccessStatus::Success,
@@ -71,13 +40,6 @@ impl CommandSuccessResponseWrapper<()> {
}
}
/// NB: This is a hack because of `()` (a.k.a. the "unit" type) and `Serialize` trait coherence and overlap.
/// We cannot currently implement a trait for `()` and generic `Serialize` because `()` itself implements
/// `Serialize`. Rust's `"specialization"`, which would let us account for this, is an unstable, nightly-only
/// feature. So instead we manually mark all types other than `()` with `SerializeMarker` as a hack to
/// distinguish the two.
pub trait SerializeMarker : Serialize {}
// NB: () is Serialize! So we can't broadly implement just `T: Serialize`.
impl From<()> for CommandSuccessResponseWrapper<()> {
fn from(_: ()) -> Self {
@@ -111,20 +73,9 @@ impl From<&str> for CommandSuccessResponseWrapper<String> {
}
}
impl From<&str> for CommandErrorResponseWrapper<(), ()> {
fn from(value: &str) -> Self {
CommandErrorResponseWrapper {
status: CommandErrorStatus::BadRequest, // NB: Default to 500 error.
error_message: Some(value.to_string()),
error_type: None,
error_details: None,
}
}
}
#[cfg(test)]
mod tests {
#[test]
fn test_empty_tuple_serialize() {
let response = super::CommandSuccessResponseWrapper::<()>::from(());
@@ -0,0 +1,51 @@
use crate::core::events::sendable_event_error::SendableEventError;
use log::info;
use serde::Serialize;
use std::fmt::Debug;
use tauri::{AppHandle, Emitter};
// Tagged enums here are waaay too "Rustic"
// #[derive(Clone, Serialize)]
// #[serde(rename_all = "camelCase", rename_all_fields = "camelCase", tag = "event", content = "data")]
// pub enum BasicEvent<T: Serialize> {
// Success(T),
// Failure(T),
// }
#[derive(Copy, Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum BasicEventStatus {
Success,
Failure,
}
/// Wrap the event with "status" and "data" fields.
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
struct BasicEventWrapper<T: Serialize + Debug> {
status: BasicEventStatus,
data: T,
}
pub trait BasicSendableEvent : Clone + Debug + Serialize {
/// This is the name of the event that the frontend subscribes to.
const FRONTEND_EVENT_NAME: &'static str;
/// This is the status of the event: Success, Failure, etc.
const EVENT_STATUS: BasicEventStatus;
/// Default implementation of send().
/// This serializes and sends the event to the frontend.
fn send(&self, app: &AppHandle) -> Result<(), SendableEventError> {
let wrapped = BasicEventWrapper {
status: Self::EVENT_STATUS,
data: self.clone(),
};
info!("Sending event to frontend: {} ; payload = {:?}", Self::FRONTEND_EVENT_NAME, wrapped);
let result = app.emit(Self::FRONTEND_EVENT_NAME, wrapped);
if let Err(err) = result {
return Err(SendableEventError::from(err));
}
Ok(())
}
}
@@ -0,0 +1,31 @@
use serde::Serialize;
#[derive(Debug, Copy, Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum GenerationServiceProvider {
Artcraft,
Fal,
Sora,
}
#[derive(Debug, Copy, Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum GenerationModel {
#[serde(rename = "flux_pro_1.1_ultra")]
FluxPro11Ultra,
#[serde(rename = "kling_1.6")]
Kling1_6,
#[serde(rename = "kling_2.0")]
Kling2_0,
Sora,
}
#[derive(Debug, Copy, Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum GenerationAction {
GenerateImage,
GenerateVideo,
RemoveBackground,
#[serde(rename = "image_to_3d")]
ImageTo3d,
}
@@ -0,0 +1,16 @@
use crate::core::events::basic_sendable_event_trait::{BasicEventStatus, BasicSendableEvent};
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use serde_derive::Serialize;
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub struct GenerationCompleteEvent {
pub action: Option<GenerationAction>,
pub service: GenerationServiceProvider,
pub model: Option<GenerationModel>,
}
impl BasicSendableEvent for GenerationCompleteEvent {
const FRONTEND_EVENT_NAME: &'static str = "generation-complete-event";
const EVENT_STATUS: BasicEventStatus = BasicEventStatus::Success;
}
@@ -0,0 +1,39 @@
use crate::core::events::basic_sendable_event_trait::{BasicEventStatus, BasicSendableEvent};
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use serde_derive::Serialize;
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub struct GenerationEnqueueFailureEvent {
pub action: GenerationAction,
pub service: GenerationServiceProvider,
pub model: Option<GenerationModel>,
/// User-facing reason for the failure, if available.
pub reason: Option<String>,
}
impl BasicSendableEvent for GenerationEnqueueFailureEvent {
const FRONTEND_EVENT_NAME: &'static str = "generation-enqueue-failure-event";
const EVENT_STATUS: BasicEventStatus = BasicEventStatus::Failure;
}
impl GenerationEnqueueFailureEvent {
pub fn no_artcraft_credentials(action: GenerationAction) -> Self {
Self {
action,
service: GenerationServiceProvider::Artcraft,
model: None,
reason: Some("Please log into Artcraft.".to_string()),
}
}
pub fn no_fal_api_key(action: GenerationAction) -> Self {
Self {
action,
service: GenerationServiceProvider::Fal,
model: None,
reason: Some("No FAL API key is set. Configure this in your settings.".to_string()),
}
}
}
@@ -0,0 +1,16 @@
use crate::core::events::basic_sendable_event_trait::{BasicEventStatus, BasicSendableEvent};
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use serde_derive::Serialize;
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub struct GenerationEnqueueSuccessEvent {
pub action: GenerationAction,
pub service: GenerationServiceProvider,
pub model: Option<GenerationModel>,
}
impl BasicSendableEvent for GenerationEnqueueSuccessEvent {
const FRONTEND_EVENT_NAME: &'static str = "generation-enqueue-success-event";
const EVENT_STATUS: BasicEventStatus = BasicEventStatus::Success;
}
@@ -0,0 +1,19 @@
use crate::core::events::basic_sendable_event_trait::{BasicEventStatus, BasicSendableEvent};
use crate::core::events::generation_events::common::{GenerationAction, GenerationModel, GenerationServiceProvider};
use serde_derive::Serialize;
#[derive(Clone, Debug, Serialize)]
#[serde(rename_all = "snake_case")]
pub struct GenerationFailedEvent {
pub action: GenerationAction,
pub service: GenerationServiceProvider,
pub model: Option<GenerationModel>,
/// User-facing reason for the failure, if available.
pub reason: Option<String>,
}
impl BasicSendableEvent for GenerationFailedEvent {
const FRONTEND_EVENT_NAME: &'static str = "generation-failed-event";
const EVENT_STATUS: BasicEventStatus = BasicEventStatus::Failure;
}
@@ -0,0 +1,5 @@
pub mod common;
pub mod generation_enqueue_failure_event;
pub mod generation_enqueue_success_event;
pub mod generation_complete_event;
pub mod generation_failed_event;
@@ -1,3 +1,4 @@
pub mod notification_event;
pub mod basic_sendable_event_trait;
pub mod generation_events;
pub mod sendable_event_error;
pub mod sendable_event_trait;
@@ -1,45 +0,0 @@
use serde::Serialize;
#[derive(Clone, Copy, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum NotificationModelType {
ImageSegmentation,
TextEncoder,
TextTokenizer,
Unet,
Vae,
Json,
ClipEncoder,
FluxModel,
}
#[derive(Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum NotificationEvent<'a> {
ModelDownloadStarted {
model_name: &'a str,
model_type: NotificationModelType,
},
ModelDownloadProgress {
model_name: &'a str,
model_type: NotificationModelType,
currently_downloaded_bytes: usize,
total_file_bytes: usize,
},
ModelDownloadComplete {
model_name: &'a str,
model_type: NotificationModelType,
},
ModelLoading {
model_name: String,
model_type: NotificationModelType,
},
ModelInference {
model_name: String,
model_type: NotificationModelType,
},
ModelInferenceComplete {
model_name: String,
model_type: NotificationModelType,
},
}
@@ -1,15 +1,17 @@
use log::info;
use crate::core::events::sendable_event_error::SendableEventError;
use log::info;
use serde::Serialize;
use tauri::{AppHandle, Emitter};
/// Implement this on a Serializable object to be able to send it as an event to the frontend.
#[deprecated(note = "Use BasicSendableEvent instead as it wraps the event with status and data fields.")]
pub trait SendableEvent : Serialize {
/// This is the name of the event that the frontend subscribes to.
const FRONTEND_EVENT_NAME: &'static str;
/// Default implementation of send().
/// This serializes and sends the event to the frontend.
#[deprecated(note = "Use BasicSendableEvent instead as it wraps the event with status and data fields.")]
fn send(&self, app: &AppHandle) -> Result<(), SendableEventError> {
info!("Sending event to frontend: {}", Self::FRONTEND_EVENT_NAME);
let result = app.emit(Self::FRONTEND_EVENT_NAME, self);
+1
View File
@@ -1,3 +1,4 @@
pub mod artcraft_error;
pub mod commands;
pub mod events;
pub mod state;
@@ -33,6 +33,7 @@ pub struct AppDataRoot {
impl AppDataRoot {
pub fn create_default() -> anyhow::Result<Self> {
let directory = get_default_data_dir()?;
println!("App data directory: {:?}", directory);
Self::create_existing(directory)
}
@@ -23,6 +23,15 @@ pub async fn main_window_thread(
let window_pos_slot: CloneSlot<MainWindowPosition> = CloneSlot::empty();
let app_startup_time = AppStartupTime::new();
// Emit debugging information to the logs (do not remove this!)
info!("git commit id: {:?}", build_metadata::git_commit_id()
.unwrap_or_else(|| "unknown"));
info!("git commit timestamp: {:?}", build_metadata::git_commit_timestamp()
.map(|t| t.to_string())
.unwrap_or_else(|| "unknown".to_string()));
info!("build timestamp: {:?}", build_metadata::build_timestamp()
.to_string());
loop {
for (window_name, window) in app.windows() {
if window_name == MAIN_WINDOW_NAME {
@@ -1,7 +1,7 @@
use crate::core::artcraft_error::ArtcraftError;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use errors::AnyhowResult;
use storyteller_client::media_files::get_media_file::get_media_file;
use storyteller_client::utils::api_host::ApiHost;
use tempfile::NamedTempFile;
@@ -9,7 +9,7 @@ use tokens::tokens::media_files::MediaFileToken;
// TODO: Better more concrete error handling
pub async fn download_media_file_to_temp_dir(app_data_root: &AppDataRoot, token: &MediaFileToken) -> AnyhowResult<NamedTempFile> {
pub async fn download_media_file_to_temp_dir(app_data_root: &AppDataRoot, token: &MediaFileToken) -> Result<NamedTempFile, ArtcraftError> {
let response = get_media_file(&ApiHost::Storyteller, token).await?;
let media_file_url = &response.media_file.media_links.cdn_url;
@@ -1,15 +1,12 @@
use crate::core::artcraft_error::ArtcraftError;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use errors::AnyhowResult;
use mimetypes::mimetype_info::mimetype_info::MimetypeInfo;
use std::io::Write;
use tempfile::NamedTempFile;
use web_base64::web_base64_decode::web_base64_decode;
// TODO: Retain error types (io/decode)
pub async fn save_base64_image_to_temp_dir(app_data_root: &AppDataRoot, base64_image: String) -> AnyhowResult<NamedTempFile> {
let bytes = BASE64_STANDARD.decode(base64_image)?;
pub async fn save_base64_image_to_temp_dir(app_data_root: &AppDataRoot, base64_image: String) -> Result<NamedTempFile, ArtcraftError> {
let bytes = web_base64_decode(&base64_image)?;
let extension = MimetypeInfo::get_for_bytes(&bytes)
.map(|info| info.file_extension())
+4
View File
@@ -6,6 +6,8 @@ use tauri::Manager;
use crate::core::commands::app_preferences::get_app_preferences_command::get_app_preferences_command;
use crate::core::commands::app_preferences::update_app_preference_command::update_app_preferences_command;
use crate::core::commands::enqueue::image::enqueue_text_to_image_command::enqueue_text_to_image_command;
use crate::core::commands::enqueue::object::enqueue_image_to_3d_object_command::enqueue_image_to_3d_object_command;
use crate::core::commands::enqueue::video::enqueue_image_to_video_command::enqueue_image_to_video_command;
use crate::core::commands::flip_image::flip_image;
use crate::core::commands::get_build_info_command::get_build_info_command;
use crate::core::commands::platform_info_command::platform_info_command;
@@ -161,6 +163,8 @@ pub fn run() {
.manage(storyteller_creds_manager_3)
.invoke_handler(tauri::generate_handler![
check_sora_session_command,
enqueue_image_to_3d_object_command,
enqueue_image_to_video_command,
enqueue_text_to_image_command,
fal_background_removal_command,
fal_hunyuan_image_to_3d_command,
@@ -1,4 +1,11 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::artcraft_error::ArtcraftError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
@@ -8,8 +15,6 @@ use crate::core::utils::save_base64_image_to_temp_dir::save_base64_image_to_temp
use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -20,7 +25,7 @@ use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use fal_client::requests::queue::remove_background_rembg::remove_background_rembg;
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
@@ -45,6 +50,7 @@ use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::error::storyteller_error::StorytellerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
@@ -85,11 +91,12 @@ pub enum BackgroundRemovalErrorType {
#[tauri::command]
pub async fn fal_background_removal_command(
app: AppHandle,
request: FalBackgroundRemovalRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
) -> CommandResult<FalBackgroundRemovalSuccessResponse, BackgroundRemovalErrorType, ()> {
) -> Response<FalBackgroundRemovalSuccessResponse, BackgroundRemovalErrorType, ()> {
info!("fal_background_removal_command called; image media token: {:?}", request.image_media_token);
@@ -99,6 +106,13 @@ pub async fn fal_background_removal_command(
if !has_credentials {
warn!("No API key found");
let event =
GenerationEnqueueFailureEvent::no_fal_api_key(GenerationAction::RemoveBackground);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(CommandErrorResponseWrapper {
status: CommandErrorStatus::Unauthorized,
error_message: Some("You need to set a FAL api key".to_string()),
@@ -107,6 +121,16 @@ pub async fn fal_background_removal_command(
});
}
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::RemoveBackground,
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
let result = remove_background(
request,
&app_data_root,
@@ -118,20 +142,21 @@ pub async fn fal_background_removal_command(
Err(err) => {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::RemoveBackground,
service: GenerationServiceProvider::Fal,
model: None,
reason: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = BackgroundRemovalErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
@@ -140,12 +165,6 @@ pub async fn fal_background_removal_command(
})
}
Ok(result) => {
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
Ok(FalBackgroundRemovalSuccessResponse {
media_token: result.0.media_file.token,
cdn_url: result.0.media_file.media_links.cdn_url,
@@ -159,9 +178,10 @@ pub async fn fal_background_removal_command(
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InnerError {
@@ -176,9 +196,9 @@ impl From<FalErrorPlus> for InnerError {
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
impl From<StorytellerError> for InnerError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -194,6 +214,17 @@ impl From<std::io::Error> for InnerError {
}
}
impl From<ArtcraftError> for InnerError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
_ => Self::ArtcraftError(value),
}
}
}
pub async fn remove_background(
request: FalBackgroundRemovalRequest,
app_data_root: &AppDataRoot,
@@ -207,9 +238,11 @@ pub async fn remove_background(
let mut temp_download;
if let Some(media_token) = request.image_media_token {
info!("From media token: {:?}", &media_token);
temp_download = download_media_file_to_temp_dir(&app_data_root, &media_token).await?;
} else if let Some(base64_bytes) = request.base64_image {
info!("From base64 bytes...");
temp_download = save_base64_image_to_temp_dir(&app_data_root, base64_bytes).await?;
} else {
return Err(InnerError::AnyhowError(anyhow!("No image media token or base64 image provided")));
@@ -220,6 +253,8 @@ pub async fn remove_background(
let filename = temp_download.path().to_path_buf();
let result = remove_background_rembg(filename, &api_key).await?;
info!("Unpacking FAL result...");
let extension_with_dot = get_url_file_extension(&result.image_url)
.map(|ext| format!(".{}", ext))
@@ -254,4 +289,3 @@ pub async fn remove_background(
Ok((response, base64_bytes))
}
@@ -1,4 +1,10 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::artcraft_error::ArtcraftError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
@@ -9,8 +15,6 @@ use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -21,8 +25,7 @@ use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use fal_client::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
@@ -47,6 +50,7 @@ use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::error::storyteller_error::StorytellerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
@@ -75,12 +79,13 @@ pub enum HunyuanErrorType {
#[tauri::command]
pub async fn fal_hunyuan_image_to_3d_command(
app: AppHandle,
request: FalHunyuanImageTo3dRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
) -> CommandResult<(), HunyuanErrorType, ()> {
) -> Response<(), HunyuanErrorType, ()> {
info!("fal_hunyuan_image_to_3d_command called; image media token: {:?}", request.image_media_token);
@@ -109,20 +114,21 @@ pub async fn fal_hunyuan_image_to_3d_command(
if let Err(err) = result {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::ImageTo3d,
service: GenerationServiceProvider::Fal,
model: None,
reason: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = HunyuanErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
return Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
@@ -130,13 +136,17 @@ pub async fn fal_hunyuan_image_to_3d_command(
error_details: None,
})
}
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::ImageTo3d,
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
Ok(().into())
}
@@ -144,9 +154,10 @@ pub async fn fal_hunyuan_image_to_3d_command(
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InnerError {
@@ -161,9 +172,9 @@ impl From<FalErrorPlus> for InnerError {
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
impl From<StorytellerError> for InnerError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -179,6 +190,19 @@ impl From<std::io::Error> for InnerError {
}
}
impl From<ArtcraftError> for InnerError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
ArtcraftError::StorytellerError(e) => Self::StorytellerError(e),
_ => Self::ArtcraftError(value),
}
}
}
pub async fn image_to_3d(
request: FalHunyuanImageTo3dRequest,
app_data_root: &AppDataRoot,
@@ -188,7 +212,6 @@ pub async fn image_to_3d(
) -> Result<(), InnerError> {
let api_key = fal_creds_manager.get_key_required()?;
let creds = storyteller_creds_manager.get_credentials_required()?;
let mut temp_download;
@@ -1,4 +1,10 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult, SerializeMarker};
use crate::core::artcraft_error::ArtcraftError;
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
@@ -9,8 +15,6 @@ use crate::core::utils::simple_http_download::simple_http_download;
use crate::core::utils::simple_http_download_to_tempfile::simple_http_download_to_tempfile;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -21,8 +25,7 @@ use base64::prelude::BASE64_STANDARD;
use base64::{DecodeError, Engine};
use errors::{AnyhowError, AnyhowResult};
use fal_client::error::fal_error_plus::FalErrorPlus;
use fal_client::requests::video_gen::enqueue_kling_16_image_to_video::{enqueue_kling_16_image_to_video, Kling16Args, Kling16Duration};
use fal_client::requests::remove_background_rembg::remove_background_rembg;
use fal_client::requests::queue::video_gen::enqueue_kling_16_pro_image_to_video::{enqueue_kling_16_pro_image_to_video, Kling16ProArgs, Kling16ProAspectRatio, Kling16ProDuration};
use filesys::file_read_bytes::file_read_bytes;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, EncodableLayout, ImageReader};
@@ -47,6 +50,7 @@ use std::path::PathBuf;
use std::time::Duration;
use storyteller_client::error::api_error::ApiError;
use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::error::storyteller_error::StorytellerError;
use storyteller_client::media_files::get_media_file::{get_media_file, GetMediaFileSuccessResponse};
use storyteller_client::media_files::upload_image_media_file_from_file::upload_image_media_file_from_file;
use storyteller_client::utils::api_host::ApiHost;
@@ -80,12 +84,13 @@ pub enum KlingImageToVideoErrorType {
#[tauri::command]
pub async fn fal_kling_image_to_video_command(
app: AppHandle,
request: FalKlingImageToVideoRequest,
app_data_root: State<'_, AppDataRoot>,
fal_creds_manager: State<'_, FalCredentialManager>,
storyteller_creds_manager: State<'_, StorytellerCredentialManager>,
fal_task_queue: State<'_, FalTaskQueue>,
) -> CommandResult<(), KlingImageToVideoErrorType, ()> {
) -> Response<(), KlingImageToVideoErrorType, ()> {
info!("fal_kling_image_to_video_command called; image media token: {:?}", request.image_media_token);
@@ -114,20 +119,21 @@ pub async fn fal_kling_image_to_video_command(
if let Err(err) = result {
error!("error: {:?}", err);
//let event = SoraImageEnqueueFailureEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateVideo,
service: GenerationServiceProvider::Fal,
model: None,
reason: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = KlingImageToVideoErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
//match (err) {
// _ => {},
//}
return Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
@@ -136,12 +142,16 @@ pub async fn fal_kling_image_to_video_command(
})
}
//let event = SoraImageEnqueueSuccessEvent {};
//let result = event.send(&app);
//if let Err(err) = result {
// error!("Failed to emit event: {:?}", err);
//}
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateVideo,
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
Ok(().into())
}
@@ -149,9 +159,10 @@ pub async fn fal_kling_image_to_video_command(
enum InnerError {
FalError(FalErrorPlus),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
StorytellerError(StorytellerError),
DecodeError(DecodeError),
IoError(std::io::Error),
ArtcraftError(ArtcraftError),
}
impl From<AnyhowError> for InnerError {
@@ -166,9 +177,9 @@ impl From<FalErrorPlus> for InnerError {
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
impl From<StorytellerError> for InnerError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -184,6 +195,18 @@ impl From<std::io::Error> for InnerError {
}
}
impl From<ArtcraftError> for InnerError {
fn from(value: ArtcraftError) -> Self {
match value {
ArtcraftError::AnyhowError(e) => Self::AnyhowError(e),
ArtcraftError::DecodeError(e) => Self::DecodeError(e),
ArtcraftError::IoError(e) => Self::IoError(e),
ArtcraftError::StorytellerError(e) => Self::StorytellerError(e),
_ => Self::ArtcraftError(value),
}
}
}
pub async fn image_to_video(
request: FalKlingImageToVideoRequest,
app_data_root: &AppDataRoot,
@@ -210,10 +233,11 @@ pub async fn image_to_video(
let filename = temp_download.path().to_path_buf();
let enqueued = enqueue_kling_16_image_to_video(Kling16Args {
let enqueued = enqueue_kling_16_pro_image_to_video(Kling16ProArgs {
image_path: filename,
api_key: &api_key,
duration: Kling16Duration::Default,
duration: Kling16ProDuration::Default,
aspect_ratio: Kling16ProAspectRatio::WideSixteenNine,
prompt: request.prompt.as_deref().unwrap_or(""),
}).await?;
@@ -1,8 +1,9 @@
use crate::core::commands::command_response_wrapper::{CommandResult, SerializeMarker};
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::core::commands::response::shorthand::Response;
use crate::core::commands::response::success_response_wrapper::SerializeMarker;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
@@ -22,7 +23,7 @@ impl SerializeMarker for GetFalApiKeyResponse {}
#[tauri::command]
pub async fn get_fal_api_key_command(
creds_manager: State<'_, FalCredentialManager>,
) -> CommandResult<GetFalApiKeyResponse, (), ()> {
) -> Response<GetFalApiKeyResponse, (), ()> {
let key = get_key(&creds_manager)
.await
@@ -1,8 +1,8 @@
use crate::core::commands::command_response_wrapper::CommandResult;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::core::commands::response::shorthand::Response;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::app_preferences::preferred_download_directory::PreferredDownloadDirectory;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use anyhow::anyhow;
use errors::AnyhowResult;
use fal_client::creds::fal_api_key::FalApiKey;
@@ -21,7 +21,7 @@ pub struct SetFalApiKeyRequest{
pub async fn set_fal_api_key_command(
request: SetFalApiKeyRequest,
creds_manager: State<'_, FalCredentialManager>,
) -> CommandResult<(), (), ()> {
) -> Response<(), (), ()> {
info!("update_app_preferences_command called");
set_key(request, &creds_manager)
@@ -0,0 +1,37 @@
use crate::core::events::generation_events::common::GenerationAction;
use fal_client::model::fal_endpoint::FalEndpoint;
/// Adapter of generation types/categories for FAL
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FalGenerationType {
TextToImage,
ImageToImage,
TextToVideo,
ImageToVideo,
ImageTo3d,
ImageBackgroundRemoval,
}
impl FalGenerationType {
pub fn from_fal_endpoint(endpoint: &FalEndpoint) -> Option<Self> {
match endpoint {
FalEndpoint::FluxProUltraTextToImage => Some(Self::TextToImage),
FalEndpoint::Hunyuan3d2Base => Some(Self::ImageTo3d),
FalEndpoint::Kling16ImageToVideo => Some(Self::ImageToVideo),
FalEndpoint::Minimax01ImageToVideo => Some(Self::ImageToVideo),
FalEndpoint::RecraftV3TextToImage => Some(Self::TextToImage),
FalEndpoint::Other(_) => None,
}
}
pub fn to_event_generation_action(&self) -> GenerationAction {
match self {
Self::TextToImage => GenerationAction::GenerateImage,
Self::ImageToImage => GenerationAction::GenerateImage,
Self::TextToVideo => GenerationAction::GenerateVideo,
Self::ImageToVideo => GenerationAction::GenerateVideo,
Self::ImageTo3d => GenerationAction::ImageTo3d,
Self::ImageBackgroundRemoval => GenerationAction::RemoveBackground,
}
}
}
@@ -1,3 +1,4 @@
use crate::services::fal::state::fal_generation_type::FalGenerationType;
use chrono::{DateTime, Utc};
use errors::AnyhowResult;
use fal_client::model::enqueued_request::EnqueuedRequest;
@@ -10,12 +11,14 @@ use std::sync::{Arc, RwLock};
pub struct FalTaskDetails {
pub enqueue_time: DateTime<Utc>,
pub enqueued_request: EnqueuedRequest,
pub generation_type: Option<FalGenerationType>,
}
impl FalTaskDetails {
pub fn for_request(request: EnqueuedRequest) -> Self {
Self {
enqueue_time: Utc::now(),
generation_type: FalGenerationType::from_fal_endpoint(&request.fal_endpoint),
enqueued_request: request,
}
}
@@ -1,3 +1,4 @@
pub mod fal_credential_manager;
pub mod fal_task_queue;
pub mod read_fal_api_key_from_disk;
pub mod read_fal_api_key_from_disk;
mod fal_generation_type;
@@ -1,11 +1,11 @@
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_complete_event::GenerationCompleteEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::download_url_to_temp_dir::download_url_to_temp_dir;
use crate::services::fal::state::fal_credential_manager::FalCredentialManager;
use crate::services::fal::state::fal_task_queue::FalTaskQueue;
use crate::services::sora::events::sora_image_generation_complete_event::SoraImageGenerationCompleteEvent;
use crate::services::sora::events::sora_image_generation_failed_event::SoraImageGenerationFailedEvent;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::storyteller::state::storyteller_credential_manager::StorytellerCredentialManager;
use errors::AnyhowResult;
@@ -146,10 +146,17 @@ async fn polling_loop(
Ok(success) => {
info!("Uploaded to API backend: {:?}", success.media_file_token);
//let event = SoraImageGenerationCompleteEvent {
// media_file_token: success.media_file_token,
//};
//event.send(&app_handle)?;
let event = GenerationCompleteEvent {
//media_file_token: result.media_file_token,
action: task.generation_type
.map(|typ| typ.to_event_generation_action()),
service: GenerationServiceProvider::Fal,
model: None,
};
if let Err(err) = event.send(&app_handle) {
error!("Failed to send GenerationCompleteEvent: {:?}", err); // Fail open
}
succeeded_tasks.push(task_id);
}
@@ -158,7 +165,6 @@ async fn polling_loop(
continue;
}
}
}
@@ -228,7 +234,7 @@ async fn polling_loop(
fal_task_queue.remove_list(&succeeded_task_ids)?;
tokio::time::sleep(std::time::Duration::from_millis(6_000)).await;
tokio::time::sleep(std::time::Duration::from_millis(1_000)).await;
}
}
@@ -1,11 +1,14 @@
use crate::core::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult};
use crate::core::commands::response::failure_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus};
use crate::core::commands::response::shorthand::Response;
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_enqueue_failure_event::GenerationEnqueueFailureEvent;
use crate::core::events::generation_events::generation_enqueue_success_event::GenerationEnqueueSuccessEvent;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::core::utils::get_url_file_extension::get_url_file_extension;
use crate::core::utils::simple_http_download::simple_http_download;
use crate::services::sora::events::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::services::sora::events::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::services::sora::state::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::services::sora::state::sora_credential_holder::SoraCredentialHolder;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -36,6 +39,7 @@ use storyteller_client::error::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::get_media_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use storyteller_client::error::storyteller_error::StorytellerError;
use tokens::tokens::media_files::MediaFileToken;
const SORA_IMAGE_UPLOAD_TIMEOUT: Duration = Duration::from_millis(1000 * 30); // 30 seconds
@@ -94,7 +98,7 @@ pub async fn sora_image_remix_command(
app_data_root: State<'_, AppDataRoot>,
sora_creds_manager: State<'_, SoraCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> CommandResult<(), SoraImageRemixErrorType, ()> {
) -> Response<(), SoraImageRemixErrorType, ()> {
info!("image_generation_command called; scene media token: {:?}, additional images: {:?}, full request: {:?}",
&request.snapshot_media_token, &request.maybe_additional_images, &request);
@@ -107,9 +111,23 @@ pub async fn sora_image_remix_command(
if !has_credentials {
warn!("No apparently completed credentials found");
let error_message = "You need to log into Sora to continue. See the settings menu.";
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
reason: Some(error_message.to_string()),
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
return Err(CommandErrorResponseWrapper {
status: CommandErrorStatus::Unauthorized,
error_message: Some("You need to log into Sora to continue. See the settings menu.".to_string()),
error_message: Some(error_message.to_string()),
error_type: Some(SoraImageRemixErrorType::SoraLoginRequired),
error_details: None,
});
@@ -119,38 +137,42 @@ pub async fn sora_image_remix_command(
match result {
Err(err) => {
error!("error: {:?}", err);
error!("Sora image remix error: {:?}", err);
let event = SoraImageEnqueueFailureEvent {};
let result = event.send(&app);
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = SoraImageRemixErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
let mut error_message = "An error occurred. Please try again. If it continues, please tell our staff about the problem.";
match (err) {
InnerError::SoraError(SoraError::TooManyConcurrentTasks) => {
error_type = SoraImageRemixErrorType::TooManyConcurrentTasks;
status = CommandErrorStatus::ServerError;
error_message = "You already have work in progress. Please wait for it to finish.";
error_message = "You have too many Sora image generation tasks running. Please wait a moment.";
},
InnerError::SoraError(SoraError::SoraUsernameNotYetCreated) => {
error_type = SoraImageRemixErrorType::SoraUsernameNotYetCreated;
status = CommandErrorStatus::BadRequest;
error_message = "You need to create a username on Sora.com to continue.";
error_message = "Your Sora username is not yet created. Please visit the Sora.com website to create it first.";
},
InnerError::SoraError(SoraError::BadGateway(_) | SoraError::CloudFlareTimeout(_)) => {
error_type = SoraImageRemixErrorType::SoraIsHavingProblems;
status = CommandErrorStatus::ServerError;
error_message = "Sora is having problems. Please wait a moment and then retry.";
error_message = "Sora is having problems. Please try again later.";
}
_ => {},
}
let event = GenerationEnqueueFailureEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
reason: Some(error_message.to_string()),
};
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
@@ -159,11 +181,14 @@ pub async fn sora_image_remix_command(
})
}
Ok(_) => {
let event = SoraImageEnqueueSuccessEvent {};
let result = event.send(&app);
let event = GenerationEnqueueSuccessEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
};
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
if let Err(err) = event.send(&app) {
error!("Failed to emit event: {:?}", err); // Fail open.
}
Ok(().into())
@@ -175,7 +200,7 @@ pub async fn sora_image_remix_command(
enum InnerError {
SoraError(SoraError),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
StorytellerError(StorytellerError),
}
impl From<AnyhowError> for InnerError {
@@ -190,9 +215,9 @@ impl From<SoraError> for InnerError {
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
impl From<StorytellerError> for InnerError {
fn from(value: StorytellerError) -> Self {
Self::StorytellerError(value)
}
}
@@ -1,8 +1,8 @@
use crate::core::commands::app_preferences::get_app_preferences_command::AppPreferencesPayload;
use crate::core::commands::command_response_wrapper::{CommandResult, CommandSuccessResponseWrapper, SimpleCommandResult};
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::core::commands::response::shorthand::SimpleResponse;
use crate::core::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
use crate::services::sora::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use errors::{AnyhowError, AnyhowResult};
use log::{error, info};
@@ -17,7 +17,7 @@ use tauri::{AppHandle, State};
#[tauri::command]
pub async fn sora_logout_command(
sora_creds_manager: State<'_, SoraCredentialManager>,
) -> SimpleCommandResult {
) -> SimpleResponse {
info!("sora_logout_command called");
if let Err(err) = sora_creds_manager.purge_credentials_from_disk() {
@@ -1,5 +1 @@
pub mod sora_image_enqueue_failure_event;
pub mod sora_image_enqueue_success_event;
pub mod sora_image_generation_complete_event;
pub mod sora_image_generation_failed_event;
pub mod sora_login_success_event;
@@ -1,11 +0,0 @@
use crate::core::events::sendable_event_trait::SendableEvent;
use serde_derive::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueFailureEvent {
// TODO: Reason.
}
impl SendableEvent for SoraImageEnqueueFailureEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-enqueue-failure";
}
@@ -1,11 +0,0 @@
use crate::core::events::sendable_event_trait::SendableEvent;
use serde_derive::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueSuccessEvent {
// TODO: Token, etc.
}
impl SendableEvent for SoraImageEnqueueSuccessEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-enqueue-success";
}
@@ -1,12 +0,0 @@
use serde_derive::Serialize;
use tokens::tokens::media_files::MediaFileToken;
use crate::core::events::sendable_event_trait::SendableEvent;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationCompleteEvent {
pub media_file_token: MediaFileToken,
}
impl SendableEvent for SoraImageGenerationCompleteEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-generation-complete";
}
@@ -1,12 +0,0 @@
use serde_derive::Serialize;
use crate::core::events::sendable_event_trait::SendableEvent;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationFailedEvent {
pub prompt: String,
}
impl SendableEvent for SoraImageGenerationFailedEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-generation-failed";
}
@@ -2,7 +2,7 @@ use crate::core::state::data_dir::app_data_root::AppDataRoot;
use anyhow::anyhow;
use errors::AnyhowResult;
use log::info;
use openai_sora_client::requests::bearer::generate::generate_bearer_with_cookie;
use openai_sora_client::requests::bearer::generate_bearer_with_cookie::generate_bearer_with_cookie;
/// Loading the JWT bearer token for the first time.
pub async fn initialize_sora_jwt_bearer_token(app_data_root: &AppDataRoot) -> AnyhowResult<()> {
@@ -1,6 +1,8 @@
use crate::core::events::basic_sendable_event_trait::BasicSendableEvent;
use crate::core::events::generation_events::common::{GenerationAction, GenerationServiceProvider};
use crate::core::events::generation_events::generation_complete_event::GenerationCompleteEvent;
use crate::core::events::generation_events::generation_failed_event::GenerationFailedEvent;
use crate::core::events::sendable_event_trait::SendableEvent;
use crate::services::sora::events::sora_image_generation_complete_event::SoraImageGenerationCompleteEvent;
use crate::services::sora::events::sora_image_generation_failed_event::SoraImageGenerationFailedEvent;
use crate::core::state::data_dir::app_data_root::AppDataRoot;
use crate::core::state::data_dir::trait_data_subdir::DataSubdir;
use crate::services::sora::state::sora_credential_manager::SoraCredentialManager;
@@ -95,10 +97,19 @@ async fn polling_loop(
for task in failed_tasks.iter() {
if sora_task_queue.contains_key(&task.id)? {
let event = SoraImageGenerationFailedEvent {
prompt: task.prompt.clone(),
let event = GenerationFailedEvent {
action: GenerationAction::GenerateImage,
service: GenerationServiceProvider::Sora,
model: None,
reason: None,
// prompt: task.prompt
// .as_deref()
// .map(|s| s.to_string())
// .unwrap_or_else(||"".to_string()),
};
event.send(&app_handle)?;
if let Err(err) = event.send(&app_handle) {
error!("Failed to send GenerationFailedEvent: {:?}", err); // Fail open
}
}
}
@@ -126,12 +137,17 @@ async fn polling_loop(
let result = upload_image_media_file_from_file(&api_host, Some(&creds), download_path).await?;
info!("Uploaded to API backend: {:?}", result.media_file_token);
let event = SoraImageGenerationCompleteEvent {
media_file_token: result.media_file_token,
let event = GenerationCompleteEvent {
//media_file_token: result.media_file_token,
action: Some(GenerationAction::GenerateImage),
service: GenerationServiceProvider::Sora,
model: None,
};
event.send(&app_handle)?;
if let Err(err) = event.send(&app_handle) {
error!("Failed to send GenerationCompleteEvent: {:?}", err); // Fail open
}
}
}
+3 -2
View File
@@ -16,7 +16,8 @@
"width": 2400,
"height": 1300,
"resizable": true,
"fullscreen": false
"fullscreen": false,
"dragDropEnabled": false
}
],
"security": {
@@ -33,7 +34,7 @@
"https://www.googletagmanager.com",
"ipc://localhost",
"tauri://localhost"
],
],
"connect-src": [
"'self'",
"'unsafe-eval'",
@@ -14,4 +14,5 @@ pub fn add_artcraft_3d(cors: Cors, _is_production: bool) -> Cors {
.allowed_origin("http://localhost:5173")
.allowed_origin("http://localhost:5174") // If already started
.allowed_origin("http://localhost:5175") // If already started
.allowed_origin("https://macaroni-1.tailce84f.ts.net") // TODO
}
+7 -2
View File
@@ -17,17 +17,22 @@ path = "src/lib.rs"
workspace-build-acceleration = { workspace = true }
# Internal
file_to_base64_url = { path = "../../files/file_to_base64_url" }
file_to_base64_url = { workspace = true }
errors = { workspace = true }
# External
anyhow = { workspace = true }
fal = { version = "0.3.6", features = ["endpoints"] }
futures = { workspace = true }
log = { workspace = true }
reqwest = { workspace = true }
serde = { workspace = true }
tokio = { workspace = true }
url = { workspace = true }
# For now, we'll vendor in fal.
# We need to implement webhooks and the upstream doesn't support them yet.
#fal = { version = "0.3.6", features = ["endpoints"] }
fal = { path = "../../../vendor/fal-rs", features = ["endpoints"] }
[dev-dependencies]
testing = { path = "../../testing" }
@@ -15,6 +15,8 @@ pub enum FalErrorPlus {
UrlParseError(url::ParseError),
/// An endpoint we don't support yet.
UnhandledEndpoint(String),
/// Error from the `reqwest` crate.
ReqwestError(reqwest::Error),
}
impl Display for FalErrorPlus {
@@ -25,6 +27,7 @@ impl Display for FalErrorPlus {
Self::AnyhowError(err) => write!(f, "FalErrorPlus::AnyhowError: {:?}", err),
Self::UrlParseError(err) => write!(f, "FalErrorPlus::UrlParseError: {:?}", err),
Self::UnhandledEndpoint(endpoint) => write!(f, "FalErrorPlus::UnhandledEndpoint: {:?}", endpoint),
Self::ReqwestError(err) => write!(f, "FalErrorPlus::ReqwestError: {:?}", err),
}
}
}
@@ -48,3 +51,9 @@ impl From<url::ParseError> for FalErrorPlus {
FalErrorPlus::UrlParseError(err)
}
}
impl From<reqwest::Error> for FalErrorPlus {
fn from(err: reqwest::Error) -> Self {
FalErrorPlus::ReqwestError(err)
}
}
@@ -1,2 +0,0 @@
pub mod enqueue_flux_pro_ultra_text_to_image;
pub mod enqueue_recraft3_text_to_image;
@@ -1,4 +1,2 @@
pub mod enqueue_hunyuan2_image_to_3d;
pub mod remove_background_rembg;
pub mod video_gen;
pub mod image_gen;
pub mod queue;
pub mod webhook;
@@ -42,7 +42,7 @@ pub async fn enqueue_hunyuan2_image_to_3d<P: AsRef<Path>>(args: Hunyuan2Args<'_,
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use crate::requests::queue::enqueue_hunyuan2_image_to_3d::{enqueue_hunyuan2_image_to_3d, Hunyuan2Args};
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
@@ -7,22 +7,25 @@ use fal::endpoints::fal_ai::flux_pro::v1_1_ultra::FluxProUltraTextToImageInput;
use futures::StreamExt;
use std::io::Write;
pub struct FluxProUltraTextToImageArgs<'a> {
pub struct FluxPro11UltraTextToImageArgs<'a> {
pub prompt: &'a str,
pub api_key: &'a FalApiKey,
// TODO: Size parameter
}
pub async fn enqueue_flux_pro_ultra_text_to_image(args: FluxProUltraTextToImageArgs<'_>) -> Result<EnqueuedRequest, FalErrorPlus> {
pub async fn enqueue_flux_pro_11_ultra_text_to_image(args: FluxPro11UltraTextToImageArgs<'_>) -> Result<EnqueuedRequest, FalErrorPlus> {
let request = FluxProUltraTextToImageInput {
prompt: args.prompt.to_string(),
safety_tolerance: Some("5".to_string()), // 1 is most strict, 5 is most permissive
enable_safety_checker: Some(false),
num_images: Some(1), // Default is 1
output_format: Some("png".to_string()),
raw: Some(true), // Generate less processed, more natural-looking images. Default is false.
// Maybe expose
aspect_ratio: None, // Default is "16:9"
num_images: Some(1), // Default is 1
seed: None,
raw: Some(true), // Generate less processed, more natural-looking images. Default is false.
// Maybe abstract
enable_safety_checker: Some(false),
safety_tolerance: Some("5".to_string()), // 1 is most strict, 5 is most permissive
// Constants
output_format: Some("png".to_string()),
sync_mode: None, // Synchronous / slow
};
@@ -42,7 +45,7 @@ pub async fn enqueue_flux_pro_ultra_text_to_image(args: FluxProUltraTextToImageA
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::image_gen::enqueue_flux_pro_ultra_text_to_image::{enqueue_flux_pro_ultra_text_to_image, FluxProUltraTextToImageArgs};
use crate::requests::queue::image_gen::enqueue_flux_pro_11_ultra_text_to_image::{enqueue_flux_pro_11_ultra_text_to_image, FluxPro11UltraTextToImageArgs};
use errors::AnyhowResult;
use std::fs::read_to_string;
@@ -54,12 +57,12 @@ mod tests {
let api_key = FalApiKey::from_str(&secret);
let args = FluxProUltraTextToImageArgs {
let args = FluxPro11UltraTextToImageArgs {
prompt: "a corgi smiles at the camera, inside retro video game store, lots of sci-fi props from Cowboy Bebop, photorealistic",
api_key: &api_key,
};
let result = enqueue_flux_pro_ultra_text_to_image(args).await?;
let result = enqueue_flux_pro_11_ultra_text_to_image(args).await?;
Ok(())
}
@@ -39,7 +39,7 @@ pub async fn enqueue_recraft3_text_to_image(args: Recraft3TextToImageArgs<'_>) -
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use crate::requests::queue::image_gen::enqueue_recraft3_text_to_image::{enqueue_recraft3_text_to_image, Recraft3TextToImageArgs};
use errors::AnyhowResult;
use std::fs::read_to_string;
@@ -0,0 +1,2 @@
pub mod enqueue_flux_pro_11_ultra_text_to_image;
pub mod enqueue_recraft3_text_to_image;
@@ -0,0 +1,4 @@
pub mod enqueue_hunyuan2_image_to_3d;
pub mod image_gen;
pub mod remove_background_rembg;
pub mod video_gen;
@@ -9,11 +9,6 @@ use futures::StreamExt;
use std::path::Path;
use url::Url;
pub struct RemBgArgs<'a, P: AsRef<Path>> {
pub image_path: P,
pub api_key: &'a FalApiKey,
}
pub struct RemBgResponse {
pub image_url: Url,
}
@@ -27,7 +22,7 @@ pub async fn remove_background_rembg<P: AsRef<Path>>(image_path: P, api_key: &Fa
crop_to_bbox: None,
sync_mode: None
};
let result = rembg(request)
.with_api_key(&api_key.0)
.queue()
@@ -59,8 +54,9 @@ pub async fn remove_background_rembg<P: AsRef<Path>>(image_path: P, api_key: &Fa
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::remove_background_rembg::remove_background_rembg;
use crate::requests::queue::remove_background_rembg::remove_background_rembg;
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
#[tokio::test]
@@ -68,9 +64,9 @@ mod tests {
async fn test_remove_background_rembg() -> AnyhowResult<()> {
let image = test_file_path("test_data/image/juno.jpg")?;
// TODO: DO NOT COMMIT SECRET !
const KEY : &str = "DO NOT COMMIT";
let api_key = FalApiKey::from_str(KEY);
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let result = remove_background_rembg(image, &api_key).await?;
@@ -8,35 +8,50 @@ use futures::StreamExt;
use std::io::Write;
use std::path::Path;
pub struct Kling16Args<'a, P: AsRef<Path>> {
pub struct Kling16ProArgs<'a, P: AsRef<Path>> {
pub image_path: P,
pub prompt: &'a str,
pub api_key: &'a FalApiKey,
pub duration: Kling16Duration,
pub duration: Kling16ProDuration,
pub aspect_ratio: Kling16ProAspectRatio,
}
#[derive(Copy, Clone, Debug)]
pub enum Kling16Duration {
pub enum Kling16ProDuration {
Default,
FiveSeconds,
TenSeconds,
}
pub async fn enqueue_kling_16_image_to_video<P: AsRef<Path>>(args: Kling16Args<'_, P>) -> Result<EnqueuedRequest, FalErrorPlus> {
#[derive(Copy, Clone, Debug)]
pub enum Kling16ProAspectRatio {
Square, // 1:1
WideSixteenNine, // 16:9
TallNineSixteen, // 9:16
}
pub async fn enqueue_kling_16_pro_image_to_video<P: AsRef<Path>>(args: Kling16ProArgs<'_, P>) -> Result<EnqueuedRequest, FalErrorPlus> {
let image_url = file_to_base64_url(args.image_path)?;
let duration = match args.duration{
Kling16Duration::Default => None,
Kling16Duration::FiveSeconds => Some("5".to_string()), // Gross...
Kling16Duration::TenSeconds => Some("10".to_string()),
Kling16ProDuration::Default => None,
Kling16ProDuration::FiveSeconds => Some("5".to_string()), // Gross...
Kling16ProDuration::TenSeconds => Some("10".to_string()),
};
let aspect_ratio = match args.aspect_ratio {
Kling16ProAspectRatio::Square => Some("1:1".to_string()),
Kling16ProAspectRatio::WideSixteenNine => Some("16:9".to_string()),
Kling16ProAspectRatio::TallNineSixteen => Some("9:16".to_string()),
};
let request = ProImageToVideoRequest {
image_url,
prompt: args.prompt.to_string(),
aspect_ratio: None,
cfg_scale: None,
aspect_ratio,
duration,
// Maybe expose these later
cfg_scale: None,
negative_prompt: None,
tail_image_url: None,
};
@@ -57,7 +72,7 @@ pub async fn enqueue_kling_16_image_to_video<P: AsRef<Path>>(args: Kling16Args<'
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::video_gen::enqueue_kling_16_image_to_video::{enqueue_kling_16_image_to_video, Kling16Args, Kling16Duration};
use crate::requests::queue::video_gen::enqueue_kling_16_pro_image_to_video::{enqueue_kling_16_pro_image_to_video, Kling16ProArgs, Kling16ProAspectRatio, Kling16ProDuration};
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
@@ -72,14 +87,15 @@ mod tests {
let api_key = FalApiKey::from_str(&secret);
let args = Kling16Args {
let args = Kling16ProArgs {
image_path: image,
prompt: "a corgi looks out over the water",
api_key: &api_key,
duration: Kling16Duration::Default,
duration: Kling16ProDuration::Default,
aspect_ratio: Kling16ProAspectRatio::Square,
};
let result = enqueue_kling_16_image_to_video(args).await?;
let result = enqueue_kling_16_pro_image_to_video(args).await?;
Ok(())
}
@@ -3,7 +3,6 @@ use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use crate::model::enqueued_request::EnqueuedRequest;
use fal::endpoints::fal_ai::minimax::image_to_video::image_to_video;
use fal::endpoints::fal_ai::minimax::image_to_video::VideoOutput;
use fal::endpoints::fal_ai::minimax::image_to_video::ImageToVideoRequest;
use fal_client::file_to_base64_url::file_to_base64_url;
use futures::StreamExt;
@@ -41,10 +40,10 @@ pub async fn enqueue_minimax_01_image_to_video<P: AsRef<Path>>(args: Minimax01Ar
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::queue::video_gen::enqueue_minimax_01_image_to_video::{enqueue_minimax_01_image_to_video, Minimax01Args};
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
use crate::requests::video_gen::enqueue_minimax_01_image_to_video::{enqueue_minimax_01_image_to_video, Minimax01Args};
#[tokio::test]
#[ignore]
@@ -0,0 +1,2 @@
pub mod enqueue_kling_16_pro_image_to_video;
pub mod enqueue_minimax_01_image_to_video;
@@ -1,2 +0,0 @@
pub mod enqueue_kling_16_image_to_video;
pub mod enqueue_minimax_01_image_to_video;
@@ -0,0 +1,106 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::flux::dev::dev;
use fal::endpoints::fal_ai::flux::dev::DevTextToImageInput;
use fal::prelude::fal_ai::flux::dev::ImageSizeProperty;
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Flux1DevArgs<'a, U: IntoUrl> {
pub prompt: &'a str,
pub webhook_url: U,
pub api_key: &'a FalApiKey,
pub aspect_ratio: Flux1DevAspectRatio,
pub num_images: Flux1DevNumImages,
}
#[derive(Copy, Clone, Debug)]
pub enum Flux1DevAspectRatio {
Square, // 1:1
SquareHd, // 1:1
LandscapeFourByThree, // 4:3
LandscapeSixteenByNine, // 16:9
PortraitThreeByFour, // 3:4
PortraitNineBySixteen, // 9:16
//Custom { width: u32, height: u32 }, // TODO
}
#[derive(Copy, Clone, Debug)]
pub enum Flux1DevNumImages {
One, // Default
Two,
Three,
Four,
}
pub async fn enqueue_flux_1_dev_text_to_image_webhook<U: IntoUrl>(
args: Flux1DevArgs<'_, U>
) -> Result<WebhookResponse, FalErrorPlus> {
let num_images = match args.num_images {
Flux1DevNumImages::One => 1,
Flux1DevNumImages::Two => 2,
Flux1DevNumImages::Three => 3,
Flux1DevNumImages::Four => 4,
};
let image_size = match args.aspect_ratio {
Flux1DevAspectRatio::Square => ImageSizeProperty::Square,
Flux1DevAspectRatio::SquareHd => ImageSizeProperty::SquareHd,
Flux1DevAspectRatio::LandscapeFourByThree => ImageSizeProperty::Landscape43,
Flux1DevAspectRatio::LandscapeSixteenByNine => ImageSizeProperty::Landscape169,
Flux1DevAspectRatio::PortraitThreeByFour => ImageSizeProperty::Portrait43,
Flux1DevAspectRatio::PortraitNineBySixteen => ImageSizeProperty::Portrait169,
};
let request = DevTextToImageInput {
prompt: args.prompt.to_string(),
num_images: Some(num_images),
image_size: Some(image_size),
// Maybe abstract
enable_safety_checker: Some(false),
// Maybe expose
seed: None,
guidance_scale: None,
num_inference_steps: None,
// Constants
sync_mode: None, // Synchronous / slow
};
let result = dev(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::image::enqueue_flux_1_dev_text_to_image_webhook::{enqueue_flux_1_dev_text_to_image_webhook, Flux1DevArgs, Flux1DevAspectRatio, Flux1DevNumImages};
use errors::AnyhowResult;
use std::fs::read_to_string;
#[tokio::test]
#[ignore]
async fn test_flux_1_dev() -> AnyhowResult<()> {
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = Flux1DevArgs {
prompt: "a giant robot fighting a dragon in a futuristic city",
api_key: &api_key,
webhook_url: "https://example.com/webhook",
num_images: Flux1DevNumImages::One,
aspect_ratio: Flux1DevAspectRatio::LandscapeFourByThree
};
let result = enqueue_flux_1_dev_text_to_image_webhook(args).await?;
Ok(())
}
}
@@ -0,0 +1,105 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::flux::schnell::schnell;
use fal::endpoints::fal_ai::flux::schnell::SchnellTextToImageInput;
use fal::prelude::fal_ai::flux::schnell::ImageSizeProperty;
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Flux1SchnellArgs<'a, U: IntoUrl> {
pub prompt: &'a str,
pub webhook_url: U,
pub api_key: &'a FalApiKey,
pub aspect_ratio: Flux1SchnellAspectRatio,
pub num_images: Flux1SchnellNumImages,
}
#[derive(Copy, Clone, Debug)]
pub enum Flux1SchnellAspectRatio {
Square, // 1:1
SquareHd, // 1:1
LandscapeFourByThree, // 4:3
LandscapeSixteenByNine, // 16:9
PortraitThreeByFour, // 3:4
PortraitNineBySixteen, // 9:16
//Custom { width: u32, height: u32 }, // TODO
}
#[derive(Copy, Clone, Debug)]
pub enum Flux1SchnellNumImages {
One, // Default
Two,
Three,
Four,
}
pub async fn enqueue_flux_1_schnell_text_to_image_webhook<U: IntoUrl>(
args: Flux1SchnellArgs<'_, U>
) -> Result<WebhookResponse, FalErrorPlus> {
let num_images = match args.num_images {
Flux1SchnellNumImages::One => 1,
Flux1SchnellNumImages::Two => 2,
Flux1SchnellNumImages::Three => 3,
Flux1SchnellNumImages::Four => 4,
};
let image_size = match args.aspect_ratio {
Flux1SchnellAspectRatio::Square => ImageSizeProperty::Square,
Flux1SchnellAspectRatio::SquareHd => ImageSizeProperty::SquareHd,
Flux1SchnellAspectRatio::LandscapeFourByThree => ImageSizeProperty::Landscape43,
Flux1SchnellAspectRatio::LandscapeSixteenByNine => ImageSizeProperty::Landscape169,
Flux1SchnellAspectRatio::PortraitThreeByFour => ImageSizeProperty::Portrait43,
Flux1SchnellAspectRatio::PortraitNineBySixteen => ImageSizeProperty::Portrait169,
};
let request = SchnellTextToImageInput {
prompt: args.prompt.to_string(),
num_images: Some(num_images),
image_size: Some(image_size),
// Maybe abstract
enable_safety_checker: Some(false),
// Maybe expose
seed: None,
num_inference_steps: None,
// Constants
sync_mode: None, // Synchronous / slow
};
let result = schnell(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::image::enqueue_flux_1_schnell_text_to_image_webhook::{enqueue_flux_1_schnell_text_to_image_webhook, Flux1SchnellArgs, Flux1SchnellAspectRatio, Flux1SchnellNumImages};
use errors::AnyhowResult;
use std::fs::read_to_string;
#[tokio::test]
#[ignore]
async fn test_flux_1_schnell() -> AnyhowResult<()> {
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = Flux1SchnellArgs {
prompt: "a giant robot fighting a dragon in a futuristic city",
api_key: &api_key,
webhook_url: "https://example.com/webhook",
num_images: Flux1SchnellNumImages::One,
aspect_ratio: Flux1SchnellAspectRatio::LandscapeSixteenByNine,
};
let result = enqueue_flux_1_schnell_text_to_image_webhook(args).await?;
Ok(())
}
}
@@ -0,0 +1,104 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::flux_pro::v1_1::{v1_1, FluxProPlusTextToImageInput, ImageSizeProperty};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct FluxPro11Args<'a, U: IntoUrl> {
pub prompt: &'a str,
pub webhook_url: U,
pub api_key: &'a FalApiKey,
pub aspect_ratio: FluxPro11AspectRatio,
pub num_images: FluxPro11NumImages,
}
#[derive(Copy, Clone, Debug)]
pub enum FluxPro11AspectRatio {
Square, // 1:1
SquareHd, // 1:1
LandscapeFourByThree, // 4:3
LandscapeSixteenByNine, // 16:9
PortraitThreeByFour, // 3:4
PortraitNineBySixteen, // 9:16
//Custom { width: u32, height: u32 }, // TODO
}
#[derive(Copy, Clone, Debug)]
pub enum FluxPro11NumImages {
One, // Default
Two,
Three,
Four,
}
pub async fn enqueue_flux_pro_11_text_to_image_webhook<U: IntoUrl>(
args: FluxPro11Args<'_, U>
) -> Result<WebhookResponse, FalErrorPlus> {
let num_images = match args.num_images {
FluxPro11NumImages::One => 1,
FluxPro11NumImages::Two => 2,
FluxPro11NumImages::Three => 3,
FluxPro11NumImages::Four => 4,
};
let image_size = match args.aspect_ratio {
FluxPro11AspectRatio::Square => ImageSizeProperty::Square,
FluxPro11AspectRatio::SquareHd => ImageSizeProperty::SquareHd,
FluxPro11AspectRatio::LandscapeFourByThree => ImageSizeProperty::Landscape43,
FluxPro11AspectRatio::LandscapeSixteenByNine => ImageSizeProperty::Landscape169,
FluxPro11AspectRatio::PortraitThreeByFour => ImageSizeProperty::Portrait43,
FluxPro11AspectRatio::PortraitNineBySixteen => ImageSizeProperty::Portrait169,
};
let request = FluxProPlusTextToImageInput {
prompt: args.prompt.to_string(),
num_images: Some(num_images),
image_size: Some(image_size),
// Maybe expose
seed: None,
// Maybe abstract
enable_safety_checker: Some(false),
safety_tolerance: Some("5".to_string()), // 1 is most strict, 5 is most permissive
// Constants
output_format: Some("png".to_string()),
sync_mode: None, // Synchronous / slow
};
let result = v1_1(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::image::enqueue_flux_pro_11_text_to_image_webhook::{enqueue_flux_pro_11_text_to_image_webhook, FluxPro11Args, FluxPro11AspectRatio, FluxPro11NumImages};
use errors::AnyhowResult;
use std::fs::read_to_string;
#[tokio::test]
#[ignore]
async fn test_flux_11_pro() -> AnyhowResult<()> {
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = FluxPro11Args {
prompt: "a giant red panda fighting a dragon in a futuristic city",
api_key: &api_key,
webhook_url: "https://example.com/webhook",
num_images: FluxPro11NumImages::One,
aspect_ratio: FluxPro11AspectRatio::LandscapeSixteenByNine,
};
let result = enqueue_flux_pro_11_text_to_image_webhook(args).await?;
Ok(())
}
}
@@ -0,0 +1,39 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::flux_pro::v1_1_ultra::{v1_1_ultra, FluxProUltraTextToImageInput};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct FluxPro11UltraArgs<'a, U: IntoUrl> {
pub prompt: &'a str,
pub webhook_url: U,
pub api_key: &'a FalApiKey,
}
pub async fn enqueue_flux_pro_11_ultra_text_to_image_webhook<U: IntoUrl>(
args: FluxPro11UltraArgs<'_, U>
) -> Result<WebhookResponse, FalErrorPlus> {
let request = FluxProUltraTextToImageInput {
prompt: args.prompt.to_string(),
// Maybe expose
num_images: Some(1), // Default is 1
aspect_ratio: None, // Default is "16:9"
seed: None,
raw: Some(true), // Generate less processed, more natural-looking images. Default is false.
// Maybe abstract
enable_safety_checker: Some(false),
safety_tolerance: Some("5".to_string()), // 1 is most strict, 5 is most permissive
// Constants
output_format: Some("png".to_string()),
sync_mode: None, // Synchronous / slow
};
let result = v1_1_ultra(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
@@ -0,0 +1,5 @@
pub mod enqueue_flux_1_dev_text_to_image_webhook;
pub mod enqueue_flux_1_schnell_text_to_image_webhook;
pub mod enqueue_flux_pro_11_text_to_image_webhook;
pub mod enqueue_flux_pro_11_ultra_text_to_image_webhook;
pub mod remove_background_rembg_webhook;
@@ -0,0 +1,62 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::imageutils::rembg::{rembg, RemoveBackgroundInput};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct RemoveBackgroundRembgWebhookArgs<'a, U: IntoUrl, V: IntoUrl> {
pub image_url: U,
pub webhook_url: V,
pub api_key: &'a FalApiKey
}
pub async fn remove_background_rembg_webhook<U: IntoUrl, V: IntoUrl>(
args: RemoveBackgroundRembgWebhookArgs<'_, U, V>
) -> Result<WebhookResponse, FalErrorPlus> {
let image_url = args.image_url.into_url()?;
let request = RemoveBackgroundInput {
image_url: image_url.to_string(),
crop_to_bbox: None,
sync_mode: None
};
let result = rembg(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::image::remove_background_rembg_webhook::remove_background_rembg_webhook;
use crate::requests::webhook::image::remove_background_rembg_webhook::RemoveBackgroundRembgWebhookArgs;
use std::fs::read_to_string;
#[tokio::test]
#[ignore] // NB: Manually test, don't run in CI!
async fn test_rembg_request() -> anyhow::Result<()> {
// XXX: Don't commit secrets!
let api_key = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&api_key);
let args = RemoveBackgroundRembgWebhookArgs {
image_url: "https://cdn-2.fakeyou.com/media/b/2/p/b/0/b2pb09bcbxdqjzgbnsapvfs1t897wdnx/image_b2pb09bcbxdqjzgbnsapvfs1t897wdnx.png",
webhook_url: "https://api.storyteller.ai/webhook",
api_key: &api_key,
};
let response = remove_background_rembg_webhook(args).await?;
println!("{:?}", response);
assert_eq!(1, 2);
Ok(())
}
}
@@ -0,0 +1,3 @@
pub mod image;
pub mod object;
pub mod video;
@@ -0,0 +1,65 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::prelude::fal_ai::hunyuan3d::v21::{v21, Hunyuan3DInput};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Hunyuan3d21Args<'a, U: IntoUrl, V: IntoUrl> {
pub image_url: U,
pub webhook_url: V,
pub api_key: &'a FalApiKey,
}
pub async fn enqueue_hunyuan_3d_2_1_image_to_3d_webhook<U: IntoUrl, V: IntoUrl>(
args: Hunyuan3d21Args<'_, U, V>
) -> Result<WebhookResponse, FalErrorPlus> {
let image_url = args.image_url.as_str().to_string();
let request = Hunyuan3DInput {
input_image_url: image_url,
textured_mesh: Some(true),
// TODO: Maybe expose these later
guidance_scale: None,
num_inference_steps: None,
octree_resolution: None,
seed: None,
};
let result = v21(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::object::enqueue_hunyuan_3d_21_image_to_3d_webhook::{enqueue_hunyuan_3d_2_1_image_to_3d_webhook, Hunyuan3d21Args};
use errors::AnyhowResult;
use std::fs::read_to_string;
#[tokio::test]
#[ignore]
async fn test_hunyuan3d_21() -> AnyhowResult<()> {
let image_url = "https://cdn-2.fakeyou.com/media/p/a/c/7/j/pac7jgp2tkehm7j7sm4sky1fpnkrnbve/image_pac7jgp2tkehm7j7sm4sky1fpnkrnbve.png";
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = Hunyuan3d21Args {
image_url: image_url,
api_key: &api_key,
webhook_url: "https://example.com/webhook",
};
let result = enqueue_hunyuan_3d_2_1_image_to_3d_webhook(args).await?;
Ok(())
}
}
@@ -0,0 +1,35 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::hunyuan3d::v2::{v2, Hunyuan3DInput};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Hunyuan3d2Args<'a, U: IntoUrl, V: IntoUrl> {
pub image_url: U,
pub webhook_url: V,
pub api_key: &'a FalApiKey,
}
pub async fn enqueue_hunyuan_3d_2_image_to_3d_webhook<U: IntoUrl, V: IntoUrl>(
args: Hunyuan3d2Args<'_, U, V>
) -> Result<WebhookResponse, FalErrorPlus> {
let image_url = args.image_url.as_str().to_string();
let request = Hunyuan3DInput {
input_image_url: image_url,
textured_mesh: Some(true),
guidance_scale: None,
num_inference_steps: None,
octree_resolution: None,
seed: None,
};
let result = v2(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
@@ -0,0 +1,2 @@
pub mod enqueue_hunyuan_3d_2_image_to_3d_webhook;
pub mod enqueue_hunyuan_3d_21_image_to_3d_webhook;
@@ -0,0 +1,98 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::kling_video::v1_6::pro::image_to_video::{image_to_video, ProImageToVideoRequest};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Kling16ProArgs<'a, U: IntoUrl, V: IntoUrl> {
pub image_url: U,
pub webhook_url: V,
pub prompt: &'a str,
pub api_key: &'a FalApiKey,
pub duration: Kling16Duration,
pub aspect_ratio: Kling16ProAspectRatio,
}
#[derive(Copy, Clone, Debug)]
pub enum Kling16Duration {
Default,
FiveSeconds,
TenSeconds,
}
#[derive(Copy, Clone, Debug)]
pub enum Kling16ProAspectRatio {
Square, // 1:1
WideSixteenNine, // 16:9
TallNineSixteen, // 9:16
}
pub async fn enqueue_kling_16_pro_image_to_video_webhook<U: IntoUrl, V: IntoUrl>(
args: Kling16ProArgs<'_, U, V>
) -> Result<WebhookResponse, FalErrorPlus> {
let duration = match args.duration {
Kling16Duration::Default => None,
Kling16Duration::FiveSeconds => Some("5".to_string()), // Gross...
Kling16Duration::TenSeconds => Some("10".to_string()),
};
let aspect_ratio = match args.aspect_ratio {
Kling16ProAspectRatio::Square => Some("1:1".to_string()),
Kling16ProAspectRatio::WideSixteenNine => Some("16:9".to_string()),
Kling16ProAspectRatio::TallNineSixteen => Some("9:16".to_string()),
};
let image_url = args.image_url.as_str().to_string();
let request = ProImageToVideoRequest {
image_url,
prompt: args.prompt.to_string(),
aspect_ratio,
duration,
// Maybe expose these later
cfg_scale: None,
negative_prompt: None,
tail_image_url: None,
};
let result = image_to_video(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
/*
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::video::enqueue_kling_16_image_to_video_webhook::{enqueue_kling_16_image_to_video_webhook, Kling16Args, Kling16Duration};
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
#[tokio::test]
#[ignore]
async fn test_kling16_video() -> AnyhowResult<()> {
let image = test_file_path("test_data/image/juno.jpg")?;
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = Kling16Args {
image_path: image,
prompt: "a corgi looks out over the water",
api_key: &api_key,
duration: Kling16Duration::Default,
};
let result = enqueue_kling_16_image_to_video_webhook(args).await?;
Ok(())
}
}
*/
@@ -0,0 +1,98 @@
use crate::creds::fal_api_key::FalApiKey;
use crate::error::classify_fal_error::classify_fal_error;
use crate::error::fal_error_plus::FalErrorPlus;
use fal::endpoints::fal_ai::kling_video::v2_1::master::image_to_video::{image_to_video, MasterImageToVideoRequest};
use fal::webhook::WebhookResponse;
use reqwest::IntoUrl;
pub struct Kling21MasterArgs<'a, U: IntoUrl, V: IntoUrl> {
pub image_url: U,
pub webhook_url: V,
pub prompt: &'a str,
pub api_key: &'a FalApiKey,
pub duration: Kling21MasterDuration,
pub aspect_ratio: Kling21MasterAspectRatio,
}
#[derive(Copy, Clone, Debug)]
pub enum Kling21MasterDuration {
Default,
FiveSeconds,
TenSeconds,
}
#[derive(Copy, Clone, Debug)]
pub enum Kling21MasterAspectRatio {
Square, // 1:1
WideSixteenNine, // 16:9
TallNineSixteen, // 9:16
}
pub async fn enqueue_kling_21_master_image_to_video_webhook<U: IntoUrl, V: IntoUrl>(
args: Kling21MasterArgs<'_, U, V>
) -> Result<WebhookResponse, FalErrorPlus> {
let duration = match args.duration {
Kling21MasterDuration::Default => None,
Kling21MasterDuration::FiveSeconds => Some("5".to_string()), // Gross...
Kling21MasterDuration::TenSeconds => Some("10".to_string()),
};
let aspect_ratio = match args.aspect_ratio {
Kling21MasterAspectRatio::Square => Some("1:1".to_string()),
Kling21MasterAspectRatio::WideSixteenNine => Some("16:9".to_string()),
Kling21MasterAspectRatio::TallNineSixteen => Some("9:16".to_string()),
};
let image_url = args.image_url.as_str().to_string();
let request = MasterImageToVideoRequest {
image_url,
prompt: args.prompt.to_string(),
aspect_ratio,
duration,
// Maybe expose these later
cfg_scale: None,
negative_prompt: None,
tail_image_url: None,
};
let result = image_to_video(request)
.with_api_key(&args.api_key.0)
.queue_webhook(args.webhook_url)
.await;
result.map_err(|err| classify_fal_error(err))
}
#[cfg(test)]
mod tests {
use crate::creds::fal_api_key::FalApiKey;
use crate::requests::webhook::video::enqueue_kling_21_master_image_to_video_webhook::{enqueue_kling_21_master_image_to_video_webhook, Kling21MasterArgs, Kling21MasterAspectRatio, Kling21MasterDuration};
use errors::AnyhowResult;
use std::fs::read_to_string;
#[tokio::test]
#[ignore]
async fn test_kling21_master_video() -> AnyhowResult<()> {
let image_url = "https://cdn-2.fakeyou.com/media/3/4/h/f/s/34hfsmt8e38rvne6mwa4pwbxr6292sgy/image_34hfsmt8e38rvne6mwa4pwbxr6292sgy.png";
// XXX: Don't commit secrets!
let secret = read_to_string("/Users/bt/Artcraft/credentials/fal_api_key.txt")?;
let api_key = FalApiKey::from_str(&secret);
let args = Kling21MasterArgs {
image_url: image_url,
prompt: "a shot of the mountains, the camera rotates around to show the mountain range, the sun begins to set, and the moon becomes visible",
api_key: &api_key,
duration: Kling21MasterDuration::Default,
aspect_ratio: Kling21MasterAspectRatio::WideSixteenNine,
webhook_url: "https://example.com/webhook",
};
let result = enqueue_kling_21_master_image_to_video_webhook(args).await?;
Ok(())
}
}

Some files were not shown because too many files have changed in this diff Show More