Compare commits

...
Author SHA1 Message Date
bflatastic 3860f784a3 shared toaster component 2025-04-18 14:57:07 +07:00
BFlat e31a2c0a91 fix nx project discoverability (#527) 2025-04-18 14:10:40 +07:00
Kasisnu ea85abdc62 image-gen: make unused bools in task response optional 2025-04-18 10:34:52 +05:30
Brandon Thomas 4f3c93635f don't log debug level in prod 2025-04-18 00:07:51 -04:00
ArEnScandMichael Chung 82b9b90ff9 Added the apis to the api folder it loads correctly now (#525)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-17 22:49:27 -04:00
bflatastic ad5ea26a81 commented out test component it was pushing the ui down 2025-04-18 08:59:53 +07:00
bflatastic 6121d118f0 reduced topbar right side gap 2025-04-18 08:54:40 +07:00
bflatastic 28ca7be96c activity update 2025-04-18 08:50:55 +07:00
Brandon Thomas a2a7f7d274 could also be a bearer token issue 2025-04-17 21:39:45 -04:00
Brandon Thomas e1255a55a0 specific error message for cookie expiration 2025-04-17 21:30:16 -04:00
9f629ee5e2 Nx working test (#524)
It works now.

---------

Co-authored-by: Brandon Thomas <echelon@gmail.com>
Co-authored-by: Tanish Manku <tanish2k09.dev@gmail.com>
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-17 16:32:30 -04:00
BFlat 6a82fcb6e0 Revert "shared component starting with toaster" (#522)
Reverts storytold/storyteller-rust#521

it was causing an application error on build.
2025-04-17 17:21:05 +07:00
BFlat 35eac53b9d shared component starting with toaster (#521)
- share the toaster component between both editor2d and editor 3d
2025-04-17 17:14:02 +07:00
bflatastic c84c36d774 escape key to deselect objects and exit FK pose mode 2025-04-17 13:33:12 +07:00
bflatastic 3269b62e14 editor2d: artcraft logo 2025-04-17 13:11:55 +07:00
BFlat cd49fe0015 pose mode toggle button on top when character is selected (#520)
- pose mode toggle button to go in and out of pose mode when a character
is selected
- tooltip text size change
- button styling fixes
2025-04-17 13:04:53 +07:00
ArEnScandMichael Chung 326fa8c4a4 Adding API imports (#519)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-16 20:16:12 -04:00
Tanish Manku 51bdd4eff8 Create a really sick precision selector (#506)
Steps to activate:
1 - Select an object
2 - In the transform box, click and hold "Scale"
3 - Drag vertically to select the incremental precision
4 - Drag horizontally to apply increments of the selected precision

Issues:
- Releasing the mouse on top of the canvas will also trigger a "mouseup"
event on the canvas (since it listens to the window, not the canvas) and
deselect the object.
2025-04-16 15:01:44 -07:00
BFlat a64ca7d4ac editor3d: activity card improvements and share lightbox with gallery (#518)
**activity:**
- dropdown cards to show better download icon for completed items
- show spinner for in progress items
- activity button itself to show a spinner when at least one job is
running
- use the new lightbox component
- click the completed item to view image in lightbox

**gallery (library modal):**
- use the new lightbox component
- the new lightbox has a download button
- use cdn images
2025-04-17 02:48:25 +07:00
Kasisnu ec3ac242b0 image-gen: generate sentinel token in rust 2025-04-16 16:19:32 +05:30
Brandon Thomas fbe7fa3c77 don't leak logged in session jobs when logging out 2025-04-15 23:26:58 -04:00
Brandon Thomas 418d824c35 session jobs for anonymous users 2025-04-15 22:43:53 -04:00
Tanish Manku 2e6823834f Create the shapes centered on the canvas (#515)
- Also match all the shape sizes
2025-04-15 17:24:29 -07:00
ArEnScandMichael Chung 7ef20bc885 Feature/system prompt modification (#514)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-15 18:32:43 -04:00
Michael Chung b031a68000 Redo and Undo for osx fixed correctly.
Issues with the app not properly loading not allowing the engine reference to be set.
2025-04-15 16:35:47 -04:00
Brandon Thomas e4b648b55d bump Rust toolchains to 1.86.0 2025-04-15 15:05:06 -04:00
Brandon Thomas d78ccbca25 system prompt enhancement 2025-04-15 14:57:16 -04:00
Kasisnu bae6bc6863 move sentinel token generation into rust 2025-04-15 21:44:58 +05:30
Brandon Thomas faa43a7361 remove node-sass 2025-04-15 05:14:34 -04:00
Brandon Thomas f8b3b708c0 unblock CI builds for now 2025-04-15 05:05:21 -04:00
ArEnScandMichael Chung 4c40379470 Jobs should add the image to the list (#512)
Jobs also show completed images in the gallery.

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-15 02:49:45 -04:00
Brandon Thomas 0c32a89828 script to clean up node modules 2025-04-15 00:30:57 -04:00
Brandon Thomas d7365ea045 remove more entries from package-lock.json 2025-04-14 19:26:57 -04:00
Brandon Thomas 3386247858 remove pnpm-lock.yaml to unblock CI - it's causing new failures 2025-04-14 19:10:44 -04:00
Brandon Thomas 7ea9c96d77 remove dead references from editor3d package 2025-04-14 19:08:49 -04:00
Brandon Thomas 059f4b4e38 add pnpm to solve build issues 2025-04-14 18:56:05 -04:00
Brandon Thomas 3c36e041ca remove "workspace:*" URL. Not working on all machines? 2025-04-14 18:40:23 -04:00
Tanish MankuandTanish Manku f027fcb29a Write Gallery component with test data (#508)
Co-authored-by: Tanish Manku <-h>
2025-04-14 10:49:35 -07:00
BFlat 6f2ed9e61e dismiss onboarding helper and local storage 2025-04-14 14:57:33 +07:00
Brandon Thomas 2b40b2f8d6 add index on jobs anonymous visitor tokens 2025-04-14 02:58:49 -04:00
Brandon Thomas 9996b41bc2 Add nx library component and import it 2025-04-14 02:24:29 -04:00
ArEnScandMichael Chung b9a5768679 Feature/polling (#505)
add text back and toasters

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-13 23:54:20 -04:00
Brandon Thomas 614a5edc55 cleanup netlify.toml files 2025-04-13 21:17:09 -04:00
Brandon Thomas 01468020bd cleanup build script 2025-04-13 21:06:24 -04:00
Brandon Thomas 289ef6f352 debug branch build (#507) 2025-04-13 21:01:09 -04:00
Brandon Thomas 14aa429878 debug netlify 2025-04-13 13:39:53 -04:00
Brandon Thomas f902be970f debug netlify 2025-04-13 13:34:23 -04:00
Brandon Thomas 783761736e debug netlify 2025-04-13 13:29:06 -04:00
Brandon Thomas 794fcf4efb fix netlify build 2025-04-13 12:06:38 -04:00
bflatastic 183b480e6d thicker logo update 2025-04-13 00:18:40 +07:00
bflatastic ae215a7901 artcraft logo update 2025-04-13 00:12:36 +07:00
bflatastic 2117566fbd artcraft logo 2025-04-13 00:07:25 +07:00
bflatastic d492b93842 alt key to slow camera down for precise framing 2025-04-12 15:07:27 +07:00
bflatastic ebcc317f31 hide transform controls and edge glow in snapshot 2025-04-12 14:22:39 +07:00
BFlat 8a19e914dc camera switching, focal length etc. (#504)
all camera updates

- adding cameras
- switching cameras
- adjusting focal length for each
- save cameras to scene
2025-04-12 14:14:38 +07:00
bflatastic 5df413af49 memoized library modal to prevent constant rerender 2025-04-12 13:06:27 +07:00
186 changed files with 10385 additions and 4897 deletions
Generated
+31
View File
@@ -5035,6 +5035,12 @@ version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70"
[[package]]
name = "hex-literal"
version = "1.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bcaaec4551594c969335c98c903c1397853d4198408ea609190f420500f6be71"
[[package]]
name = "hf-hub"
version = "0.3.2"
@@ -6130,6 +6136,15 @@ dependencies = [
"signature",
]
[[package]]
name = "keccak"
version = "0.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ecc2af9a1119c51f12a14607e783cb977bde58bc069ff0c3da1095e635d70654"
dependencies = [
"cpufeatures",
]
[[package]]
name = "kernel32-sys"
version = "0.2.2"
@@ -7849,14 +7864,20 @@ name = "openai_sora_client"
version = "0.0.1"
dependencies = [
"anyhow",
"base64 0.22.1",
"chrono",
"crockford",
"errors",
"filesys",
"hex-literal",
"idempotency",
"log",
"rand 0.9.0",
"reqwest 0.12.15",
"serde",
"serde_derive",
"serde_json",
"sha3",
"subprocess_common",
"testing",
"thiserror 2.0.11",
@@ -10605,6 +10626,16 @@ dependencies = [
"digest 0.10.7",
]
[[package]]
name = "sha3"
version = "0.10.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "75872d278a8f37ef87fa0ddbda7802605cb18344497949862c0d4dcb291eba60"
dependencies = [
"digest 0.10.7",
"keccak",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
+3 -1
View File
@@ -152,7 +152,9 @@ workspace-build-acceleration = { path = "crates/workspace-build-acceleration" }
# Centrally pinned versions so that we can uniformly upgrade/downgrade versions across the workspace
# If a crate needs to use an older/newer version, the dependency should be managed in that crate's own Cargo.toml
anyhow = "1.0.97"
base64 = "0.22.1"
chrono = { version = "0.4.40", features = ["serde"] }
hex-literal = "1.0.0"
log = "0.4.27"
once_cell = "1.21.3"
rand = "0.9.0"
@@ -162,10 +164,10 @@ serde = { version = "1.0.219", features = ["derive"] }
serde_derive = "1.0.219"
serde_json = "1.0.140"
serde_yaml = "0.9.1"
sha3 = "0.10.8"
thiserror = "2"
toml = "0.7.4"
url = "2.5.4"
# Internal libs
crockford = { path = "crates/lib/data/crockford" }
errors = { path = "crates/lib/errors" }
@@ -0,0 +1,6 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
DROP INDEX `index_maybe_creator_anonymous_visitor_token`
ON `generic_inference_jobs`;
@@ -0,0 +1,7 @@
-- noinspection SqlDialectInspectionForFile
-- noinspection SqlNoDataSourceInspectionForFile
-- noinspection SqlResolveForFile
ALTER TABLE generic_inference_jobs
ADD INDEX index_maybe_creator_anonymous_visitor_token (maybe_creator_anonymous_visitor_token),
ALGORITHM=INPLACE, LOCK=NONE;
@@ -297,6 +297,7 @@ CREATE TABLE generic_inference_jobs (
KEY fk_maybe_input_source_token_and_type (maybe_input_source_token, maybe_input_source_token_type),
KEY fk_on_success_result_entity_token (on_success_result_entity_token),
KEY fk_maybe_creator_user_token (maybe_creator_user_token),
KEY index_maybe_creator_anonymous_visitor_token (maybe_creator_anonymous_visitor_token),
KEY index_creator_ip_address (creator_ip_address),
KEY index_priority_level (priority_level),
KEY index_is_debug_request (is_debug_request),
@@ -10,7 +10,7 @@ pub fn encode_rgb_image_base64_png(
image.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)?;
let bytes = BASE64_STANDARD.encode(bytes);
let bytes = (&BASE64_STANDARD).encode(bytes);
Ok(bytes)
}
@@ -20,6 +20,7 @@ workspace-build-acceleration = { workspace = true }
crockford = { path = "../../data/crockford" }
errors = { path = "../../errors" }
subprocess_common = { path = "../../subprocess_common" }
idempotency = { path = "../../idempotency" }
# External
anyhow = { workspace = true }
@@ -28,9 +29,14 @@ reqwest = { workspace = true, features = ["multipart"] }
serde = { workspace = true }
serde_derive = { workspace = true }
serde_json = { workspace = true }
sha3 = { workspace = true }
hex-literal = { workspace = true }
base64 = { workspace = true }
tokio = { version = "1.38.0", features = ["macros"] }
toml = { workspace = true }
thiserror = { workspace = true }
chrono = { workspace = true}
rand = { workspace = true }
[dev-dependencies]
testing = { path = "../../testing" }
@@ -15,6 +15,9 @@ pub enum SoraError {
#[error("Sora sentinel block: {0}")]
SentinelBlock(String),
#[error("Sora too many concurrent tasks: {0}")]
TooManyConcurrentTasks(String),
#[error("Sora API error: {0}")]
GenericError(String),
@@ -115,10 +118,10 @@ pub enum RawSoraError {
#[derive(Deserialize, Debug, Clone)]
#[serde(rename_all = "snake_case")]
pub enum SoraErrorCode {
pub enum SoraErrorCode {
SentinelBlock,
TokenExpired,
TooManyConcurrentTasks,
Unknown(String),
}
@@ -179,6 +182,9 @@ pub (crate) async fn image_gen_http_request(sora_request: RawSoraImageGenRequest
SoraErrorCode::SentinelBlock => {
return Err(SoraError::SentinelBlock(error_response.error.message));
}
SoraErrorCode::TooManyConcurrentTasks => {
return Err(SoraError::TooManyConcurrentTasks(error_response.error.message));
}
SoraErrorCode::Unknown(_) => {
return Err(SoraError::GenericError(error_response.error.message));
}
@@ -83,16 +83,16 @@ pub struct Generation {
pub created_at: String,
pub deleted_at: Option<String>,
pub url: String,
pub seed: i64,
pub can_download: bool,
pub download_status: String,
pub is_favorite: bool,
pub is_liked: bool,
pub is_public: bool,
pub is_archived: bool,
pub is_featured: bool,
pub has_feedback: bool,
pub like_count: i32,
pub seed: Option<i64>,
pub can_download: Option<bool>,
pub download_status: Option<String>,
pub is_favorite: Option<bool>,
pub is_liked: Option<bool>,
pub is_public: Option<bool>,
pub is_archived: Option<bool>,
pub is_featured: Option<bool>,
pub has_feedback: Option<bool>,
pub like_count: Option<i32>,
pub cloudflare_metadata: Option<serde_json::Value>,
pub cf_thumbnail_url: Option<String>,
pub encodings: Encodings,
@@ -2,4 +2,5 @@ pub mod credentials;
pub mod image_gen;
pub mod job;
pub mod upload;
pub mod sentinel_refresh;
pub mod sentinel_refresh;
mod sora_error;
@@ -0,0 +1,671 @@
// cores = [8, 16, 24, 32]
// cached_scripts = ["https://cdn.oaistatic.com/_next/static/cXh69klOLzS0Gy2joLDRS/_ssgManifest.js?dpl=453ebaec0d44c2decab71692e1bfe39be35a24b3"]
// cached_dpl = ["prod-f501fe933b3edf57aea882da888e1a544df99840"]
// navigator_key = [
// "registerProtocolHandler−function registerProtocolHandler() { [native code] }",
// "storage−[object StorageManager]",
// "locks−[object LockManager]",
// "appCodeName−Mozilla",
// "permissions−[object Permissions]",
// "share−function share() { [native code] }",
// "webdriver−false",
// "managed−[object NavigatorManagedData]",
// "canShare−function canShare() { [native code] }",
// "vendor−Google Inc.",
// "vendor−Google Inc.",
// "mediaDevices−[object MediaDevices]",
// "vibrate−function vibrate() { [native code] }",
// "storageBuckets−[object StorageBucketManager]",
// "mediaCapabilities−[object MediaCapabilities]",
// "getGamepads−function getGamepads() { [native code] }",
// "bluetooth−[object Bluetooth]",
// "share−function share() { [native code] }",
// "cookieEnabled−true",
// "virtualKeyboard−[object VirtualKeyboard]",
// "product−Gecko",
// "mediaDevices−[object MediaDevices]",
// "canShare−function canShare() { [native code] }",
// "getGamepads−function getGamepads() { [native code] }",
// "product−Gecko",
// "xr−[object XRSystem]",
// "clipboard−[object Clipboard]",
// "storageBuckets−[object StorageBucketManager]",
// "unregisterProtocolHandler−function unregisterProtocolHandler() { [native code] }",
// "productSub−20030107",
// "login−[object NavigatorLogin]",
// "vendorSub−",
// "login−[object NavigatorLogin]",
// "getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
// "mediaDevices−[object MediaDevices]",
// "locks−[object LockManager]",
// "webkitGetUserMedia−function webkitGetUserMedia() { [native code] }",
// "vendor−Google Inc.",
// "xr−[object XRSystem]",
// "mediaDevices−[object MediaDevices]",
// "virtualKeyboard−[object VirtualKeyboard]",
// "virtualKeyboard−[object VirtualKeyboard]",
// "appName−Netscape",
// "storageBuckets−[object StorageBucketManager]",
// "presentation−[object Presentation]",
// "onLine−true",
// "mimeTypes−[object MimeTypeArray]",
// "credentials−[object CredentialsContainer]",
// "presentation−[object Presentation]",
// "getGamepads−function getGamepads() { [native code] }",
// "vendorSub−",
// "virtualKeyboard−[object VirtualKeyboard]",
// "serviceWorker−[object ServiceWorkerContainer]",
// "xr−[object XRSystem]",
// "product−Gecko",
// "keyboard−[object Keyboard]",
// "gpu−[object GPU]",
// "getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
// "webkitPersistentStorage−[object DeprecatedStorageQuota]",
// "doNotTrack",
// "clearAppBadge−function clearAppBadge() { [native code] }",
// "presentation−[object Presentation]",
// "serial−[object Serial]",
// "locks−[object LockManager]",
// "requestMIDIAccess−function requestMIDIAccess() { [native code] }",
// "locks−[object LockManager]",
// "requestMediaKeySystemAccess−function requestMediaKeySystemAccess() { [native code] }",
// "vendor−Google Inc.",
// "pdfViewerEnabled−true",
// "language−zh-CN",
// "setAppBadge−function setAppBadge() { [native code] }",
// "geolocation−[object Geolocation]",
// "userAgentData−[object NavigatorUAData]",
// "mediaCapabilities−[object MediaCapabilities]",
// "requestMIDIAccess−function requestMIDIAccess() { [native code] }",
// "getUserMedia−function getUserMedia() { [native code] }",
// "mediaDevices−[object MediaDevices]",
// "webkitPersistentStorage−[object DeprecatedStorageQuota]",
// "sendBeacon−function sendBeacon() { [native code] }",
// "hardwareConcurrency−32",
// "credentials−[object CredentialsContainer]",
// "storage−[object StorageManager]",
// "cookieEnabled−true",
// "pdfViewerEnabled−true",
// "windowControlsOverlay−[object WindowControlsOverlay]",
// "scheduling−[object Scheduling]",
// "pdfViewerEnabled−true",
// "hardwareConcurrency−32",
// "xr−[object XRSystem]",
// "webdriver−false",
// "getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
// "getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
// "bluetooth−[object Bluetooth]"
// ]
// document_key = ['_reactListeningo743lnnpvdg', 'location']
// window_key = [
// "0",
// "window",
// "self",
// "document",
// "name",
// "location",
// "customElements",
// "history",
// "navigation",
// "locationbar",
// "menubar",
// "personalbar",
// "scrollbars",
// "statusbar",
// "toolbar",
// "status",
// "closed",
// "frames",
// "length",
// "top",
// "opener",
// "parent",
// "frameElement",
// "navigator",
// "origin",
// "external",
// "screen",
// "innerWidth",
// "innerHeight",
// "scrollX",
// "pageXOffset",
// "scrollY",
// "pageYOffset",
// "visualViewport",
// "screenX",
// "screenY",
// "outerWidth",
// "outerHeight",
// "devicePixelRatio",
// "clientInformation",
// "screenLeft",
// "screenTop",
// "styleMedia",
// "onsearch",
// "isSecureContext",
// "trustedTypes",
// "performance",
// "onappinstalled",
// "onbeforeinstallprompt",
// "crypto",
// "indexedDB",
// "sessionStorage",
// "localStorage",
// "onbeforexrselect",
// "onabort",
// "onbeforeinput",
// "onbeforematch",
// "onbeforetoggle",
// "onblur",
// "oncancel",
// "oncanplay",
// "oncanplaythrough",
// "onchange",
// "onclick",
// "onclose",
// "oncontentvisibilityautostatechange",
// "oncontextlost",
// "oncontextmenu",
// "oncontextrestored",
// "oncuechange",
// "ondblclick",
// "ondrag",
// "ondragend",
// "ondragenter",
// "ondragleave",
// "ondragover",
// "ondragstart",
// "ondrop",
// "ondurationchange",
// "onemptied",
// "onended",
// "onerror",
// "onfocus",
// "onformdata",
// "oninput",
// "oninvalid",
// "onkeydown",
// "onkeypress",
// "onkeyup",
// "onload",
// "onloadeddata",
// "onloadedmetadata",
// "onloadstart",
// "onmousedown",
// "onmouseenter",
// "onmouseleave",
// "onmousemove",
// "onmouseout",
// "onmouseover",
// "onmouseup",
// "onmousewheel",
// "onpause",
// "onplay",
// "onplaying",
// "onprogress",
// "onratechange",
// "onreset",
// "onresize",
// "onscroll",
// "onsecuritypolicyviolation",
// "onseeked",
// "onseeking",
// "onselect",
// "onslotchange",
// "onstalled",
// "onsubmit",
// "onsuspend",
// "ontimeupdate",
// "ontoggle",
// "onvolumechange",
// "onwaiting",
// "onwebkitanimationend",
// "onwebkitanimationiteration",
// "onwebkitanimationstart",
// "onwebkittransitionend",
// "onwheel",
// "onauxclick",
// "ongotpointercapture",
// "onlostpointercapture",
// "onpointerdown",
// "onpointermove",
// "onpointerrawupdate",
// "onpointerup",
// "onpointercancel",
// "onpointerover",
// "onpointerout",
// "onpointerenter",
// "onpointerleave",
// "onselectstart",
// "onselectionchange",
// "onanimationend",
// "onanimationiteration",
// "onanimationstart",
// "ontransitionrun",
// "ontransitionstart",
// "ontransitionend",
// "ontransitioncancel",
// "onafterprint",
// "onbeforeprint",
// "onbeforeunload",
// "onhashchange",
// "onlanguagechange",
// "onmessage",
// "onmessageerror",
// "onoffline",
// "ononline",
// "onpagehide",
// "onpageshow",
// "onpopstate",
// "onrejectionhandled",
// "onstorage",
// "onunhandledrejection",
// "onunload",
// "crossOriginIsolated",
// "scheduler",
// "alert",
// "atob",
// "blur",
// "btoa",
// "cancelAnimationFrame",
// "cancelIdleCallback",
// "captureEvents",
// "clearInterval",
// "clearTimeout",
// "close",
// "confirm",
// "createImageBitmap",
// "fetch",
// "find",
// "focus",
// "getComputedStyle",
// "getSelection",
// "matchMedia",
// "moveBy",
// "moveTo",
// "open",
// "postMessage",
// "print",
// "prompt",
// "queueMicrotask",
// "releaseEvents",
// "reportError",
// "requestAnimationFrame",
// "requestIdleCallback",
// "resizeBy",
// "resizeTo",
// "scroll",
// "scrollBy",
// "scrollTo",
// "setInterval",
// "setTimeout",
// "stop",
// "structuredClone",
// "webkitCancelAnimationFrame",
// "webkitRequestAnimationFrame",
// "chrome",
// "caches",
// "cookieStore",
// "ondevicemotion",
// "ondeviceorientation",
// "ondeviceorientationabsolute",
// "launchQueue",
// "documentPictureInPicture",
// "getScreenDetails",
// "queryLocalFonts",
// "showDirectoryPicker",
// "showOpenFilePicker",
// "showSaveFilePicker",
// "originAgentCluster",
// "onpageswap",
// "onpagereveal",
// "credentialless",
// "speechSynthesis",
// "onscrollend",
// "webkitRequestFileSystem",
// "webkitResolveLocalFileSystemURL",
// "sendMsgToSolverCS",
// "webpackChunk_N_E",
// "__next_set_public_path__",
// "next",
// "__NEXT_DATA__",
// "__SSG_MANIFEST_CB",
// "__NEXT_P",
// "_N_E",
// "regeneratorRuntime",
// "__REACT_INTL_CONTEXT__",
// "DD_RUM",
// "_",
// "filterCSS",
// "filterXSS",
// "__SEGMENT_INSPECTOR__",
// "__NEXT_PRELOADREADY",
// "Intercom",
// "__MIDDLEWARE_MATCHERS",
// "__STATSIG_SDK__",
// "__STATSIG_JS_SDK__",
// "__STATSIG_RERENDER_OVERRIDE__",
// "_oaiHandleSessionExpired",
// "__BUILD_MANIFEST",
// "__SSG_MANIFEST",
// "__intercomAssignLocation",
// "__intercomReloadLocation"
// ]
use rand::Rng;
const CORES: &[usize] = &[8, 16, 24, 32];
const CACHED_SCRIPTS: &[&str] = &["https://cdn.oaistatic.com/_next/static/cXh69klOLzS0Gy2joLDRS/_ssgManifest.js?dpl=453ebaec0d44c2decab71692e1bfe39be35a24b3"];
const CACHED_DPL: &[&str] = &["prod-f501fe933b3edf57aea882da888e1a544df99840"];
const NAVIGATOR_KEY: &[&str] = &[
"registerProtocolHandler−function registerProtocolHandler() { [native code] }",
"storage−[object StorageManager]",
"locks−[object LockManager]",
"appCodeName−Mozilla",
"permissions−[object Permissions]",
"share−function share() { [native code] }",
"webdriver−false",
"managed−[object NavigatorManagedData]",
"canShare−function canShare() { [native code] }",
"vendor−Google Inc.",
"vendor−Google Inc.",
"mediaDevices−[object MediaDevices]",
"vibrate−function vibrate() { [native code] }",
"storageBuckets−[object StorageBucketManager]",
"mediaCapabilities−[object MediaCapabilities]",
"getGamepads−function getGamepads() { [native code] }",
"bluetooth−[object Bluetooth]",
"share−function share() { [native code] }",
"cookieEnabled−true",
"pdfViewerEnabled−true",
"windowControlsOverlay−[object WindowControlsOverlay]",
"scheduling−[object Scheduling]",
"pdfViewerEnabled−true",
"hardwareConcurrency−32",
"xr−[object XRSystem]",
"webdriver−false",
"getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
"getInstalledRelatedApps−function getInstalledRelatedApps() { [native code] }",
"bluetooth−[object Bluetooth]"
];
const DOCUMENT_KEY: &[&str] = &["_reactListeningo743lnnpvdg", "location"];
const WINDOW_KEY: &[&str] = &[
"0",
"window",
"self",
"document",
"name",
"location",
"customElements",
"history",
"navigation",
"locationbar",
"menubar",
"personalbar",
"scrollbars",
"statusbar",
"toolbar",
"status",
"closed",
"frames",
"length",
"top",
"opener",
"parent",
"frameElement",
"navigator",
"origin",
"external",
"screen",
"innerWidth",
"innerHeight",
"scrollX",
"pageXOffset",
"scrollY",
"pageYOffset",
"visualViewport",
"screenX",
"screenY",
"outerWidth",
"outerHeight",
"devicePixelRatio",
"clientInformation",
"screenLeft",
"screenTop",
"styleMedia",
"onsearch",
"isSecureContext",
"trustedTypes",
"performance",
"onappinstalled",
"onbeforeinstallprompt",
"crypto",
"indexedDB",
"sessionStorage",
"localStorage",
"onbeforexrselect",
"onabort",
"onbeforeinput",
"onbeforematch",
"onbeforetoggle",
"onblur",
"oncancel",
"oncanplay",
"oncanplaythrough",
"onchange",
"onclick",
"onclose",
"oncontentvisibilityautostatechange",
"oncontextlost",
"oncontextmenu",
"oncontextrestored",
"oncuechange",
"ondblclick",
"ondrag",
"ondragend",
"ondragenter",
"ondragleave",
"ondragover",
"ondragstart",
"ondrop",
"ondurationchange",
"onemptied",
"onended",
"onerror",
"onfocus",
"onformdata",
"oninput",
"oninvalid",
"onkeydown",
"onkeypress",
"onkeyup",
"onload",
"onloadeddata",
"onloadedmetadata",
"onloadstart",
"onmousedown",
"onmouseenter",
"onmouseleave",
"onmousemove",
"onmouseout",
"onmouseover",
"onmouseup",
"onmousewheel",
"onpause",
"onplay",
"onplaying",
"onprogress",
"onratechange",
"onreset",
"onresize",
"onscroll",
"onsecuritypolicyviolation",
"onseeked",
"onseeking",
"onselect",
"onslotchange",
"onstalled",
"onsubmit",
"onsuspend",
"ontimeupdate",
"ontoggle",
"onvolumechange",
"onwaiting",
"onwebkitanimationend",
"onwebkitanimationiteration",
"onwebkitanimationstart",
"onwebkittransitionend",
"onwheel",
"onauxclick",
"ongotpointercapture",
"onlostpointercapture",
"onpointerdown",
"onpointermove",
"onpointerrawupdate",
"onpointerup",
"onpointercancel",
"onpointerover",
"onpointerout",
"onpointerenter",
"onpointerleave",
"onselectstart",
"onselectionchange",
"onanimationend",
"onanimationiteration",
"onanimationstart",
"ontransitionrun",
"ontransitionstart",
"ontransitionend",
"ontransitioncancel",
"onafterprint",
"onbeforeprint",
"onbeforeunload",
"onhashchange",
"onlanguagechange",
"onmessage",
"onmessageerror",
"onoffline",
"ononline",
"onpagehide",
"onpageshow",
"onpopstate",
"onrejectionhandled",
"onstorage",
"onunhandledrejection",
"onunload",
"crossOriginIsolated",
"scheduler",
"alert",
"atob",
"blur",
"btoa",
"cancelAnimationFrame",
"cancelIdleCallback",
"captureEvents",
"clearInterval",
"clearTimeout",
"close",
"confirm",
"createImageBitmap",
"fetch",
"find",
"focus",
"getComputedStyle",
"getSelection",
"matchMedia",
"moveBy",
"moveTo",
"open",
"postMessage",
"print",
"prompt",
"queueMicrotask",
"releaseEvents",
"reportError",
"requestAnimationFrame",
"requestIdleCallback",
"resizeBy",
"resizeTo",
"scroll",
"scrollBy",
"scrollTo",
"setInterval",
"setTimeout",
"stop",
"structuredClone",
"webkitCancelAnimationFrame",
"webkitRequestAnimationFrame",
"chrome",
"caches",
"cookieStore",
"ondevicemotion",
"ondeviceorientation",
"ondeviceorientationabsolute",
"launchQueue",
"documentPictureInPicture",
"getScreenDetails",
"queryLocalFonts",
"showDirectoryPicker",
"showOpenFilePicker",
"showSaveFilePicker",
"originAgentCluster",
"onpageswap",
"onpagereveal",
"credentialless",
"speechSynthesis",
"onscrollend",
"webkitRequestFileSystem",
"webkitResolveLocalFileSystemURL",
"sendMsgToSolverCS",
"webpackChunk_N_E",
"__next_set_public_path__",
"next",
"__NEXT_DATA__",
"__SSG_MANIFEST_CB",
"__NEXT_P",
"_N_E",
"regeneratorRuntime",
"__REACT_INTL_CONTEXT__",
"DD_RUM",
"_",
"filterCSS",
"filterXSS",
"__SEGMENT_INSPECTOR__",
"__NEXT_PRELOADREADY",
"Intercom",
"__MIDDLEWARE_MATCHERS",
"__STATSIG_SDK__",
"__STATSIG_JS_SDK__",
"__STATSIG_RERENDER_OVERRIDE__",
"_oaiHandleSessionExpired",
"__BUILD_MANIFEST",
"__SSG_MANIFEST",
"__intercomAssignLocation",
"__intercomReloadLocation"
];
pub fn get_random_core_count() -> usize {
return CORES[rand::rng().random_range(0..CORES.len())];
}
pub fn get_random_navigator_key() -> String {
return NAVIGATOR_KEY[rand::rng().random_range(0..NAVIGATOR_KEY.len())].to_string();
}
pub fn get_random_document_key() -> String {
return DOCUMENT_KEY[rand::rng().random_range(0..DOCUMENT_KEY.len())].to_string();
}
pub fn get_random_window_key() -> String {
return WINDOW_KEY[rand::rng().random_range(0..WINDOW_KEY.len())].to_string();
}
@@ -0,0 +1,3 @@
pub mod request;
pub mod token;
pub(crate) mod config;
@@ -0,0 +1,225 @@
use crate::sentinel_refresh::generate::config::{get_random_core_count, get_random_document_key, get_random_navigator_key, get_random_window_key};
use chrono::FixedOffset;
use chrono::Utc;
use idempotency::uuid::generate_random_uuid;
use rand::Rng;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use sha3::{Digest, Sha3_512};
// [
// 4000, // first
// "Fri Apr 11 2025 13:28:14 GMT+0530 (India Standard Time)", // second
// None, // third
// 28, // fourth
// "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:137.0) Gecko/20100101 Firefox/137.0", // fifth
// "https://sora-cdn.oaistatic.com/_next/static/chunks/f90f2e60-61da6d33257661c4.js", // sixth
// None, // seventh
// "en-US", // eighth
// "en-US,en", // ninth
// 29, // tenth
// "productSub−20100101", // eleventh
// "location", // twelfth
// "ondrop", // thirteenth
// 10520, // fourteenth - generated from time.perf_counter() * 1000
// "48094014-a024-4d2d-8ec4-ba9c7cfef9c4", // fifteenth
// "", // sixteenth
// 16, // seventeenth - Number of cores
#[derive(Debug, Clone)]
pub struct GenerateSentinelRefreshRequest {
first: usize,
second: String, // Date
third: Option<String>,
fourth: Option<usize>,
fifth: String, // User agent
sixth: String, // URL
seventh: Option<String>, // Language
eighth: String, // Accept-Language
ninth: String, // Product Sub
tenth: Option<usize>,
eleventh: String, // Product Sub - Navigator key
twelfth: String, // Location - Document key
thirteenth: String, // On Drop - Window key
fourteenth: usize, // generated from time.perf_counter() * 1000
fifteenth: String, // UUID
sixteenth: String, // empty string
seventeenth: usize, // Number of cores
}
fn generate_date() -> String {
let now_utc = Utc::now();
let offset = FixedOffset::west_opt(5 * 60 * 60).unwrap();
let now_with_offset = Utc::now().with_timezone(&offset);
let formatted_date = now_with_offset.format("%a %b %d %Y %H:%M:%S GMT-0500 (Eastern Standard Time)");
return formatted_date.to_string();
}
impl GenerateSentinelRefreshRequest {
pub fn new() -> Self {
let first_candidates = vec![1920 + 1080, 2560 + 1440, 1920 + 1200, 2560 + 1600];
let first = first_candidates[rand::rng().random_range(0..first_candidates.len())];
let second = generate_date();
let third = None;
let fourth = None;
let fifth = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:137.0) Gecko/20100101 Firefox/137.0".to_string();
let sixth = "https://sora-cdn.oaistatic.com/_next/static/chunks/f90f2e60-61da6d33257661c4.js".to_string();
let seventh = None;
let eighth = "en-US".to_string();
let ninth = "en-US,en".to_string();
let tenth = None;
let eleventh = get_random_navigator_key();
let twelfth = get_random_document_key();
let thirteenth = get_random_window_key();
let fourteenth = 10520;
let fifteenth = generate_random_uuid();
let sixteenth = "".to_string();
let seventeenth = get_random_core_count();
Self { first, second, third, fourth, fifth, sixth, seventh, eighth, ninth, tenth, eleventh: eleventh.to_string(), twelfth: twelfth.to_string(), thirteenth: thirteenth.to_string(), fourteenth, fifteenth, sixteenth, seventeenth }
}
pub fn with_fourth(mut self, fourth: usize) -> Self {
self.fourth = Some(fourth);
self
}
pub fn with_tenth(mut self, tenth: usize) -> Self {
self.tenth = Some(tenth);
self
}
//TODO(kasisnu): This is a temporary function to build the request.
// It is extremely hacky and this whole struct should be rewritten to be more ergonomic.
pub fn build(self) -> String {
let mut request = String::new();
request.push_str("[");
request.push_str(&self.first.to_string());
request.push_str(",");
request.push_str(&string_with_quotes(self.second));
request.push_str(",");
request.push_str(&self.third.unwrap_or("null".to_string()));
request.push_str(",");
request.push_str(&self.fourth.unwrap_or(0).to_string());
request.push_str(",");
request.push_str(&string_with_quotes(self.fifth));
request.push_str(",");
request.push_str(&string_with_quotes(self.sixth));
request.push_str(",");
request.push_str(&self.seventh.unwrap_or("null".to_string()));
request.push_str(",");
request.push_str(&string_with_quotes(self.eighth));
request.push_str(",");
request.push_str(&string_with_quotes(self.ninth));
request.push_str(",");
request.push_str(&self.tenth.unwrap_or(0).to_string());
request.push_str(",");
request.push_str(&string_with_quotes(self.eleventh));
request.push_str(",");
request.push_str(&string_with_quotes(self.twelfth));
request.push_str(",");
request.push_str(&string_with_quotes(self.thirteenth));
request.push_str(",");
request.push_str(&self.fourteenth.to_string());
request.push_str(",");
request.push_str(&string_with_quotes(self.fifteenth));
request.push_str(",");
request.push_str(&string_with_quotes(self.sixteenth));
request.push_str(",");
request.push_str(&self.seventeenth.to_string());
request.push_str("]");
request
}
pub fn hash_request(seed: f64, request: String) -> Vec<u8> {
let mut hasher = Sha3_512::new();
hasher.update(seed.to_string());
let base64_request = BASE64_STANDARD.encode(request.as_bytes());
hasher.update(base64_request);
hasher.finalize().to_vec()
}
pub fn with_fourth_and_tenth(mut self) -> (Self, String) {
let mut fourth = 0;
let mut tenth = 0;
let max_iterations = 500000;
let diff_len = 6;
let target_bytes = vec![0x0f, 0xff, 0xff];
let random_seed: f64 = rand::rng().random();
let mut base64_request = BASE64_STANDARD.encode(self.clone().build().as_bytes());
for i in 0..max_iterations {
fourth = i;
tenth = i >> 1;
let request = self.clone().with_fourth(fourth).with_tenth(tenth).build();
let hash_bytes = Self::hash_request(random_seed, request.clone());
base64_request = BASE64_STANDARD.encode(request.as_bytes());
let mut is_less_or_equal = true;
for j in 0..std::cmp::min(diff_len, target_bytes.len()) {
if j < hash_bytes.len() {
if hash_bytes[j] > target_bytes[j] {
is_less_or_equal = false;
break;
} else if hash_bytes[j] < target_bytes[j] {
break;
}
}
}
if is_less_or_equal {
break;
}
}
(self.with_fourth(fourth).with_tenth(tenth), format!("gAAAAAC{}", base64_request))
}
}
fn string_with_quotes(value: String) -> String {
format!("\"{}\"", value)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn test_generate_sentinel_refresh_request() {
let request = GenerateSentinelRefreshRequest::new();
println!("{}", request.build());
}
#[test]
fn test_with_fourth_and_tenth() {
let request = GenerateSentinelRefreshRequest::new().with_fourth(100).with_tenth(200);
println!("{}", request.build());
}
#[test]
fn test_hash_request() {
let request = GenerateSentinelRefreshRequest::new().with_fourth(100).with_tenth(200);
let hash = GenerateSentinelRefreshRequest::hash_request(0.1, request.build());
println!("{}", hash.iter().map(|b| format!("{:02x}", b)).collect::<String>());
}
#[test]
fn test_with_fourth_and_tenth_2() {
let (request, base64_request) = GenerateSentinelRefreshRequest::new().with_fourth_and_tenth();
println!("{}", base64_request);
}
}
@@ -0,0 +1,69 @@
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
use thiserror::Error;
use idempotency::uuid::generate_random_uuid;
use super::request::GenerateSentinelRefreshRequest;
const SORA_IMAGE_GEN_URL: &str = "https://chatgpt.com/backend-api/sentinel/req";
/// This user agent is tied to the sentinel generation. If we need to change it, we may need to change sentinel generation too.
const USER_AGENT : &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36";
const SENTINEL_FLOW: &str = "sora_create_task";
#[derive(Debug, Serialize, Deserialize)]
pub struct SentinelRequest {
#[serde(rename = "p")]
problem: String,
#[serde(rename = "id")]
id: String,
#[serde(rename = "flow")]
flow: String,
}
impl SentinelRequest {
pub fn new(problem: String) -> Self {
let id = generate_random_uuid();
Self { problem, id, flow: SENTINEL_FLOW.to_string() }
}
}
#[derive(Debug, Serialize, Deserialize)]
pub struct SentinelResponse {
persona: String,
token: String,
}
pub async fn generate_token() -> AnyhowResult<String> {
let (_request, base64_request) = GenerateSentinelRefreshRequest::new().with_fourth_and_tenth();
let request = SentinelRequest::new(base64_request);
let client = reqwest::Client::new();
let response = client.post(SORA_IMAGE_GEN_URL)
.header("User-Agent", USER_AGENT)
.header("Content-Type", "application/json")
.json(&request)
.send()
.await?;
let response_json: SentinelResponse = response.json().await?;
Ok(response_json.token)
}
#[cfg(test)]
mod tests {
use super::*;
#[tokio::test]
async fn test_generate_token() {
let token = generate_token().await.unwrap();
println!("{}", token);
}
}
@@ -1,2 +1,4 @@
pub use refresh::refresh_sentinel;
pub mod refresh;
pub mod refresh;
pub mod generate;
@@ -0,0 +1,48 @@
use std::error::Error;
use std::fmt::{Display, Formatter};
#[derive(Debug)]
pub enum SoraError {
/// Unauthorized, cookie and/or bearer token expired. We'll need to ask for a refreshed login.
///
/// Example message, e.g. from the upload endpoint:
/// {
/// "error": {
/// "message": "Your authentication token has expired. Please try signing in again.",
/// "type": "invalid_request_error",
/// "param": null,
/// "code": "token_expired"
/// }
/// }
UnauthorizedCookieOrBearerExpired,
/// Another error occurred.
OtherBadStatus(anyhow::Error),
/// Reqwest Error
ReqwestError(reqwest::Error),
}
impl Error for SoraError {}
impl Display for SoraError {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
SoraError::UnauthorizedCookieOrBearerExpired => {
write!(f, "Unauthorized: cookie and/or bearer token expired")
}
SoraError::OtherBadStatus(err) => {
write!(f, "Other error: {}", err)
}
SoraError::ReqwestError(err) => {
write!(f, "Reqwest error: {}", err)
}
}
}
}
impl From<reqwest::Error> for SoraError {
fn from(err: reqwest::Error) -> SoraError {
SoraError::ReqwestError(err)
}
}
@@ -3,6 +3,7 @@ use errors::AnyhowResult;
use reqwest::multipart::{Form, Part};
use reqwest::Client;
use serde::Deserialize;
use crate::sora_error::SoraError;
const SORA_UPLOAD_MEDIA_URL: &str = "https://sora.com/backend/uploads";
@@ -25,6 +26,17 @@ Response:
"size_bytes": 118501,
"thumbnail_url": "https://videos.openai.com/vg-assets/assets%2Fclient_upload%2Fmedia%2Fuser-9wf6JFdgRSJLjvSJ53LNAbV8%2Fmedia_01jqt9vt20erx9zvryf3v1pecx.jpg?st=2025-04-02T02%3A39%3A01Z&se=2025-04-08T03%3A39%3A01Z&sks=b&skt=2025-04-02T02%3A39%3A01Z&ske=2025-04-08T03%3A39%3A01Z&sktid=a48cca56-e6da-484e-a814-9c849652bcb3&skoid=3d249c53-07fa-4ba4-9b65-0bf8eb4ea46a&skv=2019-02-02&sv=2018-11-09&sr=b&sp=r&spr=https%2Chttp&sig=9aU9xKI0BkhRtrTb0RD05%2BT6prBnmXk1I7nV2nTsFbY%3D&az=oaivgprodscus"
}
Error response:
Failed to upload scene media to Sora: Upload failed with status 401 Unauthorized: {
"error": {
"message": "Your authentication token has expired. Please try signing in again.",
"type": "invalid_request_error",
"param": null,
"code": "token_expired"
}
}
*/
#[derive(Debug, Deserialize)]
@@ -50,7 +62,13 @@ pub (crate) struct SoraMediaUploadRequest<'a> {
pub credentials: &'a SoraCredentials,
}
pub (crate) async fn upload_media_http_request(file_bytes: Vec<u8>, filename: String, mime_type: &str, credentials: &SoraCredentials) -> AnyhowResult<SoraMediaUploadResponse> {
pub (crate) async fn upload_media_http_request(
file_bytes: Vec<u8>,
filename: String,
mime_type: &str,
credentials: &SoraCredentials,
) -> Result<SoraMediaUploadResponse, SoraError> {
// Create multipart form
let part = Part::bytes(file_bytes) // NB: Reqwest needs to own the bytes.
.file_name(filename) // NB: Reqwest needs to own the bytes
@@ -70,12 +88,29 @@ pub (crate) async fn upload_media_http_request(file_bytes: Vec<u8>, filename: St
// Check response status
if !response.status().is_success() {
let status = response.status();
let text = response.text().await?;
return Err(anyhow::anyhow!("Upload failed with status {}: {}", status, text));
let error = classify_error(response).await;
return Err(error);
}
// Parse response
let upload_response = response.json::<SoraMediaUploadResponse>().await?;
Ok(upload_response)
}
async fn classify_error(response: reqwest::Response) -> SoraError {
let status = response.status();
let message = match response.text().await {
Ok(text) => text,
Err(err) => return SoraError::ReqwestError(err),
};
let cookie_expired =
message.contains("Your authentication token has expired. Please try signing in again.")
|| message.contains("token_expired");
if cookie_expired {
SoraError::UnauthorizedCookieOrBearerExpired
} else {
SoraError::OtherBadStatus(anyhow::anyhow!("Upload failed with status {}: {}", status, message))
}
}
@@ -4,7 +4,6 @@
/// NB: `sqlx::query` is spammy and dumps all queries as "info"-level log lines.
/// NB: `hyper::proto::h1::io` is incredibly spammy and logs every chunk of bytes in very large files being downloaded.
pub const DEFAULT_RUST_LOG: &str = concat!(
"debug,",
"actix_web=info,",
"sqlx::query=warn,",
"hyper::proto::h1::io=warn,",
@@ -21,9 +21,14 @@ use crate::helpers::boolean_converters::i8_to_bool;
use crate::payloads::generic_inference_args::generic_inference_args::{GenericInferenceArgs, PolymorphicInferenceArgs};
use crate::queries::generic_inference::web::job_status::{GenericInferenceJobStatus, RequestDetails, ResultDetails, UserDetails};
pub enum SessionUser<'a> {
User(&'a UserToken),
Anonymous(&'a AnonymousVisitorTrackingToken),
}
pub struct ListSessionJobsForUserArgs<'a> {
// TODO(bt,2024-04-23): Support AnonymousVisitorToken
pub user_token: &'a UserToken,
pub user: SessionUser<'a>,
pub maybe_include_job_statuses: Option<&'a HashSet<JobStatusPlus>>,
pub maybe_exclude_job_statuses: Option<&'a HashSet<JobStatusPlus>>,
}
@@ -63,14 +68,29 @@ pub async fn list_session_jobs_from_connection(
// Furthermore, even though the join query's `created_at` lacks an index, sort
// by id will accomplish this quickly.
{
query_builder.push(r#"
INNER JOIN (
SELECT id
FROM generic_inference_jobs
WHERE maybe_creator_user_token =
"#);
query_builder.push_bind(args.user_token.to_string());
match args.user {
SessionUser::User(user_token) => {
query_builder.push(r#"
INNER JOIN (
SELECT id
FROM generic_inference_jobs
WHERE maybe_creator_user_token =
"#);
query_builder.push_bind(user_token.to_string());
}
SessionUser::Anonymous(avt_token) => {
query_builder.push(r#"
INNER JOIN (
SELECT id
FROM generic_inference_jobs
WHERE maybe_creator_anonymous_visitor_token =
"#);
query_builder.push_bind(avt_token.to_string());
query_builder.push(r#"
AND maybe_creator_user_token IS NULL
"#);
}
}
query_builder.push(r#"
AND created_at > DATE_SUB(NOW(), INTERVAL 36 HOUR)
@@ -16,9 +16,12 @@ use log::warn;
use mysql_queries::queries::media_files::get::get_media_file::get_media_file;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::image_gen::SoraError;
use openai_sora_client::sentinel_refresh::generate::token::generate_token;
use openai_sora_client::sentinel_refresh::refresh_sentinel;
use serde::Deserialize;
use serde::Serialize;
use shared_service_components::sora_redis_credentials::set_sora_credential_field_in_redis::set_sora_credential_field_in_redis;
use shared_service_components::sora_redis_credentials::keys::RedisSoraCredentialSubkey;
use sqlx::MySqlPool;
use tempdir::TempDir;
use utoipa::ToSchema;
@@ -76,6 +79,9 @@ pub struct EnqueueStudioImageGenRequest {
/// The user's image generation prompt.
pub prompt: String,
/// Turn off the system prompt.
pub disable_system_prompt: Option<bool>,
/// Additional images to include (optional). Up to nine images.
pub maybe_additional_images: Option<Vec<MediaFileToken>>,
@@ -94,6 +100,7 @@ pub enum EnqueueImageGenRequestError {
NotAuthorized,
ServerError,
RateLimited,
TooManyConcurrentTasks,
}
impl ResponseError for EnqueueImageGenRequestError {
@@ -103,6 +110,7 @@ impl ResponseError for EnqueueImageGenRequestError {
EnqueueImageGenRequestError::NotAuthorized => StatusCode::UNAUTHORIZED,
EnqueueImageGenRequestError::ServerError => StatusCode::INTERNAL_SERVER_ERROR,
EnqueueImageGenRequestError::RateLimited => StatusCode::TOO_MANY_REQUESTS,
EnqueueImageGenRequestError::TooManyConcurrentTasks => StatusCode::TOO_MANY_REQUESTS,
}
}
@@ -112,6 +120,7 @@ impl ResponseError for EnqueueImageGenRequestError {
EnqueueImageGenRequestError::NotAuthorized => "unauthorized".to_string(),
EnqueueImageGenRequestError::ServerError => "server error".to_string(),
EnqueueImageGenRequestError::RateLimited => "rate limited".to_string(),
EnqueueImageGenRequestError::TooManyConcurrentTasks => "too many concurrent tasks".to_string(),
};
to_simple_json_error(&error_reason, self.status_code())
@@ -283,30 +292,69 @@ pub async fn enqueue_studio_image_generation_handler(http_request: HttpRequest,
None => 1,
};
let prompt = request.prompt.clone();
let prompt = create_prompt(&request);
let mut response = sora_image_gen_remix(SoraImageGenRemixRequest { prompt: prompt.clone(), num_images: NumImages::One, image_size: ImageSize::Square, sora_media_tokens: sora_media_tokens.clone(), credentials: &sora_credentials }).await;
let mut response = sora_image_gen_remix(SoraImageGenRemixRequest {
prompt: prompt.clone(),
num_images: NumImages::One,
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: &sora_credentials
}).await;
debug!("Sora image gen response: {:?}", response);
if let Err(SoraError::SentinelBlock(msg)) = &response {
error!("Sora sentinel block, attempting refresh: {}", msg);
// Try to refresh sentinel
match refresh_sentinel().await {
Ok(new_sentinel) => {
// Update the credentials with new sentinel
match generate_token().await {
Ok(sentinel_token) => {
set_sora_credential_field_in_redis(
&mut redis,
RedisSoraCredentialSubkey::Sentinel,
&sentinel_token
).map_err(|e| {
EnqueueImageGenRequestError::ServerError
})?;
let updated_sora_credentials = get_sora_credentials_from_request(&http_request, &mut redis).map_err(|e| {
error!("sora credential error: {:?}", e);
EnqueueImageGenRequestError::ServerError
})?;
// Retry the request with new sentinel
response = sora_image_gen_remix(SoraImageGenRemixRequest { prompt: prompt.clone(), num_images: NumImages::One, image_size: ImageSize::Square, sora_media_tokens: sora_media_tokens.clone(), credentials: &updated_sora_credentials }).await;
response = sora_image_gen_remix(SoraImageGenRemixRequest {
prompt: prompt.clone(),
num_images: NumImages::One,
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: &updated_sora_credentials
}).await;
},
Err(e) => {
error!("Failed to refresh Sora sentinel: {:?}", e);
},
// Fallback to the old refresh stragegy relying on the external service
match refresh_sentinel().await {
Ok(new_sentinel) => {
// Update the credentials with new sentinel
let updated_sora_credentials = get_sora_credentials_from_request(&http_request, &mut redis).map_err(|e| {
error!("sora credential error: {:?}", e);
EnqueueImageGenRequestError::ServerError
})?;
// Retry the request with new sentinel
response = sora_image_gen_remix(SoraImageGenRemixRequest {
prompt: prompt.clone(),
num_images: NumImages::One,
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: &updated_sora_credentials
}).await;
},
Err(e) => {
error!("Failed to refresh Sora sentinel: {:?}", e);
}
}
}
}
}
@@ -315,21 +363,34 @@ pub async fn enqueue_studio_image_generation_handler(http_request: HttpRequest,
SoraError::TokenExpired(msg) => {
error!("Sora token expired, needs refresh: {}", msg);
// TODO: Implement token refresh logic here
EnqueueImageGenRequestError::ServerError
},
SoraError::SentinelBlock(msg) => {
error!("Sora sentinel block still occurring after refresh: {}", msg);
EnqueueImageGenRequestError::ServerError
},
SoraError::TooManyConcurrentTasks(msg) => {
error!("Sora too many concurrent tasks: {}", msg);
EnqueueImageGenRequestError::TooManyConcurrentTasks
},
_ => {
error!("Failed to call Sora image generation: {:?}", err);
EnqueueImageGenRequestError::ServerError
},
}
EnqueueImageGenRequestError::ServerError
})?;
debug!("Sora image gen response: {:?}", response);
// Store the actual inference args that will go to the database
let inference_args = SoraImageGenArgs { prompt: Some(prompt), scene_snapshot_media_token: Some(request.snapshot_media_token.clone()), maybe_additional_media_file_tokens: Some(additional_images), maybe_number_of_samples: Some(number_of_samples), maybe_sora_media_upload_tokens: Some(sora_media_tokens), maybe_sora_task_id: Some(response.task_id) };
let inference_args = SoraImageGenArgs {
prompt: Some(prompt),
scene_snapshot_media_token: Some(request.snapshot_media_token.clone()),
maybe_additional_media_file_tokens: Some(additional_images),
maybe_number_of_samples: Some(number_of_samples),
maybe_sora_media_upload_tokens: Some(sora_media_tokens),
maybe_sora_task_id: Some(response.task_id)
};
// create the inference args here
let maybe_avt_token = server_state.avt_cookie_manager.get_avt_token_from_request(&http_request);
@@ -348,7 +409,10 @@ pub async fn enqueue_studio_image_generation_handler(http_request: HttpRequest,
maybe_cover_image_media_file_token: None,
maybe_raw_inference_text: None,
maybe_max_duration_seconds: None,
maybe_inference_args: Some(GenericInferenceArgs { inference_category: Some(InferenceCategoryAbbreviated::ImageGeneration), args: Some(PolymorphicInferenceArgs::Sg(inference_args)) }),
maybe_inference_args: Some(GenericInferenceArgs {
inference_category: Some(InferenceCategoryAbbreviated::ImageGeneration),
args: Some(PolymorphicInferenceArgs::Sg(inference_args)),
}),
maybe_creator_user_token: maybe_user_token.as_ref(),
maybe_avt_token: maybe_avt_token.as_ref(),
creator_ip_address: &ip_address,
@@ -412,7 +476,10 @@ async fn query_and_download_media_file(media_file_token: &MediaFileToken, downlo
},
};
let media_file_bucket_path = MediaFileBucketPath::from_object_hash(&media_file.public_bucket_directory_hash, media_file.maybe_public_bucket_prefix.as_deref(), media_file.maybe_public_bucket_extension.as_deref());
let media_file_bucket_path = MediaFileBucketPath::from_object_hash(
&media_file.public_bucket_directory_hash,
media_file.maybe_public_bucket_prefix.as_deref(),
media_file.maybe_public_bucket_extension.as_deref());
let extension = media_file.maybe_public_bucket_extension.clone().unwrap_or_else(|| ".jpg".to_string());
let download_filename = format!("download_file_{}.{}", media_file.token.as_str(), extension);
@@ -424,3 +491,30 @@ async fn query_and_download_media_file(media_file_token: &MediaFileToken, downlo
Ok(download_file_path)
}
const SYSTEM_PROMPT_SINGLE_IMAGE: &str =
"Use the image as a reference for the composition exactly. Copy the character pose, layout, scene, and the objects exactly - no deviation.";
const SYSTEM_PROMPT_MULTI_IMAGE: &str =
"Use the first image as a reference for the composition exactly. Copy the character pose, layout, scene, and the objects exactly from the first image - no deviation.";
fn create_prompt(request: &EnqueueStudioImageGenRequest) -> String {
let disable_system_prompt = request.disable_system_prompt
.clone()
.unwrap_or(false);
if disable_system_prompt {
return request.prompt.clone();
}
let extra_image_count = request.maybe_additional_images
.as_ref()
.map(|v| v.len())
.unwrap_or(0);
if extra_image_count == 0 {
format!("{} {}", SYSTEM_PROMPT_SINGLE_IMAGE, request.prompt)
} else {
format!("{} {}", SYSTEM_PROMPT_MULTI_IMAGE, request.prompt)
}
}
@@ -31,7 +31,7 @@ use enums::by_table::generic_inference_jobs::inference_category::InferenceCatego
use enums::common::job_status_plus::JobStatusPlus;
use enums::no_table::style_transfer::style_transfer_name::StyleTransferName;
use mysql_queries::queries::generic_inference::web::job_status::GenericInferenceJobStatus;
use mysql_queries::queries::generic_inference::web::list_session_jobs::{list_session_jobs_from_connection, ListSessionJobsForUserArgs};
use mysql_queries::queries::generic_inference::web::list_session_jobs::{list_session_jobs_from_connection, ListSessionJobsForUserArgs, SessionUser};
use primitives::numerics::i64_to_u64_zero_clamped::i64_to_u64_zero_clamped;
use redis_common::redis_keys::RedisKeys;
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
@@ -238,11 +238,6 @@ pub async fn list_session_jobs_handler(
ListSessionJobsError::ServerError
})?;
let session = match maybe_user_session {
Some(session) => session,
None => return Err(ListSessionJobsError::NotAuthorized),
};
let include_states = query.include_states
.as_deref()
.map(|s| s.split(",")
@@ -255,8 +250,23 @@ pub async fn list_session_jobs_handler(
.filter_map(|status| JobStatusPlus::from_str(status).ok())
.collect::<HashSet<_>>());
let user;
match (maybe_user_session.as_ref(), maybe_avt_token.as_ref()) {
(Some(session), _) => {
user = SessionUser::User(&session.user_token_typed);
}
(None, Some(avt_token)) => {
user = SessionUser::Anonymous(avt_token);
}
(None, None) => {
// TODO(bt,2025-04-15): We should install an AVT cookie.
return Err(ListSessionJobsError::NotAuthorized);
}
}
let args = ListSessionJobsForUserArgs {
user_token: &session.user_token_typed,
user,
maybe_include_job_statuses: include_states.as_ref(),
maybe_exclude_job_statuses: exclude_states.as_ref(),
};
+1
View File
@@ -0,0 +1 @@
node_modules
+56
View File
@@ -0,0 +1,56 @@
{
"root": true,
"ignorePatterns": [
"**/*"
],
"plugins": [
"@nx"
],
"overrides": [
{
"files": [
"*.ts",
"*.tsx",
"*.js",
"*.jsx"
],
"rules": {
"@nx/enforce-module-boundaries": [
"error",
{
"enforceBuildableLibDependency": true,
"allow": [],
"depConstraints": [
{
"sourceTag": "*",
"onlyDependOnLibsWithTags": [
"*"
]
}
]
}
]
}
},
{
"files": [
"*.ts",
"*.tsx"
],
"extends": [
"plugin:@nx/typescript"
],
"rules": {}
},
{
"files": [
"*.js",
"*.jsx"
],
"extends": [
"plugin:@nx/javascript"
],
"rules": {}
}
]
}
+2 -2
View File
@@ -1,7 +1,7 @@
{
"recommendations": [
"nrwl.angular-console",
"esbenp.prettier-vscode"
"esbenp.prettier-vscode",
"dbaeumer.vscode-eslint"
]
}
+17
View File
@@ -27,4 +27,21 @@ Run dev server:
nx dev editor3d
```
```
call when starting on main or a new branch:
./clean_modules
The names for @frontend and @storyteller come from the package.json file in the libs folder.
import { Login } from "@frontend/login";
import { api } from "@storyteller/api";
building:
npx nx build editor3d
npx nx build editor2d
dev:
npx nx dev editor3d
npx nx dev editor2d
```
+1 -2
View File
@@ -1,5 +1,5 @@
[build]
#command = "./script/netlify_build.sh"
command = "./apps/editor2d/script/netlify_build.sh"
publish = "apps/editor2d/dist/"
#[build.environment]
@@ -16,7 +16,6 @@ publish = "apps/editor2d/dist/"
for = "/build/*"
[headers.values]
"Cache-Control" = "public, max-age=31560000, immutable"
# [[redirects]]
+8 -1
View File
@@ -4,7 +4,14 @@
"version": "0.0.0",
"type": "module",
"nx": {
"name": "editor2d"
"name": "editor2d",
"targets": {
"build": {
"dependsOn": [
"^build"
]
}
}
},
"scripts": {
"dev": "echo 'editor2d' && vite",
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

@@ -15,6 +15,6 @@ cp _redirects dist/
echo "Copy netlify 404.html page into dist"
cp "404.html" dist/
echo "List files after build"
echo "List final files in build"
find dist/
@@ -0,0 +1,122 @@
import { faFolder } from "@fortawesome/pro-solid-svg-icons"
import { useEffect, useState } from "react";
import { Button } from "~/components/ui"
const testUrl = "https://api.fakeyou.com/v1/media_files/list/user/hanashi?include_user_uploads=true&page_size=24&filter_media_classes=unknown";
const getTestData = (callback: (data: any) => void) => {
const netCall = async () => {
// GET the test url
const testResults = await fetch(testUrl);
const testJson = await testResults.json();
console.log("testResults", testJson);
return testJson;
}
netCall().then(callback);
}
export const GalleryRootComponent = ({
className
}: {
className?: string
}) => {
const [testData, setTestData] = useState<any>();
useEffect(() => {
getTestData(setTestData);
}, [])
const aggregatedData = aggregateGalleryData({ data: testData });
return (
<div className={className}>
<div className="m-32">
<div className="flex w-fit gap-x-6 items-center">
<span className="text-white font-bold text-5xl">My Gallery</span>
<Button variant="secondary" icon={faFolder} className="h-8">
Open Folder
</Button>
</div>
<div className="overflow-y-auto mt-12 flex flex-col gap-y-8">
{Array.from(aggregatedData.entries()).map(([dateEpoch, results]) => (<GallerySection dateEpoch={dateEpoch} results={results} />))}
</div>
</div>
</div>
)
}
const GallerySection = ({
dateEpoch,
results
}: {
dateEpoch: number,
results: any[]
}) => {
const dateObj = new Date(dateEpoch);
const title = dateObj.toLocaleDateString("en-US", {
year: "numeric",
month: "long",
day: "numeric"
});
return (
<div className="w-full">
<span className="text-lg">{title}</span>
<div className="flex gap-4 flex-wrap mt-4">
{results.map((result) => (<GalleryCard result={result} />))}
</div>
</div>
)
}
const GalleryCard = ({ result }: {
result: {
media_links: {
cdn_url: string,
maybe_thumbnail_template: string,
},
}
}) => {
return (
<div className="rounded-md size-48 flex items-center justify-center overflow-hidden">
<img src={result.media_links.cdn_url} alt="Thumbnail" className="h-full w-full object-cover" />
</div>
)
}
const aggregateGalleryData = ({
data
}: {
data?: {
results: any[]
}
}) => {
const aggregateMap = new Map<number, any[]>();
if (!data) {
return aggregateMap;
}
data.results.forEach((result) => {
if (result.media_type !== "image") {
return;
}
const dateString = result.updated_at as string;
const dateObj = new Date(dateString);
// Move it the start of the day for comparison
dateObj.setUTCHours(0, 0, 0, 0);
const dateEpoch = dateObj.getTime();
if (!aggregateMap.has(dateEpoch)) {
aggregateMap.set(dateEpoch, []);
}
aggregateMap.get(dateEpoch)!.push(result);
})
return aggregateMap;
}
@@ -0,0 +1 @@
export * from "./GalleryRootComponent";
+40 -19
View File
@@ -148,7 +148,6 @@ export class ApiManager {
formData.append("file", blob);
}
// console.log(`HELLO${formData}`);
return this.fetchMultipartFormData<T>(endpoint, {
method: "POST",
headers: {
@@ -281,10 +280,12 @@ export class Api extends ApiManager {
}
public async enqueueImageGeneration({
disableSystemPrompt,
prompt,
snapshotMediaToken,
additionalImages,
}: {
disableSystemPrompt: boolean;
prompt: string;
snapshotMediaToken: string;
additionalImages?: string[];
@@ -294,6 +295,7 @@ export class Api extends ApiManager {
const uuidIdempotencyToken = crypto.randomUUID(); // Generate a new UUID
const body = {
uuid_idempotency_token: uuidIdempotencyToken,
disable_system_prompt: disableSystemPrompt,
prompt,
snapshot_media_token: snapshotMediaToken, // Changed from scene_media_token to snapshot_media_token
additional_images: additionalImages,
@@ -302,6 +304,7 @@ export class Api extends ApiManager {
const postResponse = await this.post<
{
uuid_idempotency_token: string;
disable_system_prompt: boolean;
prompt: string;
snapshot_media_token: string;
additional_images?: string[];
@@ -327,7 +330,10 @@ export class Api extends ApiManager {
return result;
}
public async pollJobSession(jobToken: string): Promise<
public async pollJobSession(
jobToken: string,
thumbnailWidth: number = 256,
): Promise<
ApiResponse<{
job_token: string;
request: {
@@ -338,6 +344,9 @@ export class Api extends ApiManager {
progress_percentage: number;
};
result: {
image_url: string;
thumbnail_url: string;
public_bucket_path: string;
generated_images?: string[];
error?: string;
};
@@ -345,41 +354,53 @@ export class Api extends ApiManager {
> {
const endpoint = `${this.ApiTargets.BaseApi}/v1/jobs/job/${jobToken}`;
const response = await this.get<{
success?: boolean;
status?: string;
result?: {
generated_images?: string[];
error?: string;
};
BadInput?: string;
}>({
const response = await this.get<any>({
endpoint,
});
console.log("Job Polling Response:");
console.log(response);
let image_url = response.state.maybe_result?.media_links?.cdn_url;
let public_bucket_path =
response.state.maybe_result?.maybe_public_bucket_media_path;
let thumbnail_url =
response.state.maybe_result?.media_links?.maybe_thumbnail_template?.replace(
"{WIDTH}",
thumbnailWidth.toString(),
);
let progress_percentage = response.state.status.progress_percentage;
let status_string = response.state.status.status;
console.log("Image URL:", image_url);
console.log("Thumbnail URL:", thumbnail_url);
console.log("Progress Percentage:", progress_percentage);
console.log("Status", status_string);
console.log("response FROM JOBS");
console.log(response);
const success = response.success ?? false;
const status = response.status ?? "";
const result = response.result ?? {
generated_images: [],
error: undefined,
};
const errorMessage = response.BadInput;
return {
success,
data: {
result,
result: {
image_url: image_url || "",
thumbnail_url: thumbnail_url || "",
public_bucket_path: public_bucket_path || "",
error: undefined,
},
job_token: jobToken,
request: {
maybe_model_title: "Image Generation",
},
status: {
status: status.toLowerCase(),
progress_percentage: 0,
status: status_string.toLowerCase(),
progress_percentage: progress_percentage,
},
},
errorMessage,
@@ -55,7 +55,7 @@ export class Engine {
private uiLayer: Konva.Layer;
private previewLayer: Konva.Layer;
private offScreenCanvas: OffscreenCanvas;
private realTimeDrawEngine: RealTimeDrawEngine;
public realTimeDrawEngine: RealTimeDrawEngine;
private nodesManager: NodesManager;
private nodeIsolator: NodeIsolator;
private nodeTransformer: NodeTransformer;
@@ -66,8 +66,8 @@ export class Engine {
private backgroundNode: ShapeNode | null = null;
private sceneManager: SceneManager;
private undoStackManager: UndoStackManager;
private commandManager: CommandManager;
public undoStackManager: UndoStackManager;
public commandManager: CommandManager;
private videoExtractionHandler: VideoExtractionHandler;
public segmentationButtonCanBePressed: boolean = true;
@@ -56,11 +56,8 @@ export class ShapeNode extends NetworkedNode {
switch (shapeType) {
case ShapeType.CIRCLE:
shape = new Konva.Ellipse({
x: size.width,
y: size.height,
radiusX: size.width,
radiusY: size.height,
...transform,
radiusX: size.width / 2,
radiusY: size.height / 2,
fill: color,
strokeScaleEnabled: false,
draggable: true,
@@ -80,8 +77,6 @@ export class ShapeNode extends NetworkedNode {
case ShapeType.TRIANGLE:
shape = new Konva.Shape({
x: 0,
y: 0,
width: size.width,
height: size.height,
...transform,
@@ -113,12 +108,20 @@ export class ShapeNode extends NetworkedNode {
this.size = size;
this.color = color;
// Add shape to group
// Center the shape
const centerPosition = NodeUtilities.positionNodeOnCanvasCenter({
canvasOffset: canvasPosition,
componentSize: canvasSize,
componentSize: this.size,
maxSize: canvasSize,
});
// If the shape is an ellipse, the center is the origin so we need to offset it
if (shapeType === ShapeType.CIRCLE) {
centerPosition.x += size.width / 2;
centerPosition.y += size.height / 2;
}
// Add shape to group
// bring to the front
this.kNode.moveToTop();
this.kNode.setPosition(centerPosition);
@@ -1,14 +1,12 @@
import { useCallback, useRef } from "react";
import { useCallback, useRef, useState } from "react";
// Components
import { KonvaCanvasContainer } from "./KonvaCanvasContainer";
import { ContextualButtonRetry } from "./ContextualButtonRetry";
import { ContextualToolbarNode } from "./ContextualToolbarNode";
import { ContextualLoadingBar } from "./ContextualLoadingBar";
import { SignaledCanvasDragDropFiles } from "./SignaledCanvasDragDropFiles";
import { SignaledDialogs } from "./SignaledDialogs";
import { SignaledToolbarMain } from "./SignaledToolbarMain";
import { SignaledToolbarVideoExtraction } from "./SignaledToolbarVideoExtraction";
import { PromptBox } from "~/components/PromptBox";
// The KonvaApp is the root of the Konva stage
// and only entry point for anything in Konva JS
@@ -22,9 +20,8 @@ import { useLayoutContext } from "./contextSignals/layout";
// common hooks
import { useRenderCounter } from "~/hooks/useRenderCounter";
import { useNavigate } from "react-router-dom";
import { SignaledMagicBox } from "./SignaledMagicBox";
import { SignaledPromptSlider } from "./SignaledPromptSlider";
import { SignaledPromptText } from "./SignaledPromptText";
import { UndoRedo } from "~/components/reusable/UndoRedo/UndoRedo";
export const KonvaRootComponent = ({
className,
@@ -39,44 +36,66 @@ export const KonvaRootComponent = ({
const navigate = useNavigate();
const appUiContext = useAppUiContext();
const layoutContext = useLayoutContext();
// Add state to track engine initialization
const [isEngineReady, setIsEngineReady] = useState(false);
const engineRef = useRef<EngineType | null>(null);
const konvaContainerCallbackRef = useCallback((node: HTMLDivElement) => {
if (node !== null && engineRef.current === null) {
const options = {
navigate: navigate,
sceneToken: sceneToken,
};
engineRef.current = KonvaApp(node, options);
// Only initialize if we have a node and haven't initialized yet
if (node !== null && !isEngineReady && engineRef.current === null) {
try {
const options = {
navigate: navigate,
sceneToken: sceneToken,
};
engineRef.current = KonvaApp(node, options);
// Only set ready if initialization succeeded
if (engineRef.current) {
setIsEngineReady(true);
}
} catch (error) {
console.error("Failed to initialize KonvaApp:", error);
engineRef.current = null;
setIsEngineReady(false);
}
}
}, []);
}, [navigate, sceneToken, isEngineReady]);
return (
<>
{/* Always render the canvas container */}
<KonvaCanvasContainer
ref={konvaContainerCallbackRef}
className={className}
// retreive the classNames from the parent for sizing/styling
/>
{/* <SignaledMagicBox /> */}
<SignaledCanvasDragDropFiles
openAddImage={appUiContext.openAddImage}
openAddVideo={appUiContext.openAddVideo}
/>
<PromptBox />
<SignaledToolbarMain
layoutSignal={layoutContext.signal}
appUiContext={appUiContext}
/>
{/* <SignaledToolbarVideoExtraction /> */}
<SignaledDialogs
appUiSignal={appUiContext.signal}
resetAll={appUiContext.resetAll}
/>
{/* Conditionally render the loading overlay */}
{!isEngineReady && (
<div className="absolute inset-0 flex items-center justify-center">
<div className="text-gray-500">Initializing canvas...</div>
</div>
)}
{/* <ContextualLoadingBar /> */}
<ContextualToolbarNode />
{/* <ContextualButtonRetry /> */}
{/* Conditionally render the UI components that need the engine */}
{isEngineReady && engineRef.current && (
<UndoRedo engine={engineRef.current}>
<SignaledCanvasDragDropFiles
openAddImage={appUiContext.openAddImage}
openAddVideo={appUiContext.openAddVideo}
/>
<PromptBox />
<SignaledToolbarMain
layoutSignal={layoutContext.signal}
appUiContext={appUiContext}
/>
<SignaledDialogs
appUiSignal={appUiContext.signal}
resetAll={appUiContext.resetAll}
/>
<ContextualToolbarNode />
</UndoRedo>
)}
</>
);
};
@@ -17,6 +17,7 @@ import { EditModeRootComponent } from "~/EditModeRootComponent/EditModeRootCompo
import { useRef } from "react";
import { EngineType } from "~/KonvaApp";
import { GenerationEngine } from "~/KonvaApp/GenerationEngine";
import { GalleryRootComponent } from "~/GalleryRootComponent";
export const Main = withProtectionRoute(() => {
// This is a hook that will log the number of times the component has rerendered
@@ -59,8 +60,10 @@ export const Main = withProtectionRoute(() => {
}
return (
<div className="fixed grid h-full w-full grid-cols-12 grid-rows-12">
{childView}
<div className="fixed h-full w-full">
<div className="fixed grid h-full w-full grid-cols-12 grid-rows-12">
{childView}
</div>
<div className="absolute top-0 w-full p-3.5">
<div className="relative flex w-full">
<div className="absolute left-0">
@@ -74,6 +77,8 @@ export const Main = withProtectionRoute(() => {
</div>
</div>
</div>
{appModeValue === "gallery" &&
(<GalleryRootComponent className="fixed h-full w-full z-1 bg-black" />)}
</div>
);
});
@@ -1,8 +1,10 @@
import React, { createContext, useContext, useState } from "react";
interface JobContextType {
jobToken: string | null;
setJobToken: (token: string | null) => void;
jobTokens: string[];
addJobToken: (token: string) => void;
removeJobToken: (token: string) => void;
clearJobTokens: () => void;
}
const JobContext = createContext<JobContextType | undefined>(undefined);
@@ -10,10 +12,24 @@ const JobContext = createContext<JobContextType | undefined>(undefined);
export const JobProvider: React.FC<{ children: React.ReactNode }> = ({
children,
}) => {
const [jobToken, setJobToken] = useState<string | null>(null);
const [jobTokens, setJobTokens] = useState<string[]>([]);
const addJobToken = (token: string) => {
setJobTokens((prevTokens) => [...prevTokens, token]);
};
const removeJobToken = (token: string) => {
setJobTokens((prevTokens) => prevTokens.filter((t) => t !== token));
};
const clearJobTokens = () => {
setJobTokens([]);
};
return (
<JobContext.Provider value={{ jobToken, setJobToken }}>
<JobContext.Provider
value={{ jobTokens, addJobToken, removeJobToken, clearJobTokens }}
>
{children}
</JobContext.Provider>
);
@@ -47,7 +47,8 @@ export const PromptBox = () => {
const isDesktopApp = window.navigator.userAgent.includes("Tauri");
console.log("Is this a desktop app?", isDesktopApp);
const { jobToken, setJobToken } = useJobContext();
const { jobTokens, addJobToken, removeJobToken, clearJobTokens } =
useJobContext();
//const { lastRenderedBitmap } = useCanvasSignal();
const [prompt, setPrompt] = useState("");
@@ -119,7 +120,7 @@ export const PromptBox = () => {
setUploadingImages((prev) =>
prev.filter((img) => img.id !== uploadId),
);
console.log("Reference image added:", referenceImage);
toast.success("Reference image added.");
} else if (
newState.status === UploaderStates.assetError ||
newState.status === UploaderStates.imageCreateError
@@ -127,7 +128,8 @@ export const PromptBox = () => {
setUploadingImages((prev) =>
prev.filter((img) => img.id !== uploadId),
);
console.error("Upload failed");
toast.error("Upload failed. Please try again.");
}
},
});
@@ -206,7 +208,7 @@ export const PromptBox = () => {
const api = new Api();
let image = getCanvasRenderBitmap();
if (image === undefined) {
console.error(
toast.error(
"Error: Unable to generate image. Please check the input and try again.",
);
return;
@@ -229,30 +231,28 @@ export const PromptBox = () => {
});
if (snapshotMediaToken.data === undefined) {
console.error("Error: Unable to upload scene snapshot.");
toast.error(
"Error: Unable to upload scene snapshot Please try again.",
);
return;
}
console.log("useSystemPrompt", useSystemPrompt);
console.log("Snapshot media token:", snapshotMediaToken.data);
const response = await api.enqueueImageGeneration({
disableSystemPrompt: !useSystemPrompt,
prompt: prompt,
snapshotMediaToken: snapshotMediaToken.data,
additionalImages: referenceImages.map((image) => image.mediaToken),
});
if (response.success === true) {
console.error("Successfully enqueued image generation.");
toast.success("Please wait while we process your image.");
if (response.data) {
console.log("Response:", response.data);
setJobToken(response.data);
addJobToken(response.data);
}
return;
} else {
console.log("Image generation Not Successful.");
toast.error("Failed to generate image. Please try again.");
toast.error("Failed to enqueue image generation. Please try again.");
}
}
} catch (error) {
@@ -63,12 +63,11 @@ export const ToolbarMain = ({
buttonProps={buttonProps.SELECT}
tooltip="Select"
/>
{/* <ToolbarButton
<ToolbarButton
icon={faText}
buttonProps={buttonProps.ADD_TEXT}
tooltip="Add Text"
/> */}
/>
<Popover className="relative">
{({ close }) => (
<>
@@ -41,9 +41,9 @@ export const ToolbarTopLeft = () => {
)}
>
<img
src="/brand/mira-logo.png"
src="/brand/artcraft-logo.png"
alt="logo"
className="h-[30px] select-none pb-1"
className="h-[28px] select-none"
/>
<TabSelector
@@ -1,4 +1,5 @@
import { twMerge } from "tailwind-merge";
import { Login } from "~/app/pages/login";
import { Activity } from "~/components/ui/Activity/Activity";
// import { paperWrapperStyles } from "~/components/styles";
// import { faPlus, faQuestion } from "@fortawesome/pro-solid-svg-icons";
@@ -0,0 +1,73 @@
import { useRef, useCallback, useContext, ReactNode, useEffect } from "react";
import { EngineType } from "~/KonvaApp";
interface UndoRedoProps {
children: ReactNode;
engine: EngineType | null;
}
/**
* A transparent component that provides undo/redo functionality
* with keyboard shortcuts for macOS (Command+Z, Command+Shift+Z)
*/
export const UndoRedo = ({
engine,
children,
}: UndoRedoProps) => {
console.log("UndoRedo");
// We don't need a separate ref since we want to track the prop changes
// Create the event handler with useCallback
const handleKeyDown = useCallback((e: KeyboardEvent) => {
if (!engine) {
return;
}
// Don't capture events when user is editing text
const target = e.target as HTMLElement;
const isEditingText = target.tagName === 'INPUT' ||
target.tagName === 'TEXTAREA' ||
target.isContentEditable;
// Command+Z for Undo
// Ctrl+Shift+Z for Redo (Windows/Linux alternative)
// Windows/Linux: Ctrl+Shift+Z for Redo
if (!navigator.userAgent.includes('Mac') && e.ctrlKey && e.shiftKey && e.key === 'z') {
console.log("Ctrl+Shift+Z pressed: Redo (Windows/Linux)");
e.preventDefault();
engine.undoStackManager.redo();
}
// Delete key handling - only if not editing text
if (!isEditingText && (e.key === 'Delete' || e.key === 'Backspace')) {
console.log("Delete/Backspace pressed: Delete selected nodes");
e.preventDefault();
engine.commandManager.deleteNodes();
}
// macOS: Command+Z for Undo
if (navigator.userAgent.includes('Mac') && e.metaKey && e.key === 'z' && !e.shiftKey) {
console.log("Command+Z pressed: Undo");
e.preventDefault();
engine.undoStackManager.undo();
}
// macOS: Command+Shift+Z for Redo
if (navigator.userAgent.includes('Mac') && e.metaKey && e.key === 'z' && e.shiftKey) {
console.log("Command+Shift+Z pressed: Redo");
e.preventDefault();
engine.undoStackManager.redo();
engine.realTimeDrawEngine.render();
}
}, [engine]);
// Actually set up the event listeners using useEffect
useEffect(() => {
window.addEventListener('keydown', handleKeyDown);
// Cleanup function to remove the event listener
return () => {
window.removeEventListener('keydown', handleKeyDown);
};
}, [handleKeyDown]); // Depend on the memoized handler
return children;
};
@@ -6,6 +6,8 @@ import { CompletedCard } from "./CompletedCard";
import { InProgressCard } from "./InProgressCard";
import { useJobContext } from "~/components/JobContext";
import { Api } from "~/KonvaApp/Api";
import { toast } from "sonner";
// Dummy data types
interface CompletedItem {
token: string;
@@ -65,90 +67,72 @@ const dummyCompletedItems: CompletedItem[] = [
// ];
export function Activity() {
const { jobToken, setJobToken } = useJobContext();
const { jobTokens, addJobToken, removeJobToken, clearJobTokens } =
useJobContext();
const [completedItems, setCompletedItems] = useState<CompletedItem[]>([]);
const [jobs, setJobs] = useState<ActiveJob[]>([]);
const [loading, setLoading] = useState(true);
// Dummy polling function
useEffect(() => {
const pollData = async () => {
if (jobToken) {
console.log("Current Job Token:", jobToken);
if (jobTokens.length > 0) {
console.log("Current Job Tokens:", jobTokens);
const api = new Api();
const job = await api.pollJobSession(jobToken);
console.log("Job response:", job);
if (job.success && job.data) {
const { status, progress_percentage } = job.data.status;
const { job_token } = job.data;
// Create a map to store the latest job data
const updatedJobs = new Map<string, ActiveJob>();
console.log("Job status:", status);
console.log("Progress percentage:", progress_percentage);
console.log("Job token:", job_token);
for (const jobToken of jobTokens) {
const job = await api.pollJobSession(jobToken);
// Update jobs with real job data
if (
status.toLowerCase() === "completed" ||
status.toLowerCase() === "failed"
) {
// If job is completed or failed, remove it from active jobs
setJobs((prevJobs) =>
prevJobs.filter((j) => j.job_token !== job_token),
);
if (job.success && job.data) {
const { status, progress_percentage } = job.data.status;
const { job_token } = job.data;
// Add to completed items if it was successful
if (
status.toLowerCase() === "completed" &&
job.data.result.generated_images
status.toLowerCase() === "complete_success" ||
status.toLowerCase() === "failed"
) {
const newCompletedItem: CompletedItem = {
id: job_token,
title: job.data.request.maybe_model_title || "Image Generation",
timestamp: new Date().toISOString(),
type: "image",
imageUrl: job.data.result.generated_images[0] || "",
};
// If job is complete_success or failed, we'll remove it later
removeJobToken(job_token);
setCompletedItems((prev) => [newCompletedItem, ...prev]);
// Add to complete_success items if it was successful
if (status.toLowerCase() === "complete_success") {
const newCompletedItem: CompletedItem = {
token: job_token,
maybe_title:
job.data.request.maybe_model_title || "Image Generation",
public_bucket_path: job.data.result.image_url || "",
updated_at: new Date().toISOString(),
};
toast.success("Done Processing Image");
setCompletedItems((prev) => [newCompletedItem, ...prev]);
}
} else {
// Store the job in our map
const updatedJob: ActiveJob = {
job_token,
request: {
maybe_model_title:
job.data.request.maybe_model_title || "Image Generation",
},
status: {
status: status.toUpperCase(),
progress_percentage: progress_percentage || 0,
},
};
updatedJobs.set(job_token, updatedJob);
}
} else {
// Update or add the job to the active jobs list
const updatedJob: ActiveJob = {
job_token,
request: {
maybe_model_title:
job.data.request.maybe_model_title || "Image Generation",
},
status: {
status: status.toUpperCase(),
progress_percentage: progress_percentage || 0,
},
};
setJobs((prevJobs) => {
const existingJobIndex = prevJobs.findIndex(
(j) => j.job_token === job_token,
);
if (existingJobIndex >= 0) {
// Update existing job
const newJobs = [...prevJobs];
newJobs[existingJobIndex] = updatedJob;
return newJobs;
} else {
// Add new job
return [updatedJob, ...prevJobs];
}
});
console.error("Failed to fetch job status:", job.errorMessage);
}
} else {
console.error("Failed to fetch job status:", job.errorMessage);
}
} else {
console.log("No Job Token available.");
}
if (jobs.length === 0) {
// Update jobs state with the deduplicated jobs
setJobs(Array.from(updatedJobs.values()));
}
if (jobTokens.length === 0) {
setJobs([]);
setLoading(false);
}
};
@@ -163,7 +147,7 @@ export function Activity() {
}, 5000);
return () => clearInterval(interval);
}, [jobToken]);
}, [jobTokens, removeJobToken]);
return (
<PopoverMenu
@@ -17,8 +17,10 @@ interface Props {
export function CompletedCard({ job }: Props) {
const [loadError, setLoadError] = useState(false);
const [isModalOpen, setIsModalOpen] = useState(false);
const imageUrl = job.public_bucket_path + "-thumb.gif";
const imageUrl = job.public_bucket_path;
const fullImageUrl = job.public_bucket_path;
console.log("I am hjere");
console.log(job);
return (
<>
@@ -87,19 +87,22 @@ export class EngineApi extends ApiManager {
}
public async enqueueImageGeneration({
disableSystemPrompt,
prompt,
snapshotMediaToken,
additionalImages,
}: {
disableSystemPrompt: boolean;
prompt: string;
snapshotMediaToken: string;
additionalImages?: string[];
}): Promise<ApiResponse<string>> {
const endpoint = `${this.ApiTargets.BaseApi}/v1/image_studio/prompt`;
// for now ...
const uuidIdempotencyToken = crypto.randomUUID(); // Generate a new UUID
const body = {
uuid_idempotency_token: uuidIdempotencyToken,
disable_system_prompt: disableSystemPrompt,
prompt,
snapshot_media_token: snapshotMediaToken, // Changed from scene_media_token to snapshot_media_token
additional_images: additionalImages,
@@ -108,6 +111,7 @@ export class EngineApi extends ApiManager {
const postResponse = await this.post<
{
uuid_idempotency_token: string;
disable_system_prompt: boolean;
prompt: string;
snapshot_media_token: string;
additional_images?: string[];
@@ -40,7 +40,7 @@ export class JobsApi extends ApiManager {
public ListRecentJobs(): Promise<ApiResponse<Job[]>> {
const endpoint = `${this.ApiTargets.BaseApi}/v1/jobs/session`;
return this.get<{
success: boolean;
jobs: Job[];
@@ -45,7 +45,7 @@ export const Button = ({
);
const className = twMerge(
"text-sm font-medium rounded-lg px-3 py-2 border border-[#363636] shadow-sm focus-visible:outline focus-visible:outline-2 focus-visible:outline-offset-2 transition-all duration-150 flex gap-2 items-center justify-center",
"text-sm font-medium rounded-lg px-3 py-2 border border-transparent shadow-sm focus-visible:outline focus-visible:outline-0 focus-visible:outline-offset-0 transition-all duration-150 flex gap-2 items-center justify-center",
getVariantClassNames(propsVariant),
propsClassName,
disabledClass,
@@ -15,7 +15,7 @@ export const CloseButton = ({
}: CloseButtonProps) => {
const sizeClasses = {
sm: "h-5 w-5 text-sm",
md: "h-7 w-7 text-lg",
md: "h-7 w-7 text-md",
lg: "h-9 w-9 text-xl",
};
@@ -0,0 +1,122 @@
import { CloseButton, Button } from "~/components";
import { Transition } from "@headlessui/react";
import { createPortal } from "react-dom";
import dayjs from "dayjs";
import { faDownToLine } from "@fortawesome/pro-solid-svg-icons";
interface LightboxModalProps {
isOpen: boolean;
onClose: () => void;
imageUrl?: string | null;
imageAlt?: string;
onImageError?: () => void;
title?: string;
createdAt?: string;
additionalInfo?: React.ReactNode;
downloadUrl?: string;
}
export function LightboxModal({
isOpen,
onClose,
imageUrl,
imageAlt = "",
onImageError,
title,
createdAt,
additionalInfo,
downloadUrl,
}: LightboxModalProps) {
return createPortal(
<Transition appear show={isOpen}>
<div className="fixed inset-0 z-[100]">
<Transition.Child
enter="ease-out duration-300"
enterFrom="opacity-0"
enterTo="opacity-100"
leave="ease-in duration-200"
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
<div
className="fixed inset-0 cursor-pointer bg-black/60"
onClick={onClose}
/>
</Transition.Child>
<div
className="fixed inset-0 flex items-center justify-center p-4"
onClick={onClose}
>
<Transition.Child
enter="ease-out duration-300"
enterFrom="opacity-0 scale-95"
enterTo="opacity-100 scale-100"
leave="ease-in duration-200"
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95"
>
<div
className="relative h-[90vh] w-[80vw] rounded-xl bg-[#2C2C2C]"
onClick={(e) => e.stopPropagation()}
>
<CloseButton
onClick={onClose}
className="absolute right-4 top-4 z-10"
/>
<div className="grid h-full grid-cols-3 gap-6">
<div className="col-span-2 flex h-full items-center justify-center overflow-hidden rounded-l-xl bg-black/40">
{!imageUrl ? (
<div className="flex h-full w-full items-center justify-center bg-gray-800">
<span className="text-white/60">Image not available</span>
</div>
) : (
<img
src={imageUrl}
alt={imageAlt}
className="h-full w-full object-contain"
onError={onImageError}
crossOrigin="anonymous"
referrerPolicy="no-referrer"
/>
)}
</div>
<div className="flex h-full flex-col py-5 pe-5">
<div className="flex-1 space-y-4">
<div className="text-xl font-medium">
{title || "Image Generation"}
</div>
{createdAt && (
<div className="text-sm text-white/60">
Created:{" "}
{dayjs(createdAt).format("MMM D, YYYY HH:mm:ss")}
</div>
)}
{additionalInfo}
</div>
{downloadUrl && (
<div className="flex justify-end">
<a
href={downloadUrl}
download
onClick={(e) => e.stopPropagation()}
className="no-underline"
>
<Button
icon={faDownToLine}
onClick={(e) => e.stopPropagation()}
>
Download
</Button>
</a>
</div>
)}
</div>
</div>
</div>
</Transition.Child>
</div>
</div>
</Transition>,
document.body,
);
}
@@ -17,6 +17,7 @@ interface PopoverMenuProps {
onAdd?: () => void;
triggerIcon?: ReactNode;
showAddButton?: boolean;
disableAddButton?: boolean;
showIconsInList?: boolean;
mode?: "default" | "toggle" | "button";
triggerLabel?: string;
@@ -36,6 +37,7 @@ export const PopoverMenu = ({
onAdd,
triggerIcon,
showAddButton = false,
disableAddButton = false,
showIconsInList = false,
mode = "default",
triggerLabel,
@@ -84,9 +86,15 @@ export const PopoverMenu = ({
{({ close }) => (
<>
<Popover.Button className={className}>
{triggerIcon}
{mode === "toggle" && selectedItem ? selectedItem.label : null}
{mode === "default" && triggerLabel ? triggerLabel : null}
<>
{triggerIcon}
{mode === "toggle" && selectedItem ? (
<span className="truncate">{selectedItem.label}</span>
) : null}
{mode === "default" && triggerLabel ? (
<span className="truncate">{triggerLabel}</span>
) : null}
</>
</Popover.Button>
<div
@@ -164,9 +172,9 @@ export const PopoverMenu = ({
variant="secondary"
disabled={item.disabled}
>
<div className="flex items-center gap-2">
<div className="flex items-center gap-2 truncate">
{showIconsInList && item.icon}
{item.label}
<span className="truncate">{item.label}</span>
</div>
{mode === "toggle" && (
<input
@@ -180,8 +188,14 @@ export const PopoverMenu = ({
{showAddButton && onAdd && (
<Button
variant="secondary"
className="mb-0.5 mt-2 border-none bg-[#7B7B84] py-1 hover:bg-[#8c8c96]"
className={twMerge(
"mb-0.5 mt-2 border-none py-1",
disableAddButton
? "cursor-not-allowed bg-[#7B7B84]/50 opacity-50"
: "bg-[#7B7B84] hover:bg-[#8c8c96]",
)}
onClick={onAdd}
disabled={disableAddButton}
>
+ Add
</Button>
@@ -137,7 +137,7 @@ export const Tooltip = ({
transitionTimingFunction: "ease-out",
}}
className={twMerge(
"pointer-events-none absolute z-10 w-max rounded-lg bg-[#5F5F68] px-2.5 py-1.5 text-sm font-medium text-white shadow-xl",
"pointer-events-none absolute z-10 w-max rounded-lg bg-[#5F5F68] px-2.5 py-1.5 text-[13px] font-medium text-white shadow-xl",
className ? className : "",
)}
>
@@ -27,18 +27,18 @@ export const TopBar = ({ pageName }: Props) => {
return (
<>
<header className="fixed left-0 top-0 z-30 w-full border-b border-ui-panel-border bg-ui-background">
<header className="fixed left-0 top-0 z-[60] w-full border-b border-white/5 bg-ui-background">
<nav
className="mx-auto grid h-[64px] w-screen grid-cols-3 items-center justify-between p-3"
aria-label="Global"
className="mx-auto grid h-[56px] w-screen grid-cols-3 items-center justify-between px-3"
aria-label="navigation"
>
<div className="flex gap-4">
<a href="/" className="">
<span className="sr-only">Storyteller.ai</span>
<a href="/">
<span className="sr-only">ArtCraft</span>
<img
className="h-[39px] w-auto pb-[3px]"
src="/resources/images/Storyteller-Logo-1.png"
alt="Logo StoryTeller.ai"
className="h-[28px] w-auto"
src="/resources/images/artcraft-logo-3.png"
alt="Logo ArtCraft"
/>
</a>
{!isEditorPath(currentLocation) && (
@@ -52,7 +52,7 @@ export const TopBar = ({ pageName }: Props) => {
<SceneTitleInput pageName={pageName} />
</div>
<div className="flex justify-end gap-5">
<div className="flex justify-end gap-3.5">
<div className="flex gap-2">
<Button
variant="secondary"
+1
View File
@@ -33,4 +33,5 @@ export interface JobStatusResponse {
maybe_first_started_at: string;
requires_keepalive: boolean;
status: JobStatus;
progress_percentage: number;
}
@@ -197,8 +197,6 @@ class Editor {
processingHasFailed: boolean;
stats: Stats;
cameras: Map<string, THREE.PerspectiveCamera> = new Map();
constructor() {
this.processingHasFailed = false;
console.log(
@@ -329,34 +327,6 @@ class Editor {
if (this.engineFrameBuffers.frameWorkerManager) {
this.engineFrameBuffers.frameWorkerManager.type = imageFormat;
}
// Initialize cameras from signals
cameras.value.forEach((camData) => {
const camera = new THREE.PerspectiveCamera(
camData.fov,
this.getRenderDimensions().aspectRatio,
0.01,
200,
);
camera.position.set(
camData.position.x,
camData.position.y,
camData.position.z,
);
camera.rotation.set(
camData.rotation.x,
camData.rotation.y,
camData.rotation.z,
);
this.cameras.set(camData.id, camera);
});
// Set initial camera
this.camera = this.cameras.get("main") || null;
this.render_camera = this.camera;
// Subscribe to camera changes
this.setupCameraSignalHandlers();
}
// Add helper method to convert focal length to FOV
@@ -365,99 +335,6 @@ class Editor {
return 2 * Math.atan(sensorHeight / (2 * focalLength)) * (180 / Math.PI);
}
setupCameraSignalHandlers() {
// Handle camera selection changes
selectedCameraId.subscribe((id) => {
const newCamera = this.cameras.get(id);
if (newCamera) {
this.camera = newCamera;
this.render_camera = newCamera;
// Update camera properties in signals
const camData = cameras.value.find((c) => c.id === id);
if (camData) {
// Convert focal length to FOV
const fov = this.focalLengthToFov(camData.focalLength);
newCamera.fov = fov;
newCamera.updateProjectionMatrix();
this.renderScene(); // Force a render update
}
}
});
// Handle camera updates
cameras.subscribe((newCameras) => {
newCameras.forEach((camData) => {
let camera = this.cameras.get(camData.id);
// Create new camera if it doesn't exist
if (!camera) {
const fov = this.focalLengthToFov(camData.focalLength);
camera = new THREE.PerspectiveCamera(
fov,
this.getRenderDimensions().aspectRatio,
0.01,
200,
);
this.cameras.set(camData.id, camera);
}
// Update camera properties
camera.position.set(
camData.position.x,
camData.position.y,
camData.position.z,
);
camera.rotation.set(
camData.rotation.x,
camData.rotation.y,
camData.rotation.z,
);
// Update FOV based on focal length
const fov = this.focalLengthToFov(camData.focalLength);
camera.fov = fov;
camera.updateProjectionMatrix();
// Force a render update if this is the active camera
if (camData.id === selectedCameraId.value) {
this.renderScene();
}
});
// Clean up deleted cameras
const currentIds = new Set(newCameras.map((c) => c.id));
for (const [id] of this.cameras.entries()) {
if (!currentIds.has(id) && id !== "main") {
this.cameras.delete(id);
}
}
});
}
// Update camera position in signals when it changes
updateCameraPosition() {
if (this.camera && selectedCameraId.value) {
const pos = this.camera.position;
const rot = this.camera.rotation;
const camData = cameras.value.find(
(c) => c.id === selectedCameraId.value,
);
if (camData) {
cameras.value = cameras.value.map((c) =>
c.id === selectedCameraId.value
? {
...c,
position: { x: pos.x, y: pos.y, z: pos.z },
rotation: { x: rot.x, y: rot.y, z: rot.z },
}
: c,
);
}
}
}
getRenderDimensions() {
switch (this.render_camera_aspect_ratio) {
case CameraAspectRatio.HORIZONTAL_16_9: {
@@ -590,24 +467,52 @@ class Editor {
const width = this.container.offsetWidth;
const height = this.container.offsetHeight;
// Sets up camera and base position.
this.camera = new THREE.PerspectiveCamera(70, width / height, 0.01, 200);
this.camera.position.z = 2.5;
this.camera.position.y = 2.5;
this.camera.position.x = -2.5;
this.camera.lookAt(0, 0, 0);
// Sets up camera and base position using camera configurations from camera.ts
const mainCameraConfig = cameras.value.find((cam) => cam.id === "main");
if (mainCameraConfig) {
this.camera = new THREE.PerspectiveCamera(
this.focalLengthToFov(mainCameraConfig.focalLength),
width / height,
0.01,
200,
);
this.camera.position.set(
mainCameraConfig.position.x,
mainCameraConfig.position.y,
mainCameraConfig.position.z,
);
this.camera.lookAt(
mainCameraConfig.lookAt.x,
mainCameraConfig.lookAt.y,
mainCameraConfig.lookAt.z,
);
}
this.camera.layers.enable(0);
this.camera.layers.enable(1); // This camera does not see this layer
this.camera.layers.enable(1);
this.timeline.camera = this.camera;
this.render_camera = new THREE.PerspectiveCamera(
70,
width / height,
0.01,
200,
);
const otherCameras = cameras.value.filter((cam) => cam.id !== "main");
if (otherCameras.length > 0) {
const renderCameraConfig = otherCameras[0];
this.render_camera = new THREE.PerspectiveCamera(
this.focalLengthToFov(renderCameraConfig.focalLength),
width / height,
0.01,
200,
);
this.render_camera.position.set(
renderCameraConfig.position.x,
renderCameraConfig.position.y,
renderCameraConfig.position.z,
);
this.render_camera.lookAt(
renderCameraConfig.lookAt.x,
renderCameraConfig.lookAt.y,
renderCameraConfig.lookAt.z,
);
}
this.render_camera.layers.disable(1); // This camera does not see this layer );
@@ -801,6 +706,18 @@ class Editor {
const wasGridVisible = gridVisibility.value;
gridVisibility.value = false;
// Store and hide transform controls
const wasControlVisible = this.control?.visible ?? false;
if (this.control) {
this.control.visible = false;
}
// Store and disable outline pass
const wasOutlineEnabled = this.outlinePass?.enabled ?? false;
if (this.outlinePass) {
this.outlinePass.enabled = false;
}
// High quality dimensions for each aspect ratio
let targetWidth: number;
let targetHeight: number;
@@ -894,6 +811,16 @@ class Editor {
// Restore grid visibility
gridVisibility.value = wasGridVisible;
// Restore transform controls visibility
if (this.control) {
this.control.visible = wasControlVisible;
}
// Restore outline pass
if (this.outlinePass) {
this.outlinePass.enabled = wasOutlineEnabled;
}
if (shouldDownload) {
const link = document.createElement("a");
link.download = "scene-snapshot.png";
@@ -70,7 +70,7 @@ class FreeCam extends EventDispatcher {
}
keydown(event: KeyboardEvent) {
if (event.altKey || !this.enabled) {
if (!this.enabled) {
return;
}
@@ -235,12 +235,18 @@ class FreeCam extends EventDispatcher {
const pos = this.object.position;
const rot = this.object.rotation;
// Calculate lookAt point based on camera's forward direction
const lookAtPoint = new Vector3(0, 0, -1);
lookAtPoint.applyQuaternion(this.object.quaternion);
lookAtPoint.add(pos);
cameras.value = cameras.value.map((cam) =>
cam.id === selectedCameraId.value
? {
...cam,
position: { x: pos.x, y: pos.y, z: pos.z },
rotation: { x: rot.x, y: rot.y, z: rot.z },
lookAt: { x: lookAtPoint.x, y: lookAtPoint.y, z: lookAtPoint.z },
}
: cam,
);
@@ -17,7 +17,11 @@ import { SceneManager, SceneObject } from "./scene_manager_api";
import { FreeCam } from "./free_cam";
import Stats from "three/examples/jsm/libs/stats.module.js";
import { FKHelper } from "./KinHelpers/FKHelper";
import { selectedMode } from "../signals/selectedMode";
import {
selectedMode,
poseMode,
showPoseControls,
} from "../signals/selectedMode";
export enum KinMode {
FK,
@@ -122,6 +126,10 @@ export class MouseControls {
scene: this.scene,
onDragChange: this.onFKControlsDragging.bind(this),
});
// Expose handlePoseModeChange to window
// eslint-disable-next-line @typescript-eslint/no-explicit-any
(window as any).__mouseControls = this;
}
onFKControlsDragging(dragging: boolean) {
@@ -185,7 +193,11 @@ export class MouseControls {
this.setSelected(this.selected);
this.publishSelect();
// this.editor.update_properties()
if (currentObject.userData.isCharacter) {
showPoseControls.value = true;
}
// Normal selection behavior
if (currentObject.userData["locked"] !== true && this.control) {
this.scene.add(this.control);
this.control.attach(currentObject);
@@ -235,6 +247,9 @@ export class MouseControls {
this.fkHelper.clear();
this.kinMode = KinMode.NONE;
this.reattachTransformControls();
if (poseMode.value === "pose") {
poseMode.value = "select";
}
console.log("FK mode off");
return;
}
@@ -255,6 +270,9 @@ export class MouseControls {
this.hideTransformControls();
this.kinMode = KinMode.FK;
this.fkHelper.setTarget(this.selected[0]);
if (poseMode.value === "select") {
poseMode.value = "pose";
}
console.log("FK mode on");
return;
}
@@ -305,41 +323,53 @@ export class MouseControls {
if ((event.ctrlKey || event.metaKey) && !this.isProcessing) {
if (event.key === "Z" || event.key === "z") {
if (event.shiftKey) {
// redo
this.isProcessing = true;
await this.sceneManager?.redo();
this.isProcessing = false;
} else {
// undo
this.isProcessing = true;
await this.sceneManager?.undo();
this.isProcessing = false;
}
return;
// redo
this.isProcessing = true;
await this.sceneManager?.redo();
this.isProcessing = false;
} else if (event.key === "c") {
// redo
this.isProcessing = true;
await this.sceneManager?.copy();
this.isProcessing = false;
return;
} else if (event.key === "v") {
// redo
this.isProcessing = true;
await this.sceneManager?.paste();
this.isProcessing = false;
return;
} else if (event.key === "0") {
// Stats Menu
this.enable_stats();
}
}
// Prevent browser shortcuts for Alt combinations
if (
event.altKey &&
(event.key === "Alt" || event.key.toLowerCase() === "d")
) {
event.preventDefault();
event.stopPropagation();
}
if (event.shiftKey) {
this.cameraViewControls.movementSpeed = 4;
} else if (event.altKey) {
this.cameraViewControls.movementSpeed = 0.1;
} else {
this.cameraViewControls.movementSpeed = 1.15;
}
if (event.key === "Escape") {
if (poseMode.value === "pose") {
this.toggleFKMode();
return;
} else if (this.selected && this.selected.length > 0) {
this.removeTransformControls();
hideObjectPanel();
showPoseControls.value = false;
}
}
}
// Sets new mouse location usually used in raycasts.
@@ -443,7 +473,8 @@ export class MouseControls {
this.selected = [];
this.setSelected(this.selected);
this.removeTransformControls();
//hideObjectPanel();
hideObjectPanel();
showPoseControls.value = false;
}
if (this.sceneManager) {
@@ -11,6 +11,8 @@ import {
} from "../signals";
import Editor from "./editor";
import { getArtStyle } from "~/enums";
import { cameras, selectedCameraId, Camera } from "../signals/camera";
export type EditorInitializeConfig = {
sceneToken: string;
};
@@ -98,6 +100,16 @@ export class SaveManager {
);
const timeline_json = await proxyTimeline.saveToJson();
// Save all cameras data
const camerasData = cameras.value.map((cam: Camera) => ({
id: cam.id,
label: cam.label,
focalLength: cam.focalLength,
position: cam.position,
rotation: cam.rotation,
lookAt: cam.lookAt,
}));
const save_data = {
version: this.editor.version,
scene: scene_json,
@@ -108,6 +120,9 @@ export class SaveManager {
position: this.editor.camera?.position,
rotation: this.editor.camera?.rotation,
},
// Add cameras array and selected camera
cameras: camerasData,
selectedCameraId: selectedCameraId.value,
};
// take json scene and figure out checksum
const jsonString = JSON.stringify(save_data);
@@ -146,6 +161,16 @@ export class SaveManager {
);
const timeline_json = await proxyTimeline.saveToJson();
// Save all cameras data
const camerasData = cameras.value.map((cam: Camera) => ({
id: cam.id,
label: cam.label,
focalLength: cam.focalLength,
position: cam.position,
rotation: cam.rotation,
lookAt: cam.lookAt,
}));
const save_data = {
version: this.editor.version,
scene: scene_json,
@@ -156,6 +181,9 @@ export class SaveManager {
position: this.editor.camera?.position,
rotation: this.editor.camera?.rotation,
},
// Add cameras array and selected camera
cameras: camerasData,
selectedCameraId: selectedCameraId.value,
};
// TODO turn scene information into and object ...
@@ -218,6 +246,23 @@ export class SaveManager {
this.editor.camera.rotation.copy(camera_rotation);
}
// Restore cameras data
if (scene_json.cameras) {
cameras.value = scene_json.cameras.map((cam: Camera) => ({
id: cam.id,
label: cam.label,
focalLength: cam.focalLength,
position: cam.position,
rotation: cam.rotation,
lookAt: cam.lookAt,
}));
}
// Restore selected camera
if (scene_json.selectedCameraId) {
selectedCameraId.value = scene_json.selectedCameraId;
}
// For Remixing Scenes.
// this calls the signal function to propagate the data to the UI
restoreSceneGenerationMetadata(scene_json);
@@ -17,6 +17,7 @@ import { GetCdnOrigin } from "~/api/GetCdnOrigin";
import { GetFrontendEnvironment } from "~/Classes/GetFrontendEnvironment";
import { gridVisibility } from "../signals/engine";
import { InfiniteGridHelper } from "./InfiniteGridHelper";
import { cameras, selectedCameraId } from "../signals/camera";
class Scene {
name: string;
@@ -86,7 +87,7 @@ class Scene {
this._createGrid();
this._create_base_lighting();
this._create_skybox();
this._create_camera_obj();
// this._create_camera_obj();
this.helper = new MMDAnimationHelper({ afterglow: 0.0 });
this.scene.userData["helper"] = this.helper;
@@ -110,7 +111,7 @@ class Scene {
this._createGrid();
this._create_base_lighting();
this._create_skybox();
this._create_camera_obj();
// this._create_camera_obj();
}
async instantiate(
@@ -370,31 +371,45 @@ class Scene {
}
async _create_camera_obj() {
const camera_position = new THREE.Vector3(0, 0.6, 1.5);
cameras.value.forEach((cameraConfig) => {
const camera_position = new THREE.Vector3(
cameraConfig.position.x,
cameraConfig.position.y,
cameraConfig.position.z,
);
// TODO(bt,2025-02-10): Make the local dev assets more configurable or seedable.
const camera_id = GetFrontendEnvironment().getIsLocalDev()
? "m_4djqzq89mbn6sdbbqm25ks6drhr55k" // Local development
: "m_cxh4asqhapdz10j880755dg4yevshb"; // Production
// TODO(bt,2025-02-10): Make the local dev assets more configurable or seedable.
const camera_id = GetFrontendEnvironment().getIsLocalDev()
? "m_cxh4asqhapdz10j880755dg4yevshb" // Local development
: "m_cxh4asqhapdz10j880755dg4yevshb"; // Production
const camera_obj = await this.loadGlbWithPlaceholder(
camera_id,
"Camera",
false,
camera_position,
);
camera_obj.userData["name"] = this.camera_name;
camera_obj.name = this.camera_name;
camera_obj.position.set(
camera_position.x,
camera_position.y,
camera_position.z,
);
camera_obj.layers.set(1);
camera_obj.children.forEach((child) => {
child.layers.set(1);
this.loadGlbWithPlaceholder(
camera_id,
"Camera",
false,
camera_position,
).then((camera_obj) => {
camera_obj.userData["name"] = cameraConfig.label;
camera_obj.name = cameraConfig.label;
camera_obj.position.set(
camera_position.x,
camera_position.y,
camera_position.z,
);
camera_obj.layers.set(1);
camera_obj.children.forEach((child) => {
child.layers.set(1);
});
this.scene.add(camera_obj);
if (cameraConfig.id === selectedCameraId.value) {
camera_obj.visible = false;
camera_obj.layers.disableAll(); // Make the active camera not touchable
} else {
camera_obj.visible = true;
}
});
});
this.scene.add(camera_obj);
}
renderMode(enabled: boolean = true) {
@@ -1,18 +1,16 @@
import React, { useEffect } from "react";
import { useSignals } from "@preact/signals-react/runtime";
import { LoadingDots, Toaster, TopBar } from "~/components";
import { LoadingDots, TopBar } from "~/components";
import { Controls3D } from "./comps/Controls3D";
import { ControlsTopButtons } from "./comps/ControlsTopButtons";
import { ControlPanelSceneObject } from "./comps/ControlPanelSceneObject";
import { PreviewEngineCamera } from "./comps/PreviewEngineCamera";
import { PreviewFrameImage } from "./comps/PreviewFrameImage";
import { pageHeight, pageWidth } from "~/signals";
import { PoseModeSelector } from "./comps/PoseModeSelector";
import {
timelineHeight,
sidePanelWidth,
sidePanelVisible,
dndSidePanelWidth,
dndTimelineHeight,
editorLoader,
cameraAspectRatio,
@@ -24,11 +22,13 @@ import { Outliner } from "./comps/Outliner";
import { CameraAspectRatio } from "./enums";
import { PromptBox } from "./comps/PromptBox";
import { OnboardingHelper } from "./comps/OnboardingHelper";
import { focalLengthDragging } from "./signals/camera";
import { FocalLengthDisplay } from "./comps/FocalLengthDisplay/FocalLengthDisplay";
// import { api } from "@storyteller/api";
// import { AnalyticsApi } from "@storyteller/api";
export const PageEditor = () => {
useSignals();
//console.log(api());
//To prevent the click event from propagating to the canvas: TODO: HANDLE THIS BETTER?
const handleOverlayClick = (event: React.MouseEvent<HTMLDivElement>) => {
event.stopPropagation();
@@ -42,20 +42,13 @@ export const PageEditor = () => {
};
}, []);
const dndWidth =
dndSidePanelWidth.value > -1
? dndSidePanelWidth.value
: sidePanelWidth.value;
const width = sidePanelVisible.value
? pageWidth.value - dndWidth - 84
: pageWidth.value - 84;
const height =
dndTimelineHeight.value > -1
? pageHeight.value - dndTimelineHeight.value - 64
: pageHeight.value - timelineHeight.value - 64;
? pageHeight.value - dndTimelineHeight.value - 56
: pageHeight.value - timelineHeight.value - 56;
const getScale = () => {
const height = pageHeight.value - timelineHeight.value - 64;
const height = pageHeight.value - timelineHeight.value - 56;
const scaleHeight = height < 610 ? height / 610 : 1;
if (
@@ -97,19 +90,15 @@ export const PageEditor = () => {
>
<div className="relative w-full overflow-hidden bg-transparent">
<SceneContainer>
<Toaster />
<EditorCanvas />
<PreviewFrameImage />
</SceneContainer>
{/* Focal Length Display */}
{focalLengthDragging.value.isDragging && (
<div className="absolute left-1/2 top-16 -translate-x-1/2 transform">
<div className="glass rounded-xl px-6 py-3 text-center text-3xl font-bold text-white">
{focalLengthDragging.value.focalLength}mm
</div>
</div>
)}
<FocalLengthDisplay />
{/* Pose Mode Selector */}
<PoseModeSelector />
{/* Top controls */}
<div
@@ -6,9 +6,8 @@ import { useBackgroundLoadingMedia } from "~/hooks/useBackgroundLoadingMedia";
import { useQueueHandler } from "./hooks/useQueueHandler";
import { ErrorDialog, LoadingDots } from "~/components";
import { GenerateModals } from "~/pages/PageEnigma/comps/GenerateModals/GenerateModals";
import { Toaster } from "@storyteller/ui-toaster";
import { EditorLoadingBar } from "./comps/EditorLoadingBar";
import { Wizard } from "~/pages/PageEnigma/Wizard/Wizard";
import { useSignals } from "@preact/signals-react/runtime";
import { useEffect, useState } from "react";
@@ -16,6 +15,13 @@ import * as gpu from "detect-gpu";
import { UsersApi } from "~/Classes/ApiManager";
import { PageEnigmaComponent } from "~/pages/PageEnigma/PageEnigmaComponent";
import { TurnOnGpu } from "~/pages/PageEnigma/TurnOnGpu";
import { PrecisionSelector } from "./comps/PrecisionSelector/PrecisionSelector";
import {
precisionSelectedValue,
precisionSelectorMenuCoords,
precisionSelectorValues,
showPrecisionSelector,
} from "./signals/precisionSelectorMenu";
export const PageEnigma = ({ sceneToken }: { sceneToken?: string }) => {
useSignals();
@@ -71,7 +77,7 @@ export const PageEnigma = ({ sceneToken }: { sceneToken?: string }) => {
case "apple gpu (Apple GPU)":
// TODO(bt,2025-04-08): We may want to disable this heuristic.
// We're getting lack of hardware acceleration using Tauri on Mac and Linux.
isValid = true;
isValid = true;
break;
default:
break;
@@ -93,11 +99,18 @@ export const PageEnigma = ({ sceneToken }: { sceneToken?: string }) => {
<EngineProvider sceneToken={sceneToken}>
<PageEnigmaComponent />
<DragComponent />
<PrecisionSelector
showSignal={showPrecisionSelector}
coordSignal={precisionSelectorMenuCoords}
valuesSignal={precisionSelectorValues}
selectedValueSignal={precisionSelectedValue}
/>
<GenerateModals />
<ErrorDialog />
<Wizard />
</EngineProvider>
<EditorLoadingBar />
<Toaster offsetTop={70} offsetRight={12} />
</TrackProvider>
);
};
@@ -33,6 +33,8 @@ export enum toEngineActions {
CHANGE_CAMERA_ASPECT_RATIO = "change_camera_aspect_ratio",
// data=CamperaAspectRatio
CAMERA_CHANGED = "camera_changed",
UNMUTE = "unmute",
// data={version, type, group, object_uuid} if lipsync
// data={version, type, group} if global_audio
@@ -4,16 +4,16 @@ import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { PopoverItem } from "~/components/reusable/Popover";
import { Button, Input, Label, Tooltip } from "~/components";
import { SliderV2 } from "~/components/reusable/SliderV2/SliderV2";
import {
updateCamera,
focalLengthDragging,
} from "~/pages/PageEnigma/signals/camera";
import { focalLengthDragging } from "~/pages/PageEnigma/signals/camera";
import { useState, useEffect } from "react";
import { twMerge } from "tailwind-merge";
interface ExtendedPopoverItem extends PopoverItem {
id: string;
focalLength: number;
position: { x: number; y: number; z: number };
rotation: { x: number; y: number; z: number };
lookAt: { x: number; y: number; z: number };
}
interface CameraSettingsModalProps {
@@ -24,7 +24,7 @@ interface CameraSettingsModalProps {
onCameraFocalLengthChange: (id: string, value: number) => void;
onAddCamera: () => void;
selectedCameraId: string;
onSelectCamera: (id: string) => void;
handleCameraSelect: (selectedItem: PopoverItem) => void;
onDeleteCamera: (id: string) => void;
}
@@ -36,11 +36,20 @@ export const CameraSettingsModal = ({
onCameraFocalLengthChange,
onAddCamera,
selectedCameraId,
onSelectCamera,
handleCameraSelect,
onDeleteCamera,
}: CameraSettingsModalProps) => {
const selectedCamera = cameras.find((cam) => cam.id === selectedCameraId);
const [isDragging, setIsDragging] = useState(false);
const [tempCameraName, setTempCameraName] = useState<string>("");
const [editingCameraId, setEditingCameraId] = useState<string | null>(null);
useEffect(() => {
if (selectedCamera) {
setTempCameraName(selectedCamera.label);
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [selectedCamera?.id]);
useEffect(() => {
const handlePointerUp = () => {
@@ -67,16 +76,29 @@ export const CameraSettingsModal = ({
onCameraFocalLengthChange(id, value);
};
const handleNameBlur = () => {
if (selectedCamera && tempCameraName.trim() !== "") {
onCameraNameChange(selectedCamera.id, tempCameraName);
} else if (selectedCamera) {
setTempCameraName(selectedCamera.label);
}
setEditingCameraId(null);
};
const handleNameFocus = (id: string) => {
setEditingCameraId(id);
};
return (
<TransitionDialogue
isOpen={isOpen}
onClose={onClose}
className={twMerge(
"h-[500px] max-w-3xl transition-opacity duration-200",
isDragging ? "opacity-30 hover:opacity-20" : "opacity-100",
"h-[500px] max-w-3xl duration-200",
isDragging ? "opacity-10" : "opacity-100",
)}
backdropClassName={twMerge(
"transition-opacity duration-200",
"duration-200",
isDragging ? "opacity-0" : "opacity-100",
)}
childPadding={false}
@@ -99,14 +121,16 @@ export const CameraSettingsModal = ({
{cameras.map((camera) => (
<button
key={camera.id}
className={`w-full rounded-lg p-2 text-left transition-colors duration-100 hover:bg-[#63636B]/40 ${
className={`h-9 w-full rounded-lg p-2 text-left transition-colors duration-100 hover:bg-[#63636B]/40 ${
camera.id === selectedCameraId ? "bg-[#63636B]/40" : ""
}`}
onClick={() => onSelectCamera(camera.id)}
onClick={() => handleCameraSelect(camera)}
>
<div className="flex items-center gap-2.5 text-sm">
{camera.icon}
{camera.label}
{editingCameraId === camera.id
? tempCameraName
: camera.label}
</div>
</button>
))}
@@ -154,10 +178,11 @@ export const CameraSettingsModal = ({
<Input
id="camera-name"
type="text"
value={selectedCamera?.label || ""}
onChange={(e) =>
selectedCamera &&
onCameraNameChange(selectedCamera.id, e.target.value)
value={tempCameraName}
onChange={(e) => setTempCameraName(e.target.value)}
onBlur={handleNameBlur}
onFocus={() =>
selectedCamera && handleNameFocus(selectedCamera.id)
}
className="text-sm"
/>
@@ -22,6 +22,7 @@ import { sanitize } from "./utils/sanitize";
import { objectMismatch } from "~/pages/PageEnigma/comps/ControlPanelSceneObject/utils/objectMismatch";
import { XYZ } from "~/pages/PageEnigma/datastructures/common";
import { pageHeight } from "~/signals";
import { DraggablePrecisionMutator } from "./DraggablePrecisionMutator";
// TODO this will be useful later to fix the bug on leading zeros
// const formatNumber = (input: string): number => {
@@ -147,6 +148,15 @@ export const ControlPanelSceneObject = () => {
setLocalRotation(xyz);
};
const handleUniformScaleChange = (scale: number) => {
const updatedScaleValues: Record<string, string> = {};
updatedScaleValues.x = (parseFloat(localScale.x) + scale).toString();
updatedScaleValues.y = (parseFloat(localScale.y) + scale).toString();
updatedScaleValues.z = (parseFloat(localScale.z) + scale).toString();
handleScaleChange(updatedScaleValues);
}
const handleScaleChange = (xyz: Record<string, string>) => {
if (isInvalid(xyz)) {
setLocalScale(xyz);
@@ -164,7 +174,7 @@ export const ControlPanelSceneObject = () => {
};
const getScale = () => {
const height = pageHeight.value - timelineHeight.value - 64;
const height = pageHeight.value - timelineHeight.value - 56;
return height < 620 ? height / 620 : 1;
};
@@ -255,7 +265,9 @@ export const ControlPanelSceneObject = () => {
</div>
<div className="mb-1 flex flex-col gap-1">
<H5>Scale</H5>
<DraggablePrecisionMutator onChange={handleUniformScaleChange}>
<h5>Scale</h5>
</DraggablePrecisionMutator>
<InputVector
x={localScale.x.toString()}
y={localScale.y.toString()}
@@ -0,0 +1,101 @@
import { useCallback, useRef } from "react";
import { twMerge } from "tailwind-merge";
import { precisionSelectedValue, precisionSelectorMenuCoords, precisionSelectorValues, showPrecisionSelector } from "../../signals/precisionSelectorMenu";
const DEFAULT_PRECISIONS = [
10,
1,
0.1,
0.01,
0.001
]
export const DraggablePrecisionMutator = ({
precisions = DEFAULT_PRECISIONS,
className = "",
initialValue = 0,
onChange = () => { },
onCommit = () => { },
children
}: {
precisions?: number[],
className?: string,
initialValue?: number,
onChange?: (value: number) => void,
onCommit?: (value: number) => void,
children?: React.ReactNode
}) => {
const classes = twMerge(["cursor-ew-resize", className]);
const divRef = useRef<HTMLDivElement>(null);
const isDraggingRef = useRef(false);
const dragStartCoordsRef = useRef<{ x: number, y: number } | null>(null);
const localAdjustedValueRef = useRef(initialValue);
const handleMouseMove = useCallback((event: MouseEvent) => {
// Ignore mouse movement if we're not even dragging
// or if the menu is still open (we're selecting the precision)
if (!isDraggingRef.current || showPrecisionSelector.value) {
return;
}
event.stopPropagation();
// Record the initial drag coordinates if they haven't been set yet
if (!dragStartCoordsRef.current) {
dragStartCoordsRef.current = {
x: event.clientX,
y: event.clientY
};
return;
}
// Check how far the mouse has moved, and calculate the updated change
const distX = event.x - dragStartCoordsRef.current.x;
const scaledDistX = distX / 10;
const scaledPrecision = Math.round(scaledDistX) * precisionSelectedValue.value;
localAdjustedValueRef.current = initialValue + scaledPrecision;
// Update the listeners
onChange(localAdjustedValueRef.current);
}, [initialValue, onChange]);
const handleMouseUp = useCallback((event: MouseEvent) => {
event.stopPropagation();
showPrecisionSelector.value = false;
isDraggingRef.current = false;
dragStartCoordsRef.current = null;
// clean up the mouse listeners
document.removeEventListener("mousemove", handleMouseMove);
document.removeEventListener("mouseup", handleMouseUp);
// Call back the commit listener
onCommit(localAdjustedValueRef.current);
}, [handleMouseMove, onCommit]);
const handleMouseDown = useCallback((event: React.MouseEvent) => {
if (event.button !== 0) {
return;
}
event.preventDefault();
event.stopPropagation();
precisionSelectorMenuCoords.value = {
x: event.clientX,
y: event.clientY
};
precisionSelectorValues.value = precisions;
showPrecisionSelector.value = true;
isDraggingRef.current = true;
document.addEventListener("mousemove", handleMouseMove);
document.addEventListener("mouseup", handleMouseUp);
}, [handleMouseMove, handleMouseUp, precisions]);
return (
<div className={classes} ref={divRef} onMouseDown={handleMouseDown}>
{children}
</div>
)
}
@@ -0,0 +1,25 @@
import { Transition } from "@headlessui/react";
import { useSignals } from "@preact/signals-react/runtime";
import { focalLengthDragging } from "~/pages/PageEnigma/signals/camera";
export const FocalLengthDisplay = () => {
useSignals();
return (
<Transition
show={focalLengthDragging.value.isDragging}
enter="transition-opacity duration-200"
enterFrom="opacity-0"
enterTo="opacity-100"
leave="transition-opacity duration-200"
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
<div className="absolute left-1/2 top-16 z-10 -translate-x-1/2 transform">
<div className="glass rounded-xl px-5 py-2.5 text-center text-2xl font-bold text-white">
{focalLengthDragging.value.focalLength}mm
</div>
</div>
</Transition>
);
};
@@ -4,11 +4,185 @@ import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { PopoverMenu } from "~/components/reusable/Popover/Popover";
import { CompletedCard } from "./CompletedCard";
import { InProgressCard } from "./InProgressCard";
import { H3, P, Tooltip } from "~/components";
import { activeImageGenerationJobs, userMovies } from "~/signals";
import { H3, Tooltip } from "~/components";
import { useEffect, useState, useRef, useCallback } from "react";
import { JobsApi } from "~/Classes/ApiManager/JobsApi";
import { toast } from "@storyteller/ui-toaster";
import { ActiveJob } from "~/pages/PageEnigma/models";
// TODO ensure we de-dupe all this extra code.
interface CompletedItem {
token: string;
maybe_title: string;
public_bucket_path: string;
updated_at: string;
}
// {
// "job_token": "jinf_421nnr6rj8wy2sj0s11v9fns32s",
// "request": {
// "inference_category": "image_generation",
// "maybe_model_type": "image_gen_api",
// "maybe_model_token": null,
// "maybe_model_title": "Image Generation",
// "maybe_raw_inference_text": null,
// "maybe_style_name": null,
// "maybe_live_portrait_details": null,
// "maybe_lipsync_details": null
// },
// "status": {
// "status": "complete_success",
// "maybe_extra_status_description": null,
// "maybe_assigned_worker": "inference-job-4o-image-848c55d5dd-555kn",
// "maybe_assigned_cluster": "storyteller-eks-1",
// "maybe_first_started_at": "2025-04-15T04:41:01Z",
// "maybe_current_execution_duration_seconds": null,
// "attempt_count": 1,
// "requires_keepalive": false,
// "maybe_failure_category": null,
// "progress_percentage": 100
// },
// "maybe_result": {
// "entity_type": "media_file",
// "entity_token": "m_ytrjp1fcd5tb43ebcs3ewhq6y05tfb",
// "maybe_public_bucket_media_path": "/media/p/t/7/y/g/pt7yg2xfv4ynemssfv1mwb7yxe26e2nb/image_pt7yg2xfv4ynemssfv1mwb7yxe26e2nb.png",
// "media_links": {
// "cdn_url": "https://cdn-2.fakeyou.com/media/p/t/7/y/g/pt7yg2xfv4ynemssfv1mwb7yxe26e2nb/image_pt7yg2xfv4ynemssfv1mwb7yxe26e2nb.png",
// "maybe_thumbnail_template": "https://cdn-2.fakeyou.com/cdn-cgi/image/width={WIDTH},quality=95/media/p/t/7/y/g/pt7yg2xfv4ynemssfv1mwb7yxe26e2nb/image_pt7yg2xfv4ynemssfv1mwb7yxe26e2nb.png",
// "maybe_video_previews": null
// },
// "maybe_successfully_completed_at": "2025-04-15T04:42:14Z"
// },
// "created_at": "2025-04-15T04:41:01Z",
// "updated_at": "2025-04-15T04:42:14Z"
// }
export function Activity() {
useSignals();
const [completedItems, setCompletedItems] = useState<CompletedItem[]>([]);
const [jobs, setJobs] = useState<ActiveJob[]>([]);
const processedJobs = useRef(new Set<string>());
const isFirstLoad = useRef(true);
const toastTimeoutRef = useRef<NodeJS.Timeout | null>(null);
// Debounced toast function
const showToast = useCallback((items: CompletedItem[]) => {
if (toastTimeoutRef.current) {
clearTimeout(toastTimeoutRef.current);
}
toastTimeoutRef.current = setTimeout(() => {
const count = items.length;
toast.success(
count === 1
? `${items[0].maybe_title} completed successfully`
: `${count} images completed successfully`,
);
toastTimeoutRef.current = null;
}, 100); // Small delay to catch potential double calls
}, []);
useEffect(() => {
let mounted = true;
async function fetchJobs() {
try {
const jobsApi = new JobsApi();
const response = await jobsApi.ListRecentJobs();
if (!mounted || !response.success || !response.data) return;
// Handle active jobs
const activeJobs = response.data
.filter((job) => {
const status = job.status?.status?.toLowerCase();
return (
status && status !== "complete_success" && status !== "failed"
);
})
.map(
(job) =>
({
job_token: job.job_token,
request: {
maybe_model_title:
job.request?.maybe_model_title || "Image Generation",
},
status: {
status: job.status?.status?.toUpperCase() || "",
progress_percentage: job.status?.progress_percentage || 0,
},
created_at: job.created_at || new Date().toISOString(),
updated_at: job.updated_at || new Date().toISOString(),
maybe_result: job.maybe_result || null,
}) as ActiveJob,
);
// Handle completed jobs
const newlyCompleted = response.data.filter(
(job) =>
job.status?.status?.toLowerCase() === "complete_success" &&
!processedJobs.current.has(job.job_token),
);
if (newlyCompleted.length > 0) {
// Add to processed set
newlyCompleted.forEach((job) =>
processedJobs.current.add(job.job_token),
);
// Create completed items
const newItems = newlyCompleted.map((job) => ({
token: job.job_token,
maybe_title: job.request?.maybe_model_title || "Image Generation",
public_bucket_path:
job.maybe_result?.maybe_public_bucket_media_path || "",
updated_at: job.updated_at || new Date().toISOString(),
}));
// Show toast only if not first load
if (!isFirstLoad.current) {
showToast(newItems);
}
// Update state
setCompletedItems((prev) => {
const combined = [...newItems, ...prev];
return combined
.filter(
(item, index, self) =>
index === self.findIndex((t) => t.token === item.token),
)
.sort(
(a, b) =>
new Date(b.updated_at).getTime() -
new Date(a.updated_at).getTime(),
)
.slice(0, 50);
});
}
setJobs(activeJobs);
isFirstLoad.current = false;
} catch (error) {
console.error("Failed to fetch jobs:", error);
}
}
// Initial fetch
fetchJobs();
// Set up polling
const interval = setInterval(fetchJobs, 5000);
// Cleanup
return () => {
mounted = false;
clearInterval(interval);
if (toastTimeoutRef.current) {
clearTimeout(toastTimeoutRef.current);
}
};
}, [showToast]);
return (
<Tooltip content="Activity" position="bottom" closeOnClick={true}>
@@ -20,26 +194,27 @@ export function Activity() {
align="end"
triggerIcon={
<div>
<FontAwesomeIcon icon={faBell} />
{activeImageGenerationJobs.value &&
activeImageGenerationJobs.value.length > 0 && (
<div className="bg-blue-500 absolute -right-1 -top-1 h-3 w-3 animate-pulse rounded-full" />
)}
{jobs.length > 0 ? (
<FontAwesomeIcon icon={faSpinnerThird} className="animate-spin" />
) : (
<FontAwesomeIcon icon={faBell} />
)}
{jobs.length > 0 && (
<div className="absolute -right-1 -top-1 h-3 w-3 animate-pulse rounded-full bg-brand-primary-400" />
)}
</div>
}
>
<div className="max-h-[480px] overflow-y-auto">
{/* In progress */}
{activeImageGenerationJobs.value &&
activeImageGenerationJobs.value.length > 0 && (
<div>
{activeImageGenerationJobs.value.map((movieJob) => (
<InProgressCard key={movieJob.job_token} movie={movieJob} />
))}
</div>
)}
{jobs.length > 0 && (
<div>
{jobs.map((job) => (
<InProgressCard key={job.job_token} job={job} />
))}
</div>
)}
{!userMovies.value && !activeImageGenerationJobs.value ? (
{isFirstLoad.current ? (
<div className="flex h-48 w-full flex-col justify-center gap-4 p-4 text-center align-middle">
<FontAwesomeIcon
icon={faSpinnerThird}
@@ -49,21 +224,15 @@ export function Activity() {
/>
<H3 className="text-gray-300">Retrieving Activities</H3>
</div>
) : userMovies.value &&
userMovies.value.length === 0 &&
!activeImageGenerationJobs.value ? (
) : jobs.length === 0 && completedItems.length === 0 ? (
<div className="flex h-48 w-full flex-col justify-center gap-4 p-4 text-center align-middle">
<H3 className="text-gray-300">No activities yet</H3>
<P className="text-gray-400">
Try generating something new from the featured scenes
</P>
<h3 className="text-lg text-gray-300">No activities yet</h3>
</div>
) : (
<div>
{/* Completed */}
<div className="flex flex-col">
{userMovies.value?.map((movie) => (
<CompletedCard key={movie.token} movie={movie} />
{completedItems.map((item) => (
<CompletedCard key={item.token} job={item} />
))}
</div>
</div>
@@ -1,91 +1,84 @@
import { MediaInfo } from "~/pages/PageEnigma/models/movies";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faArrowDownToLine } from "@fortawesome/pro-solid-svg-icons";
import { useRef, useState } from "react";
import { BucketConfig } from "~/api/BucketConfig";
import { useState } from "react";
import dayjs from "dayjs";
import environmentVariables from "~/Classes/EnvironmentVariables";
import { useSignals } from "@preact/signals-react/runtime";
import { downloadFile } from "~/pages/PageEnigma/comps/GenerateModals/utils/downloadFile";
import { styleList } from "~/pages/PageEnigma/styleList";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { faDownToLine } from "@fortawesome/pro-solid-svg-icons";
import { LightboxModal } from "~/components/reusable/LightboxModal";
import { GetCdnOrigin } from "~/api/GetCdnOrigin";
export function getStyleName(typeInput: string) {
const foundStyle = styleList.find((style) => {
return style.type === typeInput;
});
return foundStyle ? foundStyle.label : "Unknown Style";
}
interface Props {
movie: MediaInfo;
setMovieId?: (id: string) => void;
job: {
token: string;
maybe_title: string;
updated_at: string;
public_bucket_path: string;
maybe_style_name?: string;
};
}
export function CompletedCard({ movie, setMovieId }: Props) {
useSignals();
const bucketConfig = useRef<BucketConfig>(new BucketConfig());
export function CompletedCard({ job }: Props) {
const [loadError, setLoadError] = useState(false);
const [isModalOpen, setIsModalOpen] = useState(false);
const cdnOrigin = GetCdnOrigin();
const downloadLink = `https://${cdnOrigin}${movie.public_bucket_path}`;
const mediaPath = job?.public_bucket_path || "";
const downloadLink = `${cdnOrigin}${mediaPath}`;
const imageUrl = downloadLink;
const fullImageUrl = downloadLink;
const imageUrl = bucketConfig.current.getCdnUrl(
movie.public_bucket_path + "-thumb.gif",
360,
20,
);
const styleName =
movie.maybe_style_name && getStyleName(movie.maybe_style_name);
const handleClick = () => {
setMovieId?.(movie.token);
};
if (!job) return null;
return (
<div
className="flex w-full items-center justify-between rounded-lg p-2 text-start transition-all duration-150 hover:cursor-pointer hover:bg-white/10"
onClick={handleClick}
>
<div className="flex gap-4">
<div className="flex aspect-square h-14 w-14 justify-center overflow-hidden rounded-lg border border-[#A9A9A9]/50 bg-black/60">
<img
src={
loadError ? "/resources/images/movie-placeholder.png" : imageUrl
}
className="h-full w-full object-cover"
alt={movie.maybe_title ?? "unknown"}
crossOrigin="anonymous"
onError={() => setLoadError(true)}
loading="lazy"
/>
</div>
<div className="flex flex-col justify-center gap-1">
<div className="font-medium">{movie.maybe_title || "Untitled"}</div>
<div>
{/* {styleName && (
<div className="text-sm text-white/60">{styleName}</div>
)} */}
<>
<div
className="flex w-full items-center justify-between rounded-lg p-2 text-start transition-all duration-150 hover:cursor-pointer hover:bg-white/10"
onClick={() => setIsModalOpen(true)}
>
<div className="flex gap-4">
<div className="flex aspect-square h-14 w-14 justify-center overflow-hidden rounded-lg border border-[#A9A9A9]/50 bg-black/60">
<img
src={
loadError ? "/resources/images/movie-placeholder.png" : imageUrl
}
className="h-full w-full object-cover"
alt={job.maybe_title ?? "unknown"}
crossOrigin="anonymous"
onError={() => setLoadError(true)}
loading="lazy"
/>
</div>
<div className="flex flex-col justify-center gap-1">
<div className="font-medium">{job.maybe_title || "Untitled"}</div>
<div className="text-sm text-white/60">
{dayjs(movie.updated_at).format("MMM D, YYYY HH:mm:ss")}
{dayjs(job.updated_at).format("MMM D, YYYY HH:mm:ss")}
</div>
</div>
</div>
</div>
{/* <div className="pr-5">
<button
onClick={(event) => {
event.preventDefault();
event.stopPropagation();
const title =
movie.maybe_title !== null ? movie.maybe_title : "Untitled";
downloadFile({ url: downloadLink, title });
}}
className="text-[15px] font-medium text-white/50 transition-all duration-150 hover:text-white/100"
<a
href={downloadLink}
download
onClick={(e) => e.stopPropagation()}
className="flex h-9 w-9 items-center justify-center text-lg text-white/60 transition-all hover:text-white"
>
<FontAwesomeIcon icon={faArrowDownToLine} className="mr-2" />
Download
</button>
</div> */}
</div>
<FontAwesomeIcon icon={faDownToLine} />
</a>
</div>
<LightboxModal
isOpen={isModalOpen}
onClose={() => setIsModalOpen(false)}
imageUrl={fullImageUrl}
imageAlt={job.maybe_title ?? "unknown"}
onImageError={() => setLoadError(true)}
title={job.maybe_title}
createdAt={job.updated_at}
downloadUrl={downloadLink}
additionalInfo={
job.maybe_style_name && (
<div className="text-sm text-white/60">
Style: {job.maybe_style_name}
</div>
)
}
/>
</>
);
}
@@ -1,22 +1,15 @@
import { useCallback } from "react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faClose,
faCircleExclamation,
faSpinnerThird,
} from "@fortawesome/pro-solid-svg-icons";
import { faClose, faSpinnerThird } from "@fortawesome/pro-solid-svg-icons";
import { ToastTypes } from "~/enums";
import { addToast } from "~/signals";
import { JobState } from "~/pages/PageEnigma/enums";
import { ActiveJob } from "~/pages/PageEnigma/models";
import { Tooltip } from "~/components";
import { JobsApi } from "~/Classes/ApiManager";
import { PollRecentJobs } from "~/hooks/useActiveJobs/utilities";
interface Props {
movie: ActiveJob;
job: ActiveJob;
}
function getPercent(status: JobState, percentage: number) {
@@ -33,16 +26,16 @@ function getPercent(status: JobState, percentage: number) {
}
}
export function InProgressCard({ movie }: Props) {
export function InProgressCard({ job }: Props) {
const completePercent = getPercent(
movie.status.status as JobState,
movie.status.progress_percentage,
job.status.status as JobState,
job.status.progress_percentage,
);
const deleteJob = useCallback(async (movieJob: ActiveJob) => {
const deleteJob = useCallback(async (job: ActiveJob) => {
const jobsApi = new JobsApi();
const response = await jobsApi.DeleteJobByToken(movieJob.job_token);
const response = await jobsApi.DeleteJobByToken(job.job_token);
if (response.success) {
PollRecentJobs();
addToast(ToastTypes.SUCCESS, "File successfully deleted.");
@@ -58,32 +51,33 @@ export function InProgressCard({ movie }: Props) {
return (
<div className="flex w-full items-center justify-between rounded-lg p-2 text-start transition-all duration-150 hover:bg-white/10">
<div className="flex gap-4">
<div className="flex aspect-square h-14 w-14 items-center justify-center overflow-hidden rounded-lg border border-[#A9A9A9]/50 bg-black/60">
{(movie.status.status === JobState.STARTED ||
movie.status.status === JobState.PENDING) && (
<FontAwesomeIcon icon={faSpinnerThird} spin size={"lg"} />
<div className="flex aspect-square h-14 w-14 items-center justify-center rounded-lg border border-[#A9A9A9]/50 bg-black/60 text-white">
<FontAwesomeIcon icon={faSpinnerThird} className="animate-spin" />
{/* {(job.status.status === JobState.STARTED ||
job.status.status === JobState.PENDING) && (
<FontAwesomeIcon icon={faSpinnerThird} className="animate-spin" />
)}
{(movie.status.status === JobState.COMPLETE_FAILURE ||
movie.status.status === JobState.ATTEMPT_FAILED) && (
{(job.status.status === JobState.COMPLETE_FAILURE ||
job.status.status === JobState.ATTEMPT_FAILED) && (
<FontAwesomeIcon icon={faCircleExclamation} size={"lg"} />
)}
)} */}
</div>
<div className="flex flex-col justify-center gap-1">
<div className="font-medium">
{movie.request.maybe_model_title || "Untitled"}
{job.request.maybe_model_title || "Image Generation"}
</div>
<div className="text-sm capitalize text-white/60">
{movie.status.status.replaceAll("_", " ")}... {completePercent}%
{job.status.status.replaceAll("_", " ")}... {completePercent}%
</div>
</div>
</div>
{movie.status.status !== JobState.STARTED && (
{job.status.status !== JobState.STARTED && (
<div className="pr-5">
<Tooltip content="Cancel" position="left">
<button
onClick={() => deleteJob(movie)}
onClick={() => deleteJob(job)}
className="text-[15px] font-medium text-white/50 transition-all duration-150 hover:text-white/100"
>
<FontAwesomeIcon icon={faClose} />
@@ -24,4 +24,28 @@ export const downloadFile = ({
.catch(() => addToast(ToastTypes.ERROR, "Could not download file."));
};
export const downloadFileImage = ({
url,
title,
}: {
url: string;
title: string;
}) => {
// console.log(url);
fetch(url)
.then((resp) => resp.blob())
.then((blob) => {
const blobUrl = window.URL.createObjectURL(blob);
const a = document.createElement("a");
a.style.display = "none";
a.href = blobUrl;
a.download = `${title}.png`;
document.body.appendChild(a);
a.click();
window.URL.revokeObjectURL(url);
})
.catch(() => addToast(ToastTypes.ERROR, "Could not download file."));
};
//check
@@ -1,15 +1,14 @@
import { TransitionDialogue } from "~/components/reusable/TransitionDialogue";
import { faCheck, faXmark } from "@fortawesome/pro-solid-svg-icons";
import { faCheck } from "@fortawesome/pro-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { Button, LoadingSpinner, CloseButton } from "~/components";
import { TabSelector } from "~/components/reusable/TabSelector";
import { useState, useEffect, Fragment } from "react";
import React, { useState, useEffect, useCallback, useMemo } from "react";
import { LibraryModalApi } from "./LibraryModalApi";
import { FilterMediaClasses } from "../../../../enums";
import { BucketConfig } from "../../../../api/BucketConfig";
import { twMerge } from "tailwind-merge";
import { createPortal } from "react-dom";
import { Transition } from "@headlessui/react";
import { LightboxModal } from "~/components/reusable/LightboxModal";
import { GetCdnOrigin } from "~/api/GetCdnOrigin";
export interface LibraryItem {
id: string;
@@ -38,323 +37,292 @@ interface LibraryModalProps {
onTabChange: (tabId: string) => void;
}
export const LibraryModal = ({
isOpen,
onClose,
mode = "view",
selectedItemIds = [],
onSelectItem,
maxSelections = 4,
onUseSelected,
tabs,
activeTab,
onTabChange,
}: LibraryModalProps) => {
const [groupedItems, setGroupedItems] = useState<GroupedItems>({});
const [loading, setLoading] = useState(false);
const [lightboxImage, setLightboxImage] = useState<LibraryItem | null>(null);
const [isLightboxVisible, setIsLightboxVisible] = useState(false);
const [failedImageUrls] = useState<Set<string>>(new Set());
const bucketConfig = new BucketConfig();
const api = new LibraryModalApi();
const formatDate = (date: string) => {
const d = new Date(date);
return d.toLocaleDateString("en-US", {
weekday: "short",
month: "short",
day: "numeric",
});
};
const groupItemsByDate = (items: LibraryItem[]) => {
const grouped = items.reduce((acc: GroupedItems, item) => {
const dateKey = formatDate(item.createdAt);
if (!acc[dateKey]) {
acc[dateKey] = [];
}
acc[dateKey].push(item);
return acc;
}, {});
// Sort dates in descending order
return Object.fromEntries(
Object.entries(grouped).sort(
(a, b) =>
new Date(b[1][0].createdAt).getTime() -
new Date(a[1][0].createdAt).getTime(),
),
export const LibraryModal = React.memo(
({
isOpen,
onClose,
mode = "view",
selectedItemIds = [],
onSelectItem,
maxSelections = 4,
onUseSelected,
tabs,
activeTab,
onTabChange,
}: LibraryModalProps) => {
const [groupedItems, setGroupedItems] = useState<GroupedItems>({});
const [loading, setLoading] = useState(false);
const [lightboxImage, setLightboxImage] = useState<LibraryItem | null>(
null,
);
};
const [isLightboxVisible, setIsLightboxVisible] = useState(false);
const [failedImageUrls] = useState<Set<string>>(new Set());
const cdnOrigin = GetCdnOrigin();
const imageUrl = lightboxImage?.fullImage || "";
const api = useMemo(() => new LibraryModalApi(), []);
const handleImageError = (url: string) => {
failedImageUrls.add(url);
};
const formatDate = useCallback((date: string) => {
const d = new Date(date);
return d.toLocaleDateString("en-US", {
weekday: "short",
month: "short",
day: "numeric",
});
}, []);
const getImageUrl = (path: string | undefined | null) => {
if (!path) return null;
const url = bucketConfig.getGcsUrl(path);
return failedImageUrls.has(url) ? null : url;
};
const groupItemsByDate = useCallback(
(items: LibraryItem[]) => {
const grouped = items.reduce((acc: GroupedItems, item) => {
const dateKey = formatDate(item.createdAt);
if (!acc[dateKey]) {
acc[dateKey] = [];
}
acc[dateKey].push(item);
return acc;
}, {});
useEffect(() => {
const loadItems = async () => {
setLoading(true);
try {
const response = await api.listUserMediaFiles({
filter_media_classes: [FilterMediaClasses.IMAGE],
});
if (response.success && response.data) {
const newItems = response.data.map((item) => ({
id: item.token,
label: item.maybe_title || "Untitled",
thumbnail: getImageUrl(
item.cover_image.maybe_cover_image_public_bucket_path ||
item.public_bucket_path,
),
fullImage: getImageUrl(item.public_bucket_path),
createdAt: item.created_at,
}));
setGroupedItems(groupItemsByDate(newItems));
// Sort dates in descending order
return Object.fromEntries(
Object.entries(grouped).sort(
(a, b) =>
new Date(b[1][0].createdAt).getTime() -
new Date(a[1][0].createdAt).getTime(),
),
);
},
[formatDate],
);
const handleImageError = useCallback(
(url: string) => {
failedImageUrls.add(url);
},
[failedImageUrls],
);
const getImageUrl = useCallback(
(path: string | undefined | null) => {
if (!path) return null;
const normalizedPath = path.startsWith("/") ? path : `/${path}`;
const url = `${cdnOrigin}${normalizedPath}`;
return failedImageUrls.has(url) ? null : url;
},
[cdnOrigin, failedImageUrls],
);
useEffect(() => {
const loadItems = async () => {
setLoading(true);
try {
const response = await api.listUserMediaFiles({
filter_media_classes: [FilterMediaClasses.IMAGE],
});
if (response.success && response.data) {
const newItems = response.data.map((item) => ({
id: item.token,
label: item.maybe_title || "Image Generation",
thumbnail: getImageUrl(item.public_bucket_path),
fullImage: getImageUrl(item.public_bucket_path),
createdAt: item.created_at,
}));
setGroupedItems(groupItemsByDate(newItems));
}
} catch (error) {
console.error("Failed to fetch library items:", error);
}
} catch (error) {
console.error("Failed to fetch library items:", error);
setLoading(false);
};
if (isOpen) {
loadItems();
}
setLoading(false);
};
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [activeTab, isOpen]);
if (isOpen) {
loadItems();
}
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [activeTab, isOpen]);
// useEffect(() => {
// console.log("LibraryModal render caused by:", {
// isOpen,
// activeTab,
// selectedItemIds,
// });
// }, [isOpen, activeTab, selectedItemIds]);
const handleItemClick = (item: LibraryItem) => {
if (mode === "select" && onSelectItem) {
onSelectItem(item.id);
} else {
setLightboxImage(item);
setIsLightboxVisible(true);
}
};
const handleItemClick = useCallback(
(item: LibraryItem) => {
if (mode === "select" && onSelectItem) {
onSelectItem(item.id);
} else {
setLightboxImage(item);
setIsLightboxVisible(true);
}
},
[mode, onSelectItem],
);
const handleCloseLightbox = () => {
setIsLightboxVisible(false);
// Wait for animation to complete before removing the image
setTimeout(() => setLightboxImage(null), 300);
};
const handleCloseLightbox = useCallback(() => {
setIsLightboxVisible(false);
// Wait for animation to complete before removing the image
setTimeout(() => setLightboxImage(null), 300);
}, []);
return (
<>
<TransitionDialogue
isOpen={isOpen}
onClose={onClose}
className={twMerge(
"h-[620px] max-w-4xl",
mode === "view" && "h-[80vh] max-h-[870px] max-w-7xl",
)}
childPadding={false}
>
<div className="flex h-full flex-col">
<div className="border-b border-white/10 p-4 py-3">
<div className="grid grid-cols-3 items-center">
<h2 className="text-xl font-semibold">
{mode === "select" ? "Select Images" : "My Gallery"}
</h2>
<div className="flex items-center justify-center">
<TabSelector
tabs={tabs}
activeTab={activeTab}
onTabChange={onTabChange}
className="w-auto"
/>
</div>
<div className="flex justify-end">
<CloseButton onClick={onClose} />
</div>
</div>
</div>
const handleDeselectAll = useCallback(() => {
selectedItemIds.forEach((id) => onSelectItem?.(id));
}, [selectedItemIds, onSelectItem]);
<div className="flex-1 overflow-y-auto p-4">
{loading ? (
<div className="flex h-full items-center justify-center">
<div className="text-white/60">
<LoadingSpinner className="h-12 w-12" />
const handleUseSelected = useCallback(() => {
const selectedItems = Object.values(groupedItems)
.flat()
.filter((item) => selectedItemIds.includes(item.id));
onUseSelected?.(selectedItems);
}, [groupedItems, selectedItemIds, onUseSelected]);
return (
<>
<TransitionDialogue
isOpen={isOpen}
onClose={onClose}
className={twMerge(
"h-[620px] max-w-4xl",
mode === "view" && "h-[80vh] max-h-[870px] max-w-7xl",
)}
childPadding={false}
>
<div className="flex h-full flex-col">
<div className="border-b border-white/10 p-4 py-3">
<div className="grid grid-cols-3 items-center">
<h2 className="text-xl font-semibold">
{mode === "select" ? "Select Images" : "My Gallery"}
</h2>
<div className="flex items-center justify-center">
<TabSelector
tabs={tabs}
activeTab={activeTab}
onTabChange={onTabChange}
className="w-auto"
/>
</div>
<div className="flex justify-end">
<CloseButton onClick={onClose} />
</div>
</div>
) : (
<div className="space-y-6">
{Object.entries(groupedItems).map(([date, dateItems]) => (
<div key={date}>
<h3 className="text-md mb-2 font-medium text-white/60">
{date}
</h3>
<div
className={twMerge(
"grid grid-cols-5 gap-1",
mode === "view" && "grid-cols-5",
)}
>
{dateItems.map((item) => (
<button
key={item.id}
className={twMerge(
"group relative aspect-square overflow-hidden rounded-md border-[3px] transition-all",
mode === "select" &&
selectedItemIds.includes(item.id)
? "border-brand-primary"
: "border-transparent hover:border-white",
)}
onClick={() => handleItemClick(item)}
>
<div className="relative h-full w-full">
{!item.thumbnail ? (
<div className="flex h-full w-full items-center justify-center bg-gray-800">
<span className="text-white/60">
Image not available
</span>
</div>
) : (
<img
src={item.thumbnail}
alt={item.label}
className="h-full w-full object-cover"
onError={() =>
handleImageError(item.thumbnail!)
}
crossOrigin="anonymous"
referrerPolicy="no-referrer"
/>
</div>
<div className="flex-1 overflow-y-auto p-4">
{loading ? (
<div className="flex h-full items-center justify-center">
<div className="text-white/60">
<LoadingSpinner className="h-12 w-12" />
</div>
</div>
) : (
<div className="space-y-6">
{Object.entries(groupedItems).map(([date, dateItems]) => (
<div key={date}>
<h3 className="text-md mb-2 font-medium text-white/60">
{date}
</h3>
<div
className={twMerge(
"grid grid-cols-5 gap-1",
mode === "view" && "grid-cols-5",
)}
>
{dateItems.map((item) => (
<button
key={item.id}
className={twMerge(
"group relative aspect-square overflow-hidden rounded-md border-[3px] transition-all",
mode === "select" &&
selectedItemIds.includes(item.id)
? "border-brand-primary"
: "border-transparent hover:border-white",
)}
onClick={() => handleItemClick(item)}
>
<div className="relative h-full w-full">
{!item.thumbnail ? (
<div className="flex h-full w-full items-center justify-center bg-gray-800">
<span className="text-white/60">
Image not available
</span>
</div>
) : (
<img
src={item.thumbnail}
alt={item.label}
className="h-full w-full object-cover"
onError={() =>
handleImageError(item.thumbnail!)
}
crossOrigin="anonymous"
referrerPolicy="no-referrer"
/>
)}
{mode === "select" &&
selectedItemIds.includes(item.id) && (
<div className="absolute inset-0 bg-white/30" />
)}
</div>
{mode === "select" &&
selectedItemIds.includes(item.id) && (
<div className="absolute inset-0 bg-white/30" />
<div className="absolute right-2 top-2 flex h-6 w-6 items-center justify-center rounded-full bg-brand-primary">
<FontAwesomeIcon
icon={faCheck}
className="text-sm"
/>
</div>
)}
</div>
{mode === "select" &&
selectedItemIds.includes(item.id) && (
<div className="absolute right-2 top-2 flex h-6 w-6 items-center justify-center rounded-full bg-brand-primary">
<FontAwesomeIcon
icon={faCheck}
className="text-sm"
/>
</div>
)}
</button>
))}
</button>
))}
</div>
</div>
))}
</div>
)}
</div>
{mode === "select" && (
<div className="flex items-center justify-between border-t border-white/10 bg-black/25 p-4">
<div className="flex items-center gap-3">
<div className="text-sm font-semibold text-white/80">
{selectedItemIds.length}/{maxSelections} selected
</div>
))}
{selectedItemIds.length > 0 && (
<span className="text-white/10">|</span>
)}
{selectedItemIds.length > 0 && (
<button
onClick={handleDeselectAll}
className="text-sm text-white/60 hover:text-white"
>
Deselect All
</button>
)}
</div>
<Button
onClick={handleUseSelected}
disabled={selectedItemIds.length === 0}
>
Use selected
</Button>
</div>
)}
</div>
</TransitionDialogue>
{mode === "select" && (
<div className="flex items-center justify-between border-t border-white/10 bg-black/25 p-4">
<div className="flex items-center gap-3">
<div className="text-sm font-semibold text-white/80">
{selectedItemIds.length}/{maxSelections} selected
</div>
{selectedItemIds.length > 0 && (
<span className="text-white/10">|</span>
)}
{selectedItemIds.length > 0 && (
<button
onClick={() => {
selectedItemIds.forEach((id) => onSelectItem?.(id));
}}
className="text-sm text-white/60 hover:text-white"
>
Deselect All
</button>
)}
</div>
<Button
onClick={() => {
const selectedItems = Object.values(groupedItems)
.flat()
.filter((item) => selectedItemIds.includes(item.id));
onUseSelected?.(selectedItems);
}}
disabled={selectedItemIds.length === 0}
>
Use selected
</Button>
</div>
)}
</div>
</TransitionDialogue>
<LightboxModal
isOpen={isLightboxVisible}
onClose={handleCloseLightbox}
imageUrl={imageUrl}
imageAlt={lightboxImage?.label || ""}
onImageError={() => imageUrl && handleImageError(imageUrl)}
title={lightboxImage?.label}
createdAt={lightboxImage?.createdAt}
downloadUrl={imageUrl}
/>
</>
);
},
);
{/* Lightbox Modal */}
{lightboxImage &&
createPortal(
<Transition appear show={isLightboxVisible} as={Fragment}>
<div className="fixed inset-0 z-[100]">
<Transition.Child
as={Fragment}
enter="ease-out duration-300"
enterFrom="opacity-0"
enterTo="opacity-100"
leave="ease-in duration-200"
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
<div
className="fixed inset-0 cursor-pointer bg-black/60"
onClick={handleCloseLightbox}
/>
</Transition.Child>
<div
className="fixed inset-0 flex items-center justify-center p-4"
onClick={handleCloseLightbox}
>
<Transition.Child
as={Fragment}
enter="ease-out duration-300"
enterFrom="opacity-0 scale-95"
enterTo="opacity-100 scale-100"
leave="ease-in duration-200"
leaveFrom="opacity-100 scale-100"
leaveTo="opacity-0 scale-95"
>
<div
className="relative h-[90vh] w-[80vw] rounded-xl bg-[#2C2C2C]"
onClick={(e) => e.stopPropagation()}
>
<CloseButton
onClick={handleCloseLightbox}
className="absolute right-4 top-4 z-10"
/>
<div className="grid h-full grid-cols-3 gap-6">
<div className="col-span-2 flex h-full items-center justify-center overflow-hidden bg-black/40">
{!lightboxImage.fullImage ? (
<div className="flex h-full w-full items-center justify-center bg-gray-800">
<span className="text-white/60">
Image not available
</span>
</div>
) : (
<img
src={lightboxImage.fullImage}
alt={lightboxImage.label}
className="h-full w-full object-contain"
onError={() =>
handleImageError(lightboxImage.fullImage!)
}
crossOrigin="anonymous"
referrerPolicy="no-referrer"
/>
)}
</div>
<div className="py-5">Prompt</div>
</div>
</div>
</Transition.Child>
</div>
</div>
</Transition>,
document.body,
)}
</>
);
};
LibraryModal.displayName = "LibraryModal";
@@ -1,10 +1,41 @@
import { useState, useEffect } from "react";
import { Transition } from "@headlessui/react";
import { Key, KeyGroup, Mouse, Plus } from "./ControlsTopButtons/Help/Help";
import { CloseButton } from "~/components";
const STORAGE_KEY = "onboarding_helper_dismissed";
const EXPIRATION_DAYS = 3;
export const OnboardingHelper = () => {
const [actionPerformed, setActionPerformed] = useState(false);
const [keyPressed, setKeyPressed] = useState(false);
const [isVisible, setIsVisible] = useState(true);
useEffect(() => {
// Check if the helper was dismissed
const dismissedData = localStorage.getItem(STORAGE_KEY);
if (dismissedData) {
const { timestamp } = JSON.parse(dismissedData);
const expirationTime = timestamp + EXPIRATION_DAYS * 24 * 60 * 60 * 1000; // 3 days in milliseconds
if (Date.now() < expirationTime) {
setIsVisible(false);
} else {
// Clear expired data
localStorage.removeItem(STORAGE_KEY);
}
}
}, []);
const handleClose = () => {
setIsVisible(false);
// Store dismissal timestamp
localStorage.setItem(
STORAGE_KEY,
JSON.stringify({
timestamp: Date.now(),
})
);
};
useEffect(() => {
const handleKeyDown = (event: KeyboardEvent) => {
@@ -30,7 +61,7 @@ export const OnboardingHelper = () => {
return (
<Transition
show={!(actionPerformed && keyPressed)}
show={isVisible && !(actionPerformed && keyPressed)}
enter="transition-opacity duration-500"
enterFrom="opacity-0"
enterTo="opacity-100"
@@ -38,7 +69,7 @@ export const OnboardingHelper = () => {
leaveFrom="opacity-100"
leaveTo="opacity-0"
>
<div className="glass pointer-events-none absolute left-1/2 top-52 z-10 flex -translate-x-1/2 transform items-center justify-center gap-2 rounded-2xl border-2 border-brand-primary !bg-brand-primary/10 px-5 py-4">
<div className="glass pointer-events-none absolute left-1/2 top-48 z-10 flex -translate-x-1/2 transform items-center justify-center gap-2 rounded-2xl border-2 border-brand-primary !bg-brand-primary/10 px-5 py-4">
<div className="flex items-center">
<p className="pr-3 text-lg font-semibold">Hold</p>
<Mouse button="left" />
@@ -56,6 +87,9 @@ export const OnboardingHelper = () => {
</div>
</div>
<p className="text-lg font-semibold">to move around the scene</p>
<div className="pointer-events-auto ml-2">
<CloseButton onClick={handleClose} />
</div>
</div>
</Transition>
);
@@ -140,7 +140,7 @@ export const Outliner = () => {
useEffect(() => {
const updateEditorHeight = () => {
setEditorHeight(pageHeight.value - timelineHeight.value - 64);
setEditorHeight(pageHeight.value - timelineHeight.value - 56);
};
updateEditorHeight();
@@ -0,0 +1,73 @@
import React from "react";
import { useSignals } from "@preact/signals-react/runtime";
import { poseMode, showPoseControls } from "../signals/selectedMode";
import { Button, Tooltip } from "~/components";
import {
faCheck,
faCheckDouble,
faPersonRunning,
} from "@fortawesome/pro-solid-svg-icons";
declare global {
interface Window {
__mouseControls?: {
toggleFKMode: () => void;
};
}
}
interface PoseModeSelectorProps {}
export const PoseModeSelector: React.FC<PoseModeSelectorProps> = () => {
useSignals();
const handleModeChange = () => {
if (poseMode.value === "select") {
poseMode.value = "pose";
} else {
poseMode.value = "select";
}
window.__mouseControls?.toggleFKMode();
};
if (showPoseControls.value === false) {
return null;
}
return (
<>
<div
className="fixed left-1/2 top-32 flex -translate-x-1/2 transform items-center justify-center gap-2"
onClick={(e) => e.stopPropagation()}
onMouseDown={(e) => e.stopPropagation()}
>
<Tooltip
content={"Toggle pose mode (K)"}
position={"bottom"}
delay={300}
closeOnClick={true}
>
<>
{poseMode.value === "select" ? (
<Button
icon={faPersonRunning}
onClick={handleModeChange}
className="rounded-xl shadow-xl outline-none focus-visible:outline-none"
>
Enter Pose Mode
</Button>
) : (
<Button
icon={faCheck}
onClick={handleModeChange}
className="rounded-xl outline-none focus-visible:outline-none"
>
Done
</Button>
)}
</>
</Tooltip>
</div>
</>
);
};
@@ -0,0 +1,43 @@
import { Signal } from "@preact/signals-react";
import { useSignals } from "@preact/signals-react/runtime";
export const PrecisionSelector = ({
showSignal,
coordSignal,
valuesSignal,
selectedValueSignal
}: {
showSignal: Signal<boolean>,
coordSignal: Signal<{ x: number; y: number }>,
valuesSignal: Signal<number[]>
selectedValueSignal: Signal<number>
}) => {
useSignals();
const handleMouseLeave = () => {
showSignal.value = false;
}
const handleMouseEnterItem = (scale: number) => {
selectedValueSignal.value = scale;
console.log("selectedValueSignal", selectedValueSignal.value);
}
return (
<div
className="fixed z-50 bg-red-600 -translate-x-1/2 -translate-y-1/2 flex-col gap-[1px] shadow-md bg-ui-divider border-ui-divider border-2 rounded-md overflow-clip"
style={{
top: coordSignal.value.y,
left: coordSignal.value.x,
display: showSignal.value ? "flex" : "none"
}}
onMouseLeave={handleMouseLeave}
>
{valuesSignal.value.map((scale, index) => (
<span onMouseEnter={() => handleMouseEnterItem(scale)} key={index}
className="flex bg-ui-panel text-sm justify-center align-middle justify-items-center px-2 hover:bg-ui-controls"
>{scale}</span>
))}
</div>
)
}
@@ -24,16 +24,19 @@ import { CameraAspectRatio, EditorStates } from "~/pages/PageEnigma/enums";
import { CameraViewCanvas } from "~/pages/PageEnigma/comps/EngineCanvases";
import { Button, ButtonIcon, Tooltip } from "~/components";
import { useEffect } from "react";
export const PreviewEngineCamera = () => {
useSignals();
const handleButtonCameraView = () => {
console.log("Camera view button clicked");
Queue.publish({
queueName: QueueNames.TO_ENGINE,
action: toEngineActions.TOGGLE_CAMERA_STATE,
data: null,
});
console.log("Action published to queue");
};
const getLargeScreenHeightClass = () => {
@@ -67,11 +70,15 @@ export const PreviewEngineCamera = () => {
}
};
useEffect(() => {
console.log("Editor state changed:", editorState.value);
}, [editorState.value]);
return (
<div
id="preview-engine-camera"
className={twMerge(
"origin-bottom-left shadow-lg", //hidden right now with css
"hidden origin-bottom-left shadow-lg", //hidden right now with css
editorState.value === EditorStates.PREVIEW
? "invisible h-0 w-0"
: "visible",
@@ -27,7 +27,7 @@ export const PreviewFrameImage = () => {
pageWidth.value -
(sidePanelVisible.value ? sidePanelWidth.value : 0) -
84,
height: pageHeight.value - timelineHeight.value - 64,
height: pageHeight.value - timelineHeight.value - 56,
}}
>
<div className="relative flex h-full w-full flex-col items-center justify-center gap-5">
@@ -48,7 +48,7 @@ export const PreviewFrameImage = () => {
pageWidth.value -
(sidePanelVisible.value ? sidePanelWidth.value : 0) -
84,
height: pageHeight.value - timelineHeight.value - 64,
height: pageHeight.value - timelineHeight.value - 56,
}}
/>
);
@@ -46,7 +46,9 @@ import {
addCamera,
deleteCamera,
updateCamera,
focalLengthDragging,
} from "~/pages/PageEnigma/signals/camera";
import { toast } from "@storyteller/ui-toaster";
interface ReferenceImage {
id: string;
@@ -93,7 +95,6 @@ export const PromptBox = () => {
[],
);
const [activeLibraryTab, setActiveLibraryTab] = useState("my-media");
const [isGridVisible, setIsGridVisible] = useState(true);
useEffect(() => {
if (textareaRef.current) {
@@ -121,36 +122,138 @@ export const PromptBox = () => {
// eslint-disable-next-line react-hooks/exhaustive-deps
}, [cameraAspectRatio.value]);
useEffect(() => {
setGridVisibility(isGridVisible);
}, [isGridVisible]);
const handleCameraSelect = (selectedItem: PopoverItem) => {
const selectedCamera = cameras.value.find(
(cam) => cam.label === selectedItem.label,
);
if (selectedCamera) {
if (selectedCamera && editorEngine) {
selectedCameraId.value = selectedCamera.id;
// Force update camera properties
// Show focal length display temporarily
// TODO: Rename dragging to visible - BFlat
focalLengthDragging.value = {
isDragging: true,
focalLength: selectedCamera.focalLength,
};
setTimeout(() => {
focalLengthDragging.value = {
isDragging: false,
focalLength: selectedCamera.focalLength,
};
}, 1500);
// Update the main camera to match the selected camera's properties
if (editorEngine.camera) {
// First update position and lookAt
editorEngine.camera.position.set(
selectedCamera.position.x,
selectedCamera.position.y,
selectedCamera.position.z,
);
editorEngine.camera.lookAt(
selectedCamera.lookAt.x,
selectedCamera.lookAt.y,
selectedCamera.lookAt.z,
);
// Update FOV
editorEngine.camera.fov = editorEngine.focalLengthToFov(
selectedCamera.focalLength,
);
editorEngine.camera.updateProjectionMatrix();
// Reset and update camera controls
if (editorEngine.cameraViewControls) {
editorEngine.cameraViewControls.reset();
editorEngine.cameraViewControls.update(0);
}
// Force a render to update the view
editorEngine.renderScene();
// Queue an update to ensure the engine processes the camera change
// Queue.publish({
// queueName: QueueNames.TO_ENGINE,
// action: toEngineActions.CAMERA_CHANGED,
// data: selectedCamera,
// });
}
// Force update camera properties in the state
updateCamera(selectedCamera.id, {
focalLength: selectedCamera.focalLength,
position: selectedCamera.position,
rotation: selectedCamera.rotation,
lookAt: selectedCamera.lookAt,
});
}
};
const handleAddCamera = () => {
// Check if we've reached the maximum number of cameras
if (cameras.value.length >= 6) {
console.warn("Maximum number of cameras (6) reached");
return;
}
const newIndex = cameras.value.length + 1;
const newId = `cam${newIndex}`;
// This is for generating random orbital position for the new camera using spherical coordinates
const radius = Math.random() * 5 + 7; // Distance from center: 7 to 12 units
const theta = Math.random() * Math.PI * 2; // Azimuthal angle: 0 to 2π
const phi = Math.PI / 3 + (Math.random() * Math.PI) / 6; // Polar angle: π/3 to π/2 (60° to 90° from horizontal)
// Convert spherical coordinates to Cartesian coordinates
const randomX = radius * Math.sin(phi) * Math.cos(theta);
const randomY = Math.abs(radius * Math.cos(phi)) + 2; // Ensure Y is positive and at least 2 units up
const randomZ = radius * Math.sin(phi) * Math.sin(theta);
addCamera({
id: newId,
label: `Camera ${newIndex}`,
focalLength: 35,
fov: 70,
position: { x: 0, y: 2.5, z: -5 },
focalLength: 24,
position: {
x: randomX,
y: randomY,
z: randomZ,
},
rotation: { x: 0, y: 0, z: 0 },
lookAt: { x: 0, y: 0, z: 0 },
});
// Switch to the newly created camera
selectedCameraId.value = newId;
// Update the engine camera to match the new camera's properties
if (editorEngine && editorEngine.camera) {
editorEngine.camera.position.set(randomX, randomY, randomZ);
editorEngine.camera.lookAt(0, 0, 0);
editorEngine.camera.fov = editorEngine.focalLengthToFov(24);
editorEngine.camera.updateProjectionMatrix();
// Reset and update camera controls
if (editorEngine.cameraViewControls) {
editorEngine.cameraViewControls.reset();
editorEngine.cameraViewControls.update(0);
}
// Force a render to update the view
editorEngine.renderScene();
// Queue.publish({
// queueName: QueueNames.TO_ENGINE,
// action: toEngineActions.CAMERA_CHANGED,
// data: {
// id: newId,
// label: `Camera ${newIndex}`,
// focalLength: 24,
// position: { x: randomX, y: randomY, z: randomZ },
// rotation: { x: 0, y: 0, z: 0 },
// lookAt: { x: 0, y: 0, z: 0 },
// },
// });
}
};
const handleCameraNameChange = (id: string, newName: string) => {
@@ -329,7 +432,9 @@ export const PromptBox = () => {
sceneMediaToken: "",
});
console.log("useSystemPrompt", useSystemPrompt);
await engineApi.enqueueImageGeneration({
disableSystemPrompt: !useSystemPrompt,
prompt: prompt,
snapshotMediaToken: snapshotResult.data || "",
additionalImages: referenceImages.map((image) => image.mediaToken),
@@ -347,6 +452,7 @@ export const PromptBox = () => {
await new Promise((resolve) => setTimeout(resolve, 2000));
} finally {
setisEnqueueing(false);
toast.success("Image added to queue");
}
}
setisEnqueueing(false);
@@ -534,6 +640,9 @@ export const PromptBox = () => {
<FontAwesomeIcon icon={faCamera} className="h-4 w-4" />
),
focalLength: cam.focalLength,
position: cam.position,
rotation: cam.rotation,
lookAt: cam.lookAt,
}))}
onSelect={handleCameraSelect}
onAdd={handleAddCamera}
@@ -541,11 +650,13 @@ export const PromptBox = () => {
<FontAwesomeIcon icon={faCamera} className="h-4 w-4" />
}
showAddButton
disableAddButton={cameras.value.length >= 6}
showIconsInList
mode="toggle"
panelTitle="Camera"
panelActionLabel="Settings"
onPanelAction={() => setIsCameraSettingsOpen(true)}
buttonClassName="max-w-[130px] h-9"
/>
</Tooltip>
<Tooltip
@@ -630,12 +741,15 @@ export const PromptBox = () => {
selected: cam.id === selectedCameraId.value,
icon: <FontAwesomeIcon icon={faCamera} className="h-4 w-4" />,
focalLength: cam.focalLength,
position: cam.position,
rotation: cam.rotation,
lookAt: cam.lookAt,
}))}
onCameraNameChange={handleCameraNameChange}
onCameraFocalLengthChange={handleCameraFocalLengthChange}
onAddCamera={handleAddCamera}
selectedCameraId={selectedCameraId.value}
onSelectCamera={(id: string) => (selectedCameraId.value = id)}
handleCameraSelect={handleCameraSelect}
onDeleteCamera={deleteCamera}
/>
</div>
@@ -11,7 +11,7 @@ export const SceneContainer = ({ children }: { children: React.ReactNode }) => {
const containerWidth = pageWidth.value;
const containerHeight = pageHeight.value - timelineHeight.value - 64;
const containerHeight = pageHeight.value - timelineHeight.value - 56;
const callbackRef = useCallback(
(node: HTMLDivElement) => {
@@ -35,7 +35,7 @@ export const SidePanelMenu = ({
sidePanelVisible.value ? "border-transparent" : "border-[#363636]",
)}
style={{
height: pageHeight.value - 64,
height: pageHeight.value - 56,
}}
>
<div className="flex w-full flex-col gap-2">
@@ -4,9 +4,9 @@ export interface Camera {
id: string;
label: string;
focalLength: number;
fov: number;
position: { x: number; y: number; z: number };
rotation: { x: number; y: number; z: number };
lookAt: { x: number; y: number; z: number };
}
export const cameras = signal<Camera[]>([
@@ -14,17 +14,17 @@ export const cameras = signal<Camera[]>([
id: "main",
label: "Main View",
focalLength: 17,
fov: 70,
position: { x: 2.5, y: 2.5, z: -2.5 },
position: { x: -2.5, y: 2.5, z: 2.5 },
rotation: { x: 0, y: 0, z: 0 },
lookAt: { x: 0, y: 0, z: 0 },
},
{
id: "cam2",
label: "Camera 2",
focalLength: 35,
fov: 70,
position: { x: -2.5, y: 2.5, z: 2.5 },
focalLength: 10,
position: { x: 0, y: 0.6, z: 1.5 },
rotation: { x: 0, y: 0, z: 0 },
lookAt: { x: 0, y: 0, z: 0 },
},
]);
@@ -0,0 +1,6 @@
import { Signal, signal } from "@preact/signals-react";
export const showPrecisionSelector = signal(false);
export const precisionSelectorMenuCoords = signal({ x: 0, y: 0 });
export const precisionSelectorValues: Signal<number[]> = signal([]);
export const precisionSelectedValue = signal(0);
@@ -1,3 +1,5 @@
import { signal } from "@preact/signals-react";
export const selectedMode = signal("move");
export const showPoseControls = signal(false);
export const poseMode = signal("select");
+3
View File
@@ -16,6 +16,7 @@ import normalizeCss from "./styles/normalize.css?url";
import tailwindCss from "./styles/tailwind.css?url";
import baseCss from "./styles/base.css?url";
import { Environment, Configs } from "./configs";
//import { api } from "@storyteller/api";
// The following import prevents a Font Awesome icon server-side rendering bug,
// where the icons flash from a very large icon down to a properly sized one:
@@ -71,6 +72,8 @@ export async function loader() {
uploadApiVideo = "http://localhost:12345";
break;
}
//console.log(">>>> API LIBRARY IMPORT TEST", api());
const env = {
// @ts-expect-error ProvessEnv is correct
+1 -1
View File
@@ -52,7 +52,7 @@ img {
}
.topbar-spacer {
height: 64px;
height: 56px;
width: 100vw;
}
+4 -6
View File
@@ -1,15 +1,14 @@
[build]
command = "./script/netlify_build.sh"
#publish = "build/client"
command = "./apps/editor3d/script/netlify_build.sh"
publish = "apps/editor3d/dist/"
[build.environment]
REACT_APP_PUBLIC_POSTHOG_KEY = "phc_jBFgac0mVALAFk3negnSfYCcHgvkT00yBLQDmCDYNBb"
REACT_APP_PUBLIC_POSTHOG_UI = "https://us.i.posthog.com"
[dev]
command = "npm run dev"
framework = "vite"
#[dev]
#command = "npm run dev"
#framework = "vite"
# Set immutable caching for static files, because they have fingerprinted filenames
@@ -17,7 +16,6 @@ framework = "vite"
for = "/build/*"
[headers.values]
"Cache-Control" = "public, max-age=31560000, immutable"
[[redirects]]
+8 -4
View File
@@ -4,7 +4,14 @@
"sideEffects": false,
"type": "module",
"nx": {
"name": "editor3d"
"name": "editor3d",
"targets": {
"build": {
"dependsOn": [
"^build"
]
}
}
},
"scripts": {
"build": "remix vite:build",
@@ -44,14 +51,12 @@
"@zip.js/zip.js": "^2.7.51",
"ammo.js": "^0.0.10",
"ccapture.js": "github:aceslowman/ccapture.js#npm-fix",
"common": "file:common",
"csstype": "^3.1.3",
"dayjs": "^1.11.10",
"deep-equal": "^2.2.3",
"detect-gpu": "^5.0.38",
"http-proxy-middleware": "^3.0.0",
"isbot": "^4.1.0",
"kalidokit": "file:./vendor/kalidokit",
"lodash.debounce": "^4.0.8",
"posthog-js": "^1.141.0",
"react": "^18.2.0",
@@ -101,7 +106,6 @@
"eslint-plugin-storybook": "^0.8.0",
"husky": "^9.0.11",
"lint-staged": "^15.2.2",
"node-sass": "^9.0.0",
"postcss": "^8.4.37",
"postcss-import": "^16.0.1",
"postcss-nesting": "^12.1.0",
Binary file not shown.

After

Width:  |  Height:  |  Size: 6.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 6.1 KiB

+16 -27
View File
@@ -2,40 +2,29 @@
set -euxo pipefail
# echo "Run build script"
# npm install
# npm run build
#
# #echo "Emit built files."
# #pwd
# #find .
#
# # echo "Copy test files into dist"
# # cp -r test/* dist/
# #
# # echo "Copy netlify configs into dist"
# # cp _headers dist/
# # cp _redirects dist/
# #
# # echo "Copy netlify 404.html page into dist"
# # cp "404.html" dist/
# #
# # echo "List files after build"
# # find dist/
echo "Install packages"
npm install
echo "Run build script (TODO: Make strict)"
nx build editor3d
echo "Build Editor 3D"
nx build editor3d --verbose
echo "Change to project dir"
pushd apps/editor3d/
#echo "Copy netlify configs into dist"
#cp _headers dist/
#cp _redirects dist/
echo "List build directory files"
ls -lA build/
#echo "Copy netlify 404.html page into dist"
echo "Make output directory"
mkdir dist/
echo "Copy files to putput directory"
cp -r build/server/* dist/
cp -r build/client/* dist/
cp _headers dist/
#cp _redirects dist/
#cp "404.html" dist/
echo "List files after build"
echo "List final files in build"
find dist/
+14 -17
View File
@@ -1,34 +1,31 @@
{
"include": [
"env.d.ts",
"**/*.ts",
"**/*.tsx",
"netlify-edge-plugin.ts"
],
"extends": "../../tsconfig.base.json",
"include": ["env.d.ts", "**/*.ts", "**/*.tsx", "netlify-edge-plugin.ts"],
"compilerOptions": {
"lib": [
"DOM",
"DOM.Iterable",
"ESNext"
],
"lib": ["DOM", "DOM.Iterable", "ESNext"],
"isolatedModules": true,
"esModuleInterop": true,
"jsx": "react-jsx",
"module": "ESNext",
"moduleResolution": "node",
"moduleResolution": "bundler",
"resolveJsonModule": true,
"target": "ESNext",
"strict": true,
"allowJs": true,
"skipLibCheck": true,
"forceConsistentCasingInFileNames": true,
"baseUrl": ".",
"paths": {
"~/*": [
"./app/*"
]
"~/*": ["apps/editor3d/app/*"]
},
// Vite takes care of building everything, not tsc.
"noEmit": true
}
},
"references": [
{
"path": "../../libs/login"
},
{
"path": "../../libs/components/toaster"
}
]
}
+9
View File
@@ -14,6 +14,15 @@ export default defineConfig({
// 'kalidokit': `${path.resolve(__dirname, 'src')}/`,
// },
//},
//resolve: {
// alias: [
// {
// find: /@frontend\/login/,
// //replacement: path.resolve(__dirname, 'node_modules', '@frontend', 'login'),
// replacement: path.resolve(__dirname, '../', '../', 'libs', 'login'),
// },
// ],
//},
//rollupOptions: {
// // make sure to externalize deps that shouldn't be bundled
// // into your library
+10
View File
@@ -0,0 +1,10 @@
#!/bin/bash
set -euxo pipefail
echo "Removing Node Modules..."
npx nx reset
rm -rf ./node_modules/
rm -rf ./apps/editor2d/node_modules/
rm -rf ./apps/editor3d/node_modules/
+50
View File
@@ -0,0 +1,50 @@
{
"extends": [
"../../.eslintrc.json"
],
"ignorePatterns": [
"!**/*"
],
"overrides": [
{
"files": [
"*.ts",
"*.tsx",
"*.js",
"*.jsx"
],
"rules": {}
},
{
"files": [
"*.ts",
"*.tsx"
],
"rules": {}
},
{
"files": [
"*.js",
"*.jsx"
],
"rules": {}
},
{
"files": [
"*.json"
],
"parser": "jsonc-eslint-parser",
"rules": {
"@nx/dependency-checks": [
"error",
{
"ignoredFiles": [
"{projectRoot}/eslint.config.{js,cjs,mjs}",
"{projectRoot}/vite.config.{js,ts,mjs,mts}"
]
}
]
}
}
]
}
+11
View File
@@ -0,0 +1,11 @@
# api
This library was generated with [Nx](https://nx.dev).
## Building
Run `nx build api` to build the library.
## Running unit tests
Run `nx test api` to execute the unit tests via [Vitest](https://vitest.dev/).
+19
View File
@@ -0,0 +1,19 @@
{
"name": "@storyteller/api",
"version": "0.0.1",
"private": true,
"type": "module",
"main": "./dist/index.js",
"module": "./dist/index.js",
"types": "./dist/index.d.ts",
"exports": {
"./package.json": "./package.json",
".": {
"development": "./src/index.ts",
"types": "./dist/index.d.ts",
"import": "./dist/index.js",
"default": "./dist/index.js"
}
},
"dependencies": {}
}
+17
View File
@@ -0,0 +1,17 @@
export * from './lib/api.js';
export * from "./lib/AnalyticsApi.js";
export * from "./lib/BillingApi.js";
export * from "./lib/CommentsApi.js";
export * from "./lib/EngineApi.js";
export * from "./lib/FeaturedItemsApi.js";
export * from "./lib/JobsApi.js";
export * from "./lib/MediaFilesApi.js";
export * from "./lib/MediaUploadApi.js";
export * from "./lib/MiscApi.js";
export * from "./lib/ModerationApi.js";
export * from "./lib/PromptsApi.js";
export * from "./lib/TtsApi.js";
export * from "./lib/UserBookmarksApi.js";
export * from "./lib/UsersApi.js";
export * from "./lib/VideoApi.js";
export * from "./lib/WeightsApi.js";
+38
View File
@@ -0,0 +1,38 @@
import { ApiManager, ApiResponse } from "./ApiManager.js";
export class AnalyticsApi extends ApiManager {
public PostAnalytics({
maybeLastAction,
maybeLogToken,
}: {
maybeLastAction: string;
maybeLogToken: string;
}): Promise<ApiResponse<string>> {
const endpoint = `${this.ApiTargets.BaseApi}/v1/analytics/log_session`;
const body = {
maybe_last_action: maybeLastAction,
maybe_log_token: maybeLogToken,
};
return this.post<
{
maybe_last_action: string;
maybe_log_token: string;
},
{
success?: boolean;
log_token?: string;
BadInput?: string;
}
>({ endpoint, body })
.then((response) => ({
success: response.success ?? false,
data: response.log_token,
errorMessage: response.BadInput,
}))
.catch((err) => {
return { success: false, errorMessage: err.message };
});
}
}
+17
View File
@@ -0,0 +1,17 @@
// import { GetCdnOrigin } from "~/api/GetCdnOrigin";
// import environmentVariables from "./EnvironmentVariables";
// export async function get_media_url(mediaId: string) {
// //This is for prod when we have the proper info on the url.
// const api_base_url = environmentVariables.values.BASE_API;
// const url = `${api_base_url}/v1/media_files/file/${mediaId}`;
// const response = await fetch(url);
// const json = await JSON.parse(await response.text());
// const bucketPath = json["media_file"]["public_bucket_path"];
// const media_api_base_url = GetCdnOrigin();
// const media_url = `${media_api_base_url}${bucketPath}`;
// return media_url;
// }
+235
View File
@@ -0,0 +1,235 @@
// import { bool } from "@techstark/opencv-js";
import { API_TARGETS } from "./enums/Api.js";
type NonNullableObject<T extends object> = NonNullable<T>;
export interface ApiResponse<T, P = undefined> {
success: boolean;
errorMessage?: string;
data?: T;
pagination?: P;
}
export class ApiManager {
ApiTargets: Record<string, string> = {};
constructor() {
this.ApiTargets = {
BaseApi: API_TARGETS.BASE_API,
GoggleApi: API_TARGETS.GOOGLE_API,
FunnelApi: API_TARGETS.FUNNEL_API,
CdnApi: API_TARGETS.CDN_API,
GravatarApi: API_TARGETS.GRAVATAR_API,
};
}
public async fetch<B, T>(
endpoint: string,
{
method,
query,
body,
}: {
method: string;
query?: Record<string, string | boolean | number | undefined>;
body?: B;
},
): Promise<T> {
const queryInString =
query &&
Object.entries(query).reduce(
(allOptions, [key, value]) => {
if (!value) {
return allOptions;
}
allOptions[key] = value.toString();
return allOptions;
},
{} as Record<string, string>,
);
const endpointWithQueries = queryInString
? endpoint + "?" + new URLSearchParams(queryInString)
: endpoint;
const bodyInString = JSON.stringify(body);
const response = await fetch(endpointWithQueries, {
method,
headers: {
Accept: "application/json",
"Content-Type": "application/json",
},
credentials: "include",
body: bodyInString,
});
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
return response.json();
}
public async fetchMultipartFormData<T>(
endpoint: string,
{
method,
body,
}: {
method: string;
body: FormData;
},
): Promise<T> {
const response = await fetch(endpoint, {
method,
headers: {
Accept: "application/json",
},
credentials: "include",
body: body,
});
return response.json();
}
protected get<T>({
endpoint,
query,
}: {
endpoint: string;
query?: Record<string, string | boolean | number | undefined>;
}): Promise<T> {
return this.fetch<null, T>(endpoint, { method: "GET", query });
}
protected post<B, T>({
endpoint,
query,
body,
}: {
endpoint: string;
query?: Record<string, string | boolean | number | undefined>;
body?: B;
}): Promise<T> {
return this.fetch<B, T>(endpoint, {
method: "POST",
query,
body,
});
}
protected delete<B, T>({
endpoint,
query,
body,
}: {
endpoint: string;
query?: Record<string, string | boolean | number | undefined>;
body?: B;
}): Promise<T> {
return this.fetch<B, T>(endpoint, {
method: "DELETE",
query,
body,
});
}
protected async postFormVideo<T>({
endpoint,
formRecord,
uuid,
blob,
blobFileName,
}: {
endpoint: string;
formRecord: Record<string, string>;
uuid: string;
blob?: Blob | File;
blobFileName?: string;
}): Promise<T> {
const formData = new FormData();
formData.append("uuid_idempotency_token", uuid);
Object.entries(formRecord).forEach(([key, value]) => {
formData.append(key, value);
});
if (blob && blobFileName) {
formData.append("video", blob, blobFileName);
} else if (blob) {
formData.append("video", blob);
}
return this.fetchMultipartFormData<T>(endpoint, {
method: "POST",
body: formData,
});
}
protected async postForm<T>({
endpoint,
formRecord,
uuid,
blob,
blobFileName,
}: {
endpoint: string;
formRecord: Record<string, string>;
uuid: string;
blob?: Blob | File;
blobFileName?: string;
}): Promise<T> {
const formData = new FormData();
formData.append("uuid_idempotency_token", uuid);
Object.entries(formRecord).forEach(([key, value]) => {
formData.append(key, value);
});
if (blob && blobFileName) {
formData.append("file", blob, blobFileName);
} else if (blob) {
formData.append("file", blob);
}
return this.fetchMultipartFormData<T>(endpoint, {
method: "POST",
body: formData,
});
}
protected camelToSnakeCase(str: string) {
return str.replace(/([a-z0])([A-Z])/g, "$1_$2").toLowerCase();
}
protected parseQueryValues(
params: Record<string, string | string[] | boolean | number | undefined>,
): Record<string, string> {
return Object.entries(params).reduce(
(allParams, [key, value]) => {
if (!value) {
return allParams;
}
const snakeKey = this.camelToSnakeCase(key);
if (Array.isArray(value)) {
return { ...allParams, [snakeKey]: value.join(",") };
}
return { ...allParams, [snakeKey]: value.toString() };
},
{} as Record<string, string>,
);
}
protected parseBodyValues<T extends object, B extends object>(
params: NonNullableObject<T>,
): B {
return Object.entries(params).reduce((allParams, [key, value]) => {
if (!value) {
return allParams;
}
const snakeKey = this.camelToSnakeCase(key);
if (Array.isArray(value)) {
return { ...allParams, [snakeKey]: value };
}
return { ...allParams, [snakeKey]: value };
}, {} as B);
}
}

Some files were not shown because too many files have changed in this diff Show More