Compare commits

...
Author SHA1 Message Date
Tanish Manku ad7a0bcc7f Change the save button disabled logic 2025-05-20 16:14:11 -07:00
Tanish Manku 9bc3a57390 Add a bone json helper to save and load bones 2025-05-20 16:13:44 -07:00
Tanish Manku 109a9dd1a3 Add a rigData field to save json 2025-05-20 16:13:24 -07:00
Tanish Manku b6807a2a3d Use fetch proxy from tauri to save scene 2025-05-16 17:38:16 -07:00
Tanish Manku 4d738c0384 Add userData back to objects on load 2025-05-16 17:37:09 -07:00
Tanish Manku 7227156711 Bump up editor version 2025-05-16 17:36:55 -07:00
Brandon Thomas a438dd13dc add Google Analytics GA4 2025-05-12 18:10:23 -04:00
Tanish Manku f7b76a7abe Add manual mouse support in Safari and AppleWebkit (#606) 2025-05-12 15:02:56 -07:00
Tanish Manku 9435af3e2a Change the relative pathing from base config (#605) 2025-05-12 14:23:09 -07:00
Tanish Manku 77efda1d66 Make 2D and 3D mount at the same time (#604) 2025-05-12 14:15:02 -07:00
Brandon Thomas fdc27b6ccd surface non-onboarding; do not retry non-onboarded requests 2025-05-12 17:00:25 -04:00
Brandon Thomas e3abd500db Explicit CSP on Mac instead of wildcards; this works on production Mac but may not fix Windows (#603) 2025-05-12 15:32:49 -04:00
Brandon Thomas ee4de4126d timeouts for uploads / remix (#602) 2025-05-12 12:26:41 -04:00
Brandon Thomas 4810ed30ae cache sora stats 2025-05-11 04:41:51 -04:00
Brandon Thomas 0b6593a063 fix preferences 2025-05-11 04:00:25 -04:00
Brandon Thomas b0e2dce140 fix event listeners for toasts; extra sounds (#601) 2025-05-11 03:41:14 -04:00
Brandon Thomas c4f401923b various sora / generate improvements (#600) 2025-05-11 01:02:00 -04:00
Brandon Thomas 24559a3554 username error 2025-05-09 19:00:28 -04:00
ArEnScandMichael Chung 5a26aed728 The bug was due to forggeting to bind this to the snapShotOfCurrentFr… (#599)
fix the generation problem

---------

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-09 18:07:55 -04:00
Tanish Manku 95773c1c23 Point 3D tsconfig to the right subdirectory (#598) 2025-05-09 14:27:13 -07:00
Michael Chung 2d01073434 Remove tab selector 2025-05-09 13:56:00 -04:00
BFlat 1c62092d94 fix 3d resizing bug (#597) 2025-05-09 13:13:53 -04:00
bflatastic 5e22e17dfb popover width fix 2025-05-09 22:49:05 +07:00
bflatastic 611da775a9 bug fix: add image fix 2025-05-09 22:23:42 +07:00
bflatastic 6fc3f73579 artcraft block color update 2025-05-09 22:10:53 +07:00
BFlat 562cb99c51 2d editor fixes (#596)
- left toolbar now updates properly
- modals open properly
- styling fixes
- adjusted canvas center positioning to accommodate top header bar
2025-05-09 21:53:58 +07:00
bflatastic 67d280df84 remove old demo modal 2025-05-09 20:38:05 +07:00
8ccc8927a9 Feature/merge 2d 3d diverge (#593)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
Co-authored-by: Tanish Manku <tanish2k09.dev@gmail.com>
2025-05-08 23:46:30 -04:00
Brandon Thomas a21e7331cc persist window position to disk and restore it 2025-05-08 22:54:58 -04:00
Brandon Thomas 0f0f8eb39c window size heuristic to keep dev tools from resizing window 2025-05-08 22:32:02 -04:00
Brandon Thomas b64ed411b1 artcraft account login/logout (#595) 2025-05-08 21:33:38 -04:00
ArEnScandMichael Chung a155b419e1 Fix 3d editor menu buttons hide the scene saving and loading. (#594)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-05-08 18:53:04 -04:00
Brandon Thomas e6177005df never block the prompt box 2025-05-08 17:59:32 -04:00
Brandon Thomas 1c78e17565 events for enqueue success / fail + more sounds and toasts (#592) 2025-05-08 17:43:12 -04:00
Brandon Thomas 0769bfad28 sora logout handler 2025-05-08 14:00:14 -04:00
BFlat 183946a943 Bflat/settings pane update (#591) 2025-05-08 17:40:54 +07:00
BFlat d43ee0f6c9 login modal and settings panel update (#590)
- login now flows automatically smoothly to the next step after login
- settings modal: misc/audio panes styling and toggle switch
- account settings disconnect sora account button

remaining TODO: 
- link logout sora endpoint
2025-05-08 14:21:01 +07:00
Brandon Thomas eec59c218d add login success event 2025-05-08 01:07:33 -04:00
Brandon Thomas 5c1e72f8f4 rename package 2025-05-07 23:59:44 -04:00
Brandon Thomas d91243b1f5 success and failure toasts; less discord info spam 2025-05-07 23:47:30 -04:00
Brandon Thomas 83cecad280 build script for mac 2025-05-07 23:13:23 -04:00
Brandon Thomas 06313f54a5 update settings panes (#588) 2025-05-07 22:52:47 -04:00
Brandon Thomas ef3ca88054 sound playing restored 2025-05-07 22:02:38 -04:00
Brandon Thomas cfcf7ab5aa refactor events: centrally located in rust for better discoverability 2025-05-07 21:15:13 -04:00
Brandon Thomas 4cc215495a slower polling 2025-05-07 19:11:19 -04:00
Brandon Thomas f1d14ab73f better scripts for unix development 2025-05-07 17:59:49 -04:00
Brandon Thomas 5cb8f5b93a cloudflare 524 2025-05-07 17:35:38 -04:00
Brandon Thomas 9decf8bf88 fix windows builds 2025-05-07 15:35:28 -04:00
Brandon Thomas 63a7f99c53 fix windows build issue 2025-05-07 14:55:48 -04:00
Brandon Thomas 9dd16f2e80 windows production build script 2025-05-07 14:42:40 -04:00
Brandon Thomas abe2d2a093 Windows scripts for running dev mode 2025-05-07 14:31:07 -04:00
bflatastic 552d5553c0 uncommented login modal and added temp next button 2025-05-07 23:15:59 +07:00
BFlat 0d33358ead upload flow update (#587)
- changed and moved the main upload asset button to the left sidebar
above the categories for better visibility
- added individual upload buttons to all the category pages except
skybox - they open the same upload modal but sepcific to its category
only.
- flow update: once an upload succeeds the upload modal closes itself
and automatically reopens the asset modal to the mine tab and previously
uploaded category (so users don't have to have a hard time finding or
navigating to it)
2025-05-07 16:18:45 +07:00
Brandon Thomas 908ff43be3 fix sora / chatgpt login, enhance login flow (#586) 2025-05-07 03:33:53 -04:00
430 changed files with 24952 additions and 4869 deletions
@@ -17,7 +17,15 @@ pub struct AppPreferencesPayload {
/// Play sounds on events.
pub play_sounds: bool,
/// Key pointing to file; defined in the frontend code.
/// Defined for enqueue since image enqueue can be async
pub enqueue_success_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
/// Defined for enqueue since image enqueue can be async
pub enqueue_failure_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
pub generation_success_sound: Option<String>,
@@ -25,6 +33,7 @@ pub struct AppPreferencesPayload {
pub generation_failure_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
#[deprecated]
pub generation_enqueue_sound: Option<String>,
}
@@ -52,6 +61,8 @@ async fn get_prefs(app_prefs: &AppPreferencesManager) -> AnyhowResult<AppPrefere
Ok(AppPreferencesPayload {
preferred_download_directory: prefs.preferred_download_directory,
play_sounds: prefs.play_sounds,
enqueue_success_sound: prefs.enqueue_success_sound,
enqueue_failure_sound: prefs.enqueue_failure_sound,
generation_success_sound: prefs.generation_success_sound,
generation_failure_sound: prefs.generation_failure_sound,
generation_enqueue_sound: prefs.generation_enqueue_sound,
@@ -29,9 +29,10 @@ pub enum ValueType {
pub enum PreferenceName {
PreferredDownloadDirectory,
PlaySounds,
EnqueueSuccessSound,
EnqueueFailureSound,
GenerationSuccessSound,
GenerationFailureSound,
GenerationEnqueueSound,
}
#[derive(Serialize)]
@@ -85,6 +86,16 @@ async fn update_prefs(
return Err(anyhow!("Invalid value: {:?}", request.value)),
}
}
PreferenceName::EnqueueSuccessSound => {
prefs.enqueue_success_sound = request.value
.map(|val| string_value(&val))
.transpose()?;
}
PreferenceName::EnqueueFailureSound => {
prefs.enqueue_failure_sound = request.value
.map(|val| string_value(&val))
.transpose()?;
}
PreferenceName::GenerationSuccessSound => {
prefs.generation_success_sound = request.value
.map(|val| string_value(&val))
@@ -95,11 +106,6 @@ async fn update_prefs(
.map(|val| string_value(&val))
.transpose()?;
}
PreferenceName::GenerationEnqueueSound => {
prefs.generation_enqueue_sound = request.value
.map(|val| string_value(&val))
.transpose()?;
}
}
app_prefs.set_clone(&prefs)?;
@@ -0,0 +1,132 @@
use serde::Serialize;
use std::error::Error;
use std::str::FromStr;
/// Easy to use Result<T, E> type.
pub type CommandResult<SuccessPayload, ErrType, ErrPayload> =
Result<CommandSuccessResponseWrapper<SuccessPayload>, CommandErrorResponseWrapper<ErrType, ErrPayload>>;
/// No inner payloads for this type. Just strings as messages.
pub type SimpleCommandResult = CommandResult<(), (), ()>;
/// Statuses for successful commands.
#[derive(Serialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum CommandSuccessStatus {
Success, // 200
}
/// Statuses for errorful commands.
#[derive(Serialize, Clone, Debug)]
#[serde(rename_all = "snake_case")]
pub enum CommandErrorStatus {
BadRequest, // 400
Unauthorized, // 401
TooManyRequests, // 429
ServerError, // 500
}
#[derive(Serialize, Debug)]
pub struct CommandSuccessResponseWrapper<T: Serialize> {
/// Both "success" and "error" types have a `status` field for the frontend.
pub status: CommandSuccessStatus,
#[serde(skip_serializing_if = "Option::is_none")]
pub success_message: Option<String>,
#[serde(skip_serializing_if = "Option::is_none")]
pub payload: Option<T>,
}
#[derive(Serialize, Debug)]
pub struct CommandErrorResponseWrapper<ErrType: Serialize, ErrPayload: Serialize> {
/// Both "success" and "error" types have a `status` field for the frontend.
pub status: CommandErrorStatus,
#[serde(skip_serializing_if = "Option::is_none")]
pub error_message: Option<String>,
/// Represents a "type" of error, preferably an enum without payloads.
/// Preferably this serializes to a string.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_type: Option<ErrType>,
/// Represents a full error details object or payload.
/// This could be a structure, message, or whatever.
#[serde(skip_serializing_if = "Option::is_none")]
pub error_details: Option<ErrPayload>,
}
impl CommandSuccessResponseWrapper<()> {
pub fn empty() -> Self {
CommandSuccessResponseWrapper {
status: CommandSuccessStatus::Success,
success_message: None,
payload: None,
}
}
}
/// NB: This is a hack because of `()` (a.k.a. the "unit" type) and `Serialize` trait coherence and overlap.
/// We cannot currently implement a trait for `()` and generic `Serialize` because `()` itself implements
/// `Serialize`. Rust's `"specialization"`, which would let us account for this, is an unstable, nightly-only
/// feature. So instead we manually mark all types other than `()` with `SerializeMarker` as a hack to
/// distinguish the two.
trait SerializeMarker : Serialize {}
// NB: () is Serialize! So we can't broadly implement just `T: Serialize`.
impl From<()> for CommandSuccessResponseWrapper<()> {
fn from(_: ()) -> Self {
CommandSuccessResponseWrapper {
status: CommandSuccessStatus::Success,
success_message: None,
payload: None,
}
}
}
/// NB: Note the `"+ SerializeMarker"` bound here. This is a hack to distinguish between `()` and other types
/// for which we will manually mark as `SerializeMarker`.
impl <T> From<T> for CommandSuccessResponseWrapper<T> where T: Serialize + SerializeMarker {
fn from(val: T) -> Self {
CommandSuccessResponseWrapper {
status: CommandSuccessStatus::Success,
success_message: None,
payload: Some(val),
}
}
}
impl From<&str> for CommandSuccessResponseWrapper<String> {
fn from(msg: &str) -> Self {
CommandSuccessResponseWrapper {
status: CommandSuccessStatus::Success,
success_message: Some(msg.to_string()),
payload: None,
}
}
}
impl From<&str> for CommandErrorResponseWrapper<(), ()> {
fn from(value: &str) -> Self {
CommandErrorResponseWrapper {
status: CommandErrorStatus::BadRequest, // NB: Default to 500 error.
error_message: Some(value.to_string()),
error_type: None,
error_details: None,
}
}
}
#[cfg(test)]
mod tests {
#[test]
fn test_empty_tuple_serialize() {
let response = super::CommandSuccessResponseWrapper::<()>::from(());
let serialized = serde_json::to_string(&response).unwrap();
// NB: This tests that the blanket "Serialize" impl doesn't produce a "null payload" field.
assert_eq!(serialized, r#"{"status":"success"}"#);
}
}
@@ -1,4 +1,5 @@
pub mod app_preferences;
pub mod command_response_wrapper;
pub mod flip_image;
pub mod platform_info_command;
pub mod sora;
@@ -0,0 +1,114 @@
use std::ops::Add;
use chrono::{DateTime, TimeDelta, Utc};
use crate::commands::app_preferences::get_app_preferences_command::AppPreferencesPayload;
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use errors::AnyhowResult;
use log::{error, info};
use once_cell::sync::Lazy;
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::list_media::list_media::list_media;
use serde_derive::Serialize;
use tauri::{AppHandle, State};
const CACHE_PERIOD : TimeDelta = TimeDelta::milliseconds(1000 * 60 * 5); // 5 minutes
#[derive(Serialize)]
#[serde(rename_all = "snake_case")]
pub struct CheckSoraSessionCommand {
pub state: SoraSessionState,
pub maybe_account_email: Option<String>,
}
#[derive(Serialize)]
#[serde(rename_all = "snake_case")]
pub enum SoraSessionState{
#[serde(rename = "not_set_up")]
NotSetUp,
#[serde(rename = "expired_or_error")]
ExpiredOrError,
#[serde(rename = "valid")]
Valid,
}
#[tauri::command]
pub async fn check_sora_session_command(
app: AppHandle,
sora_creds_manager: State<'_, SoraCredentialManager>,
) -> Result<CheckSoraSessionCommand, CheckSoraSessionCommand> {
info!("check_sora_session_command called");
match do_check(&sora_creds_manager).await {
Ok(state) => Ok(state),
Err(err) => {
error!("Error checking session state: {:?}", err);
Err(CheckSoraSessionCommand {
state: SoraSessionState::ExpiredOrError,
maybe_account_email: None,
})
}
}
}
async fn do_check(
sora_creds_manager: &SoraCredentialManager,
) -> AnyhowResult<CheckSoraSessionCommand> {
let mut creds = match sora_creds_manager.get_credentials()? {
Some(creds) => creds,
None => {
return Ok(CheckSoraSessionCommand {
state: SoraSessionState::NotSetUp,
maybe_account_email: None,
});
}
};
let mut needs_upgrade_or_check = false;
let stats = sora_creds_manager.get_credential_stats()?;
match stats.last_consecutive_credential_success {
None => {
info!("Session never checked, needs upgrade or status checking.");
needs_upgrade_or_check= true;
}
Some(last_success) => {
let now = chrono::Utc::now();
let expires = last_success.add(CACHE_PERIOD);
if now.gt(&expires) {
info!("Session success expired, needs upgrade or status checking.");
needs_upgrade_or_check = true;
}
}
}
let mut upgraded = false;
if needs_upgrade_or_check {
info!("Attempting to upgrade session...");
upgraded = maybe_upgrade_or_renew_session(&mut creds).await?;
info!("Attempting to check session...");
list_media(&creds).await?;
sora_creds_manager.record_credential_success()?;
}
if upgraded {
info!("Saving upgraded session to manager and disk...");
sora_creds_manager.set_credentials(&creds)?;
sora_creds_manager.persist_all_to_disk()?;
}
let maybe_account_email = creds
.jwt_bearer_token
.map(|token| token.jwt_claims().email.clone());
Ok(CheckSoraSessionCommand {
state: SoraSessionState::Valid,
maybe_account_email,
})
}
@@ -1,3 +1,5 @@
pub mod check_sora_session_command;
pub mod open_sora_login_command;
pub mod sora_image_generation_command;
pub mod sora_image_remix_command;
pub mod sora_logout_command;
@@ -1,31 +1,21 @@
use crate::threads::sora_session_login_thread::{LOGIN_WINDOW_NAME, SORA_LOGIN_URL};
use crate::utils::clear_all_webview_cookies::clear_all_webview_cookies;
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use errors::AnyhowResult;
use log::{error, info};
use once_cell::sync::Lazy;
use std::time::Duration;
use tauri::{AppHandle, Manager, Url, WebviewUrl, WebviewWindowBuilder, WindowBuilder};
use tokio::time::sleep_until;
pub const START_URL_STR: &str = "https://storyteller.ai/";
pub static START_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(START_URL_STR).expect("URL should parse")
});
pub const SORA_HOMEPAGE_URL_STR: &str = "https://sora.com/";
pub static SORA_HOMEPAGE_URL : Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_HOMEPAGE_URL_STR).expect("URL should parse")
});
use tauri::{AppHandle, State};
#[tauri::command]
pub async fn open_sora_login_command(
app: AppHandle,
app_data_root: State<'_, AppDataRoot>,
sora_creds_manager: State<'_, SoraCredentialManager>,
) -> Result<String, String> {
info!("open_login_command called");
do_open_login(&app)
do_open_login(&app, &app_data_root, &sora_creds_manager)
.await
.map_err(|err| {
error!("Error opening login: {:?}", err);
@@ -35,37 +25,14 @@ pub async fn open_sora_login_command(
Ok("result".to_string())
}
async fn do_open_login(app: &AppHandle) -> AnyhowResult<()> {
async fn do_open_login(
app: &AppHandle,
app_data_root: &AppDataRoot,
sora_creds_manager: &SoraCredentialManager,
) -> AnyhowResult<()> {
info!("Building login window...");
let url = WebviewUrl::External(START_URL.clone());
let window = WebviewWindowBuilder::new(app, LOGIN_WINDOW_NAME, url)
//.user_agent(openai_sora_client::credentials::USER_AGENT)
.always_on_top(true)
.title("Login to OpenAI")
.center()
.resizable(true)
.visible(true)
.closable(true)
.focused(true)
.devtools(true)
.build()?;
for (name, webview) in window.webviews() {
if name != LOGIN_WINDOW_NAME {
continue;
}
clear_all_webview_cookies(&webview)?;
// NB: We're starting to get Cloudflare protection screens. Let's try to avoid.
webview.navigate(SORA_HOMEPAGE_URL.clone())?;
tokio::time::sleep(Duration::from_millis(100)).await;
webview.navigate(SORA_LOGIN_URL.clone())?;
break;
}
open_sora_login_window(app, app_data_root, sora_creds_manager).await?;
info!("Done.");
Ok(())
@@ -14,8 +14,13 @@ use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_g
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use std::fs::read_to_string;
use std::io::Cursor;
use std::time::Duration;
use tauri::{AppHandle, Manager, State};
const SORA_IMAGE_UPLOAD_TIMEOUT: Duration = Duration::from_millis(1000 * 30); // 30 seconds
const SORA_IMAGE_REMIX_TIMEOUT: Duration = Duration::from_millis(1000 * 30); // 30 seconds
#[tauri::command]
pub async fn sora_image_generation_command(
prompt: &str,
@@ -55,8 +60,13 @@ pub async fn generate_image(
let image_bytes = BASE64_STANDARD.decode(image)?;
let filename = "image.png".to_string();
let response = sora_media_upload_from_bytes(image_bytes, filename, CredentialMigrationRef::New(&creds))
let response = sora_media_upload_from_bytes(
image_bytes,
filename,
CredentialMigrationRef::New(&creds),
Some(SORA_IMAGE_UPLOAD_TIMEOUT))
.await
.map_err(|err| {
error!("Failed to upload image to Sora: {:?}", err);
@@ -72,6 +82,7 @@ pub async fn generate_image(
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: CredentialMigrationRef::New(&creds),
request_timeout: Some(SORA_IMAGE_REMIX_TIMEOUT),
}).await
.map_err(|err| {
error!("Failed to call Sora image generation: {:?}", err);
@@ -1,3 +1,7 @@
use crate::commands::command_response_wrapper::{CommandErrorResponseWrapper, CommandErrorStatus, CommandResult};
use crate::events::sendable_event_trait::SendableEvent;
use crate::events::sora::sora_image_enqueue_failure_event::SoraImageEnqueueFailureEvent;
use crate::events::sora::sora_image_enqueue_success_event::SoraImageEnqueueSuccessEvent;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::state::sora::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
@@ -8,7 +12,7 @@ use crate::utils::get_url_file_extension::get_url_file_extension;
use crate::utils::simple_http_download::simple_http_download;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use errors::AnyhowResult;
use errors::{AnyhowError, AnyhowResult};
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, ImageReader};
use log::{debug, error, info};
@@ -21,23 +25,21 @@ use openai_sora_client::requests::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::requests::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::requests::upload::upload_media_from_file::sora_media_upload_from_file;
use openai_sora_client::sora_error::SoraError;
use serde_derive::{Deserialize, Serialize};
use std::fs::read_to_string;
use std::io::Cursor;
use std::time::Duration;
use storyteller_client::api_error::ApiError;
use storyteller_client::api_error::ApiError::InternalServerError;
use storyteller_client::media_files::get_media_file::get_media_file;
use storyteller_client::utils::api_host::ApiHost;
use tauri::{AppHandle, Emitter, Manager, State};
use tokens::tokens::media_files::MediaFileToken;
use crate::threads::sora_task_polling_thread::SoraImageGenerationComplete;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueFailure {
// TODO: Reason.
}
const SORA_IMAGE_UPLOAD_TIMEOUT: Duration = Duration::from_millis(1000 * 30); // 30 seconds
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueSuccess {
}
const SORA_IMAGE_REMIX_TIMEOUT: Duration = Duration::from_millis(1000 * 30); // 30 seconds
#[derive(Deserialize)]
pub struct SoraImageRemixCommand {
@@ -56,6 +58,20 @@ pub struct SoraImageRemixCommand {
pub maybe_number_of_samples: Option<u32>,
}
#[derive(Serialize, Debug)]
#[serde(rename_all = "snake_case")]
pub enum SoraImageRemixErrorType {
/// Generic server error
ServerError,
/// The user is sending too many requests
TooManyConcurrentTasks,
/// The user needs to create a Sora account
SoraUsernameNotYetCreated,
/// The Sora service is having problems. Try again soon.
SoraIsHavingProblems,
}
#[tauri::command]
pub async fn sora_image_remix_command(
app: AppHandle,
@@ -63,7 +79,7 @@ pub async fn sora_image_remix_command(
app_data_root: State<'_, AppDataRoot>,
sora_creds_manager: State<'_, SoraCredentialManager>,
sora_task_queue: State<'_, SoraTaskQueue>,
) -> Result<String, String> {
) -> CommandResult<(), SoraImageRemixErrorType, ()> {
info!("image_generation_command called; scene media token: {:?}, additional images: {:?}",
request.snapshot_media_token, request.maybe_additional_images);
@@ -72,23 +88,83 @@ pub async fn sora_image_remix_command(
let result = generate_image(request, &app_data_root, &sora_creds_manager, &sora_task_queue).await;
if let Err(err) = result {
error!("error: {:?}", err);
match result {
Err(err) => {
error!("error: {:?}", err);
let result = app.emit("sora-image-enqueue-failure", SoraImageEnqueueFailure{});
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
let event = SoraImageEnqueueFailureEvent {};
let result = event.send(&app);
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
}
let mut status = CommandErrorStatus::ServerError;
let mut error_type = SoraImageRemixErrorType::ServerError;
let mut error_message = "A server error occurred. Please try again. If it continues, please tell our staff about the problem.";
match (err) {
InnerError::SoraError(SoraError::TooManyConcurrentTasks) => {
error_type = SoraImageRemixErrorType::TooManyConcurrentTasks;
status = CommandErrorStatus::ServerError;
error_message = "You already have work in progress. Please wait for it to finish.";
},
InnerError::SoraError(SoraError::SoraUsernameNotYetCreated) => {
error_type = SoraImageRemixErrorType::SoraUsernameNotYetCreated;
status = CommandErrorStatus::BadRequest;
error_message = "You need to create a username on Sora.com to continue.";
},
InnerError::SoraError(SoraError::BadGateway(_) | SoraError::CloudFlareTimeout(_)) => {
error_type = SoraImageRemixErrorType::SoraIsHavingProblems;
status = CommandErrorStatus::ServerError;
error_message = "Sora is having problems. Please wait a moment and then retry.";
}
_ => {},
}
Err(CommandErrorResponseWrapper {
status,
error_message: Some(error_message.to_string()),
error_type: Some(error_type),
error_details: None,
})
}
return Err("there was an error".to_string());
}
Ok(_) => {
let event = SoraImageEnqueueSuccessEvent {};
let result = event.send(&app);
let result = app.emit("sora-image-enqueue-success", SoraImageEnqueueSuccess{});
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
}
if let Err(err) = result {
error!("Failed to emit event: {:?}", err);
}
Ok("success".to_string())
Ok(().into())
}
}
}
#[derive(Debug)]
enum InnerError {
SoraError(SoraError),
AnyhowError(AnyhowError),
StorytellerApiError(ApiError),
}
impl From<AnyhowError> for InnerError {
fn from(value: AnyhowError) -> Self {
Self::AnyhowError(value)
}
}
impl From<SoraError> for InnerError {
fn from(value: SoraError) -> Self {
Self::SoraError(value)
}
}
impl From<ApiError> for InnerError {
fn from(value: ApiError) -> Self {
Self::StorytellerApiError(value)
}
}
pub async fn generate_image(
@@ -96,7 +172,7 @@ pub async fn generate_image(
app_data_root: &AppDataRoot,
sora_creds_manager: &SoraCredentialManager,
sora_task_queue: &SoraTaskQueue,
) -> AnyhowResult<()> {
) -> Result<(), InnerError> {
let response = get_media_file(&ApiHost::Storyteller, &request.snapshot_media_token).await?;
@@ -135,6 +211,7 @@ pub async fn generate_image(
image_upload_from_file_with_session_auto_renew(ImageUploadFromFileAutoRenewRequest {
file_path,
credentials: &creds,
request_timeout: Some(SORA_IMAGE_UPLOAD_TIMEOUT),
}).await?;
if let Some(new_creds) = maybe_new_credentials {
@@ -159,6 +236,7 @@ pub async fn generate_image(
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: &creds,
request_timeout: Some(SORA_IMAGE_REMIX_TIMEOUT),
}).await?;
if let Some(new_creds) = maybe_new_creds {
@@ -0,0 +1,39 @@
use crate::commands::app_preferences::get_app_preferences_command::AppPreferencesPayload;
use crate::commands::command_response_wrapper::{CommandResult, CommandSuccessResponseWrapper, SimpleCommandResult};
use crate::state::app_preferences::app_preferences_manager::AppPreferencesManager;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::windows::sora_login_window::open_sora_login_window::open_sora_login_window;
use errors::{AnyhowError, AnyhowResult};
use log::{error, info};
use once_cell::sync::Lazy;
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::requests::list_media::list_media::list_media;
use serde_derive::Serialize;
use std::fmt::format;
use tauri::{AppHandle, State};
#[tauri::command]
pub async fn sora_logout_command(
sora_creds_manager: State<'_, SoraCredentialManager>,
) -> SimpleCommandResult {
info!("sora_logout_command called");
if let Err(err) = sora_creds_manager.purge_credentials_from_disk() {
error!("Error purging credentials from disk: {:?}", err);
return Err("error purging credentials from disk".into());
}
if let Err(err) = sora_creds_manager.clear_credentials() {
error!("Error clearing credentials from memory: {:?}", err);
return Err("error clearing credentials from memory".into());
}
if let Err(err) = sora_creds_manager.reset_credential_stats() {
error!("Error resetting credential stats: {:?}", err);
return Err("error resetting credential stats".into());
}
Ok(().into())
}
+4 -1
View File
@@ -1 +1,4 @@
pub mod notification_event;
pub mod notification_event;
pub mod sendable_event_error;
pub mod sendable_event_trait;
pub mod sora;
@@ -0,0 +1,31 @@
use std::error::Error;
use std::fmt::{Display, Formatter};
#[derive(Debug)]
pub enum SendableEventError {
TauriError(tauri::Error),
AnyhowError(anyhow::Error),
}
impl Error for SendableEventError {}
impl Display for SendableEventError {
fn fmt(&self, f: &mut Formatter<'_>) -> std::fmt::Result {
match self {
Self::TauriError(err) => write!(f, "SendableEventError::TauriError: {}", err),
Self::AnyhowError(err) => write!(f, "SendableEventError::AnyhowError: {}", err),
}
}
}
impl From<anyhow::Error> for SendableEventError {
fn from(err: anyhow::Error) -> Self {
SendableEventError::AnyhowError(err)
}
}
impl From<tauri::Error> for SendableEventError {
fn from(err: tauri::Error) -> Self {
SendableEventError::TauriError(err)
}
}
@@ -0,0 +1,21 @@
use log::info;
use crate::events::sendable_event_error::SendableEventError;
use serde::Serialize;
use tauri::{AppHandle, Emitter};
/// Implement this on a Serializable object to be able to send it as an event to the frontend.
pub trait SendableEvent : Serialize {
/// This is the name of the event that the frontend subscribes to.
const FRONTEND_EVENT_NAME: &'static str;
/// Default implementation of send().
/// This serializes and sends the event to the frontend.
fn send(&self, app: &AppHandle) -> Result<(), SendableEventError> {
info!("Sending event to frontend: {}", Self::FRONTEND_EVENT_NAME);
let result = app.emit(Self::FRONTEND_EVENT_NAME, self);
if let Err(err) = result {
return Err(SendableEventError::from(err));
}
Ok(())
}
}
@@ -0,0 +1,5 @@
pub mod sora_image_enqueue_failure_event;
pub mod sora_image_enqueue_success_event;
pub mod sora_image_generation_complete_event;
pub mod sora_image_generation_failed_event;
pub mod sora_login_success_event;
@@ -0,0 +1,11 @@
use crate::events::sendable_event_trait::SendableEvent;
use serde_derive::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueFailureEvent {
// TODO: Reason.
}
impl SendableEvent for SoraImageEnqueueFailureEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-enqueue-failure";
}
@@ -0,0 +1,11 @@
use crate::events::sendable_event_trait::SendableEvent;
use serde_derive::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageEnqueueSuccessEvent {
// TODO: Token, etc.
}
impl SendableEvent for SoraImageEnqueueSuccessEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-enqueue-success";
}
@@ -0,0 +1,12 @@
use serde_derive::Serialize;
use tokens::tokens::media_files::MediaFileToken;
use crate::events::sendable_event_trait::SendableEvent;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationCompleteEvent {
pub media_file_token: MediaFileToken,
}
impl SendableEvent for SoraImageGenerationCompleteEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-generation-complete";
}
@@ -0,0 +1,12 @@
use serde_derive::Serialize;
use crate::events::sendable_event_trait::SendableEvent;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationFailedEvent {
pub prompt: String,
}
impl SendableEvent for SoraImageGenerationFailedEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-image-generation-failed";
}
@@ -0,0 +1,11 @@
use crate::events::sendable_event_trait::SendableEvent;
use serde_derive::Serialize;
#[derive(Debug, Clone, Serialize)]
pub struct SoraLoginSuccessEvent {
// TODO: Reason.
}
impl SendableEvent for SoraLoginSuccessEvent {
const FRONTEND_EVENT_NAME: &'static str = "sora-login-success";
}
+24 -6
View File
@@ -3,30 +3,33 @@ pub mod events;
pub mod state;
pub mod threads;
pub mod utils;
pub mod windows;
use tauri::Manager;
use crate::commands::flip_image::flip_image;
use crate::commands::app_preferences::get_app_preferences_command::get_app_preferences_command;
use crate::commands::app_preferences::update_app_preference_command::update_app_preferences_command;
use crate::commands::flip_image::flip_image;
use crate::commands::platform_info_command::platform_info_command;
use crate::commands::sora::check_sora_session_command::check_sora_session_command;
use crate::commands::sora::open_sora_login_command::open_sora_login_command;
use crate::commands::sora::sora_image_generation_command::sora_image_generation_command;
use crate::commands::sora::sora_image_remix_command::sora_image_remix_command;
use crate::commands::sora::sora_logout_command::sora_logout_command;
use crate::state::app_preferences::app_preferences_manager::load_app_preferences_or_default;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::state::sora::sora_task_queue::SoraTaskQueue;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::threads::discord_presence_thread::discord_presence_thread;
use crate::threads::main_window_thread::main_window_thread::main_window_thread;
use crate::threads::sora_session_login_thread::sora_session_login_thread;
use crate::threads::sora_task_polling_thread::sora_task_polling_thread;
use crate::utils::webview_unsafe::webview_unsafe_for_app;
use crate::state::app_preferences::app_preferences_manager::load_app_preferences_or_default;
use tauri_plugin_http;
use tauri_plugin_dialog;
use tauri_plugin_http;
use tauri_plugin_log::Target;
use tauri_plugin_log::TargetKind;
@@ -108,7 +111,20 @@ pub fn run() {
}
}
tauri::async_runtime::spawn(sora_session_login_thread(app_2, app_data_root_2, sora_creds_manager_2));
match MainWindowPosition::from_filesystem_configs(&app_data_root_3) {
Ok(None) => {}
Ok(Some(pos)) => {
println!("Moving window to: {:?}", pos);
let result = pos.apply_to_main_window(&app);
if let Err(err) = result {
eprintln!("Could not set window position: {:?}", err);
}
}
Err(err) => {
eprintln!("Failed to read window position from disk: {:?}", err);
}
}
tauri::async_runtime::spawn(main_window_thread(app_3, app_data_root_3, storyteller_creds_manager_2));
tauri::async_runtime::spawn(sora_task_polling_thread(app_4, app_data_root_4, sora_creds_manager_3, storyteller_creds_manager_3, sora_task_queue_2));
tauri::async_runtime::spawn(discord_presence_thread());
@@ -120,13 +136,15 @@ pub fn run() {
.manage(sora_creds_manager)
.manage(sora_task_queue)
.invoke_handler(tauri::generate_handler![
check_sora_session_command,
flip_image,
get_app_preferences_command,
update_app_preferences_command,
open_sora_login_command,
platform_info_command,
sora_image_generation_command,
sora_image_remix_command,
sora_logout_command,
update_app_preferences_command,
])
.run(tauri::generate_context!("tauri.conf.json"))
.expect("error while running tauri application");
@@ -9,6 +9,14 @@ pub struct AppPreferences {
/// Play sounds on events.
pub play_sounds: bool,
/// Key pointing to file; defined in the frontend code.
/// Defined for enqueue since image enqueue can be async
pub enqueue_success_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
/// Defined for enqueue since image enqueue can be async
pub enqueue_failure_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
pub generation_success_sound: Option<String>,
@@ -17,6 +25,7 @@ pub struct AppPreferences {
pub generation_failure_sound: Option<String>,
/// Key pointing to file; defined in the frontend code.
#[deprecated]
pub generation_enqueue_sound: Option<String>,
}
@@ -26,7 +35,9 @@ impl Default for AppPreferences {
preferred_download_directory: PreferredDownloadDirectory::System(SystemDownloadDirectory::Downloads),
play_sounds: true,
// NB: These are defined in the frontend.
generation_success_sound: Some("flower".to_string()),
enqueue_success_sound: Some("done".to_string()),
enqueue_failure_sound: Some("spike_throw".to_string()),
generation_success_sound: Some("special_flower".to_string()),
generation_failure_sound: Some("crumble".to_string()),
generation_enqueue_sound: Some("done".to_string()),
}
@@ -136,6 +136,10 @@ impl AppDataRoot {
pub fn get_window_size_config_file(&self) -> PathBuf {
self.path.join("window_size.json")
}
pub fn get_window_position_config_file(&self) -> PathBuf {
self.path.join("window_position.json")
}
}
// eg. /home/bob/artcraft, /Users/bob/artcraft, or C:\Users\Alice\artcraft
@@ -0,0 +1,73 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::windows::main_window::constants::MAIN_WINDOW_NAME;
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
use std::fs::OpenOptions;
use std::io::Write;
use tauri::{AppHandle, Manager, PhysicalPosition, Window};
#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub struct MainWindowPosition {
pub x: i32,
pub y: i32,
}
impl MainWindowPosition {
pub fn new(x: i32, y: i32) -> Self {
Self { x, y }
}
pub fn from_main_window(app: &AppHandle) -> AnyhowResult<Self> {
let windows = app.windows();
windows.get(MAIN_WINDOW_NAME)
.map(|window| Self::from_window(window))
.unwrap_or_else(|| Err(anyhow::anyhow!("Main window not found")))
}
pub fn from_window(window: &Window) -> AnyhowResult<Self> {
let position = window.outer_position()?;
Ok(Self {
x: position.x,
y: position.y,
})
}
pub fn from_filesystem_configs(app_data_root: &AppDataRoot) -> AnyhowResult<Option<Self>> {
let filename = app_data_root.get_window_position_config_file();
if !filename.exists() {
return Ok(None);
}
let contents = std::fs::read_to_string(filename)?;
let pos : MainWindowPosition = serde_json::from_str(&contents)?;
Ok(Some(pos))
}
pub fn apply_to_main_window(&self, app: &AppHandle) -> AnyhowResult<()> {
let windows = app.windows();
let window = windows.get(MAIN_WINDOW_NAME)
.ok_or_else(|| anyhow::anyhow!("Main window not found"))?;
window.set_position(PhysicalPosition::new(self.x, self.y))?;
Ok(())
}
pub fn persist_to_filesystem(&self, app_data_root: &AppDataRoot) -> AnyhowResult<()> {
let filename = app_data_root.get_window_position_config_file();
let json = serde_json::to_string(self)?;
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(filename)?;
file.write_all(json.as_bytes())?;
file.flush()?;
Ok(())
}
pub fn to_physical_position(&self) -> PhysicalPosition<i32> {
PhysicalPosition::new(self.x, self.y)
}
pub fn matches_physical_position(&self, pos: &PhysicalPosition<i32>) -> bool {
self.x == pos.x && self.y == pos.y
}
}
@@ -1,12 +1,12 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use crate::windows::main_window::constants::MAIN_WINDOW_NAME;
use errors::AnyhowResult;
use serde_derive::{Deserialize, Serialize};
use std::fs::OpenOptions;
use std::io::Write;
use tauri::{AppHandle, Manager, PhysicalSize, Window};
const MAIN_WINDOW_NAME: &str = "main";
#[derive(Serialize, Deserialize, Clone, Debug, Eq, PartialEq)]
pub struct MainWindowSize {
pub width: u32,
@@ -26,7 +26,7 @@ impl MainWindowSize {
}
pub fn from_window(window: &Window) -> AnyhowResult<Self> {
let size = window.inner_size()?;
let size = best_window_size_heuristic(window)?;
Ok(Self {
width: size.width,
height: size.height,
+1
View File
@@ -2,6 +2,7 @@ pub mod app_preferences;
pub mod app_startup_time;
pub mod data_dir;
pub mod expanduser;
pub mod main_window_position;
pub mod main_window_size;
pub mod os_platform;
pub mod sora;
@@ -2,4 +2,5 @@ pub mod read_sora_credentials_from_disk;
pub mod read_sora_credentials_from_disk_with_upgrade;
pub mod sora_credential_holder;
pub mod sora_credential_manager;
pub mod sora_credential_stats;
pub mod sora_task_queue;
@@ -1,21 +1,29 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::read_sora_credentials_from_disk::read_sora_credentials_from_disk;
use crate::state::sora::sora_credential_holder::SoraCredentialHolder;
use crate::state::sora::sora_credential_stats::SoraCredentialStats;
use anyhow::anyhow;
use errors::AnyhowResult;
use log::{error, info, warn};
use memory_store::clone_cell::CloneCell;
use openai_sora_client::creds::sora_cookies::SoraCookies;
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
use openai_sora_client::creds::sora_sentinel::SoraSentinel;
use openai_sora_client::requests::sentinel_refresh::generate::token::generate_token;
use std::fs::OpenOptions;
use std::io::Write;
use openai_sora_client::creds::sora_cookies::SoraCookies;
use openai_sora_client::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
use std::path::Path;
use std::{fs, io};
#[derive(Clone)]
pub struct SoraCredentialManager {
holder: SoraCredentialHolder,
app_data_root: AppDataRoot,
/// This is meant to help debug credential issues, keep credentials cached, fresh, etc.
// TODO: Put this into the same structure as the credentials and also persist to disk.
stats: CloneCell<SoraCredentialStats>,
}
impl SoraCredentialManager {
@@ -33,9 +41,36 @@ impl SoraCredentialManager {
Self {
holder,
app_data_root: app_data_root.clone(),
stats: CloneCell::with_owned(SoraCredentialStats::new()),
}
}
pub fn record_credential_success(&self) -> AnyhowResult<()> {
let mut stats = self.stats.get_clone()?;
stats.record_credential_success();
self.stats.set_owned(stats)?;
Ok(())
}
pub fn record_credential_failure(&self) -> AnyhowResult<()> {
let mut stats = self.stats.get_clone()?;
stats.record_credential_failure();
self.stats.set_owned(stats)?;
Ok(())
}
pub fn reset_credential_stats(&self) -> AnyhowResult<()> {
self.stats.set_owned(SoraCredentialStats::new())
}
pub fn get_credential_stats(&self) -> AnyhowResult<SoraCredentialStats> {
self.stats.get_clone()
}
pub fn set_credential_stats(&self, stats: SoraCredentialStats) -> AnyhowResult<()> {
self.stats.set_owned(stats)
}
pub fn set_credentials(&self, creds: &SoraCredentialSet) -> AnyhowResult<()> {
self.holder.set_credentials(creds)
}
@@ -45,6 +80,31 @@ impl SoraCredentialManager {
Ok(())
}
/// This is meant to be infallible.
pub fn purge_credentials_from_disk(&self) -> Result<(), io::Error> {
let creds_dir = self.app_data_root.credentials_dir();
remove_file_if_exists(creds_dir.get_sora_cookie_file_path())?;
remove_file_if_exists(creds_dir.get_sora_bearer_token_file_path())?;
remove_file_if_exists(creds_dir.get_sora_sentinel_file_path())?;
Ok(())
}
/// This is meant to be infallible.
pub fn try_purge_credentials_from_disk(&self) {
let creds_dir = self.app_data_root.credentials_dir();
if let Err(err) = fs::remove_file(creds_dir.get_sora_cookie_file_path()) {
error!("Failed to remove cookie file: {:?}", err);
}
if let Err(err) = fs::remove_file(creds_dir.get_sora_bearer_token_file_path()) {
error!("Failed to remove bearer token file: {:?}", err);
}
if let Err(err) = fs::remove_file(creds_dir.get_sora_sentinel_file_path()) {
error!("Failed to remove sentinel file: {:?}", err);
}
}
pub fn get_credentials(&self) -> AnyhowResult<Option<SoraCredentialSet>> {
self.holder.get_credentials()
}
@@ -155,3 +215,11 @@ fn persist_sentinel_to_disk(sentinel: &SoraSentinel, app_data_root: &AppDataRoot
Ok(())
}
fn remove_file_if_exists<P: AsRef<Path>>(path: P) -> Result<(), io::Error> {
let p = path.as_ref();
if p.exists() && p.is_file() {
fs::remove_file(p)?;
}
Ok(())
}
@@ -0,0 +1,47 @@
/// This is meant to track 401s and other credential failures, not 500s.
#[derive(Clone)]
pub struct SoraCredentialStats {
pub last_credential_failure: Option<chrono::DateTime<chrono::Utc>>,
pub last_credential_success: Option<chrono::DateTime<chrono::Utc>>,
pub credential_success_count: u32,
pub credential_failure_count: u32,
/// This resets on each failure
pub last_consecutive_credential_success: Option<chrono::DateTime<chrono::Utc>>,
/// This resets on each failure
pub consecutive_credential_success_count: u32,
/// This resets on each success
pub consecutive_credential_failure_count: u32,
}
impl SoraCredentialStats {
pub fn new() -> Self {
Self {
last_credential_failure: None,
last_credential_success: None,
last_consecutive_credential_success: None,
credential_success_count: 0,
credential_failure_count: 0,
consecutive_credential_success_count: 0,
consecutive_credential_failure_count: 0,
}
}
pub fn record_credential_success(&mut self) {
let now = chrono::Utc::now();
self.last_credential_success = Some(now);
self.last_consecutive_credential_success = Some(now);
self.credential_success_count += 1;
self.consecutive_credential_success_count += 1;
self.consecutive_credential_failure_count = 0;
}
pub fn record_credential_failure(&mut self) {
let now = chrono::Utc::now();
self.last_credential_failure = Some(now);
self.last_consecutive_credential_success = None;
self.credential_failure_count += 1;
self.consecutive_credential_failure_count += 1;
self.consecutive_credential_success_count = 0;
}
}
@@ -1,7 +1,7 @@
use anyhow::anyhow;
use discord_rich_presence::{DiscordIpc, DiscordIpcClient};
use errors::AnyhowResult;
use log::info;
use log::{debug, info};
/// Our Discord App ID. Not sure if this is a secret.
const DISCORD_APP_ID : &str = "1366596912593113138";
@@ -15,13 +15,13 @@ pub async fn discord_presence_thread() -> ! {
}
async fn discord_main_loop(mut client: DiscordIpcClient) -> AnyhowResult<()> {
info!("Connecting Discord IPC client");
info!("(Re)-connecting Discord IPC client...");
client.connect()
.map_err(|err| anyhow!("Error connecting to Discord IPC: {:?}", err))?;
loop {
info!("Notifying discord...");
debug!("Notifying discord presence API...");
let assets = discord_rich_presence::activity::Assets::new()
.large_image("https://storyteller.ai/android-chrome-512x512.png")
@@ -37,6 +37,6 @@ async fn discord_main_loop(mut client: DiscordIpcClient) -> AnyhowResult<()> {
.assets(assets))
.map_err(|err| anyhow!("Error setting Discord activity: {:?}", err))?;
tokio::time::sleep(std::time::Duration::from_millis(30_000)).await;
tokio::time::sleep(std::time::Duration::from_millis(60_000)).await;
}
}
@@ -1,13 +1,15 @@
use crate::state::app_startup_time::AppStartupTime;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::threads::main_window_thread::persist_storyteller_cookies_task::persist_storyteller_cookies_task;
use crate::threads::main_window_thread::persist_window_position_task::persist_window_position_task;
use crate::threads::main_window_thread::persist_window_resize_task::persist_window_resize_task;
use errors::AnyhowResult;
use log::{error, info};
use memory_store::clone_slot::CloneSlot;
use tauri::{AppHandle, Manager, Webview, Window};
use crate::state::app_startup_time::AppStartupTime;
const MAIN_WINDOW_NAME : &str = "main";
@@ -18,6 +20,7 @@ pub async fn main_window_thread(
) -> ! {
// TODO: Move these into some kind of dependency injection framework
let window_size_slot: CloneSlot<MainWindowSize> = CloneSlot::empty();
let window_pos_slot: CloneSlot<MainWindowPosition> = CloneSlot::empty();
let app_startup_time = AppStartupTime::new();
loop {
@@ -29,6 +32,7 @@ pub async fn main_window_thread(
&app_data_root,
&storyteller_creds_manager,
&window_size_slot,
&window_pos_slot,
&app_startup_time,
).await;
if let Err(err) = result {
@@ -46,10 +50,12 @@ pub async fn handle_main_window(
app_data_root: &AppDataRoot,
storyteller_creds_manager: &StorytellerCredentialManager,
window_size_slot: &CloneSlot<MainWindowSize>,
window_pos_slot: &CloneSlot<MainWindowPosition>,
app_startup_time: &AppStartupTime,
) -> AnyhowResult<()> {
loop {
log_errors(persist_window_resize_task(window, app_data_root, window_size_slot).await);
log_errors(persist_window_position_task(window, app_data_root, window_pos_slot).await);
log_errors(persist_storyteller_cookies_task(app, storyteller_creds_manager, app_startup_time).await);
tokio::time::sleep(std::time::Duration::from_millis(1_000)).await;
}
@@ -1,3 +1,4 @@
pub mod main_window_thread;
pub mod persist_storyteller_cookies_task;
pub mod persist_window_position_task;
pub mod persist_window_resize_task;
@@ -1,7 +1,6 @@
use crate::state::app_startup_time::AppStartupTime;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::storyteller::storyteller_credential_manager::StorytellerCredentialManager;
use crate::threads::sora_session_login_thread::LOGIN_WINDOW_NAME;
use anyhow::anyhow;
use chrono::TimeDelta;
use errors::AnyhowResult;
@@ -0,0 +1,34 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_position::MainWindowPosition;
use crate::state::main_window_size::MainWindowSize;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use errors::AnyhowResult;
use log::info;
use memory_store::clone_slot::CloneSlot;
use tauri::Window;
pub async fn persist_window_position_task(
window: &Window,
app_data_root: &AppDataRoot,
window_pos_slot: &CloneSlot<MainWindowPosition>,
) -> AnyhowResult<()> {
let current_window_position = MainWindowPosition::from_window(window)?;
let current_physical_pos = window.outer_position()?;
let mut save_size_to_disk = true;
if let Ok(Some(old_size)) = window_pos_slot.get_clone() {
if old_size.matches_physical_position(&current_physical_pos) {
save_size_to_disk = false;
}
}
if save_size_to_disk {
info!("Saving window position configs to disk...");
current_window_position.persist_to_filesystem(app_data_root)?;
window_pos_slot.set_clone(&current_window_position)?;
}
Ok(())
}
@@ -1,5 +1,6 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::main_window_size::MainWindowSize;
use crate::utils::best_window_size_heuristic::best_window_size_heuristic;
use errors::AnyhowResult;
use log::info;
use memory_store::clone_slot::CloneSlot;
@@ -12,7 +13,7 @@ pub async fn persist_window_resize_task(
) -> AnyhowResult<()> {
let current_window_size = MainWindowSize::from_window(window)?;
let current_physical_size = window.inner_size()?;
let current_physical_size = best_window_size_heuristic(window)?;
let mut save_size_to_disk = true;
@@ -1,4 +1,3 @@
pub mod discord_presence_thread;
pub mod main_window_thread;
pub mod sora_session_login_thread;
pub mod sora_task_polling_thread;
@@ -1,178 +0,0 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_holder::SoraCredentialHolder;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::utils::cookies::sora_webview_cookies::get_all_sora_cookies_as_string;
use crate::utils::sora::initialize_sora_jwt_bearer_token::initialize_sora_jwt_bearer_token;
use anyhow::anyhow;
use chrono::{DateTime, NaiveDateTime, TimeDelta};
use errors::AnyhowResult;
use log::{error, info, warn};
use once_cell::sync::Lazy;
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::utils::has_session_cookie::{has_session_cookie, SessionCookiePresence};
use reqwest::Url;
use std::fs;
use std::fs::{read_to_string, OpenOptions};
use std::io::Write;
use std::ops::Sub;
use tauri::{AppHandle, Manager, Webview};
pub const LOGIN_WINDOW_NAME: &str = "login_window";
pub const SORA_LOGIN_URL_STR: &str = "https://sora.com/auth/login?callback_path=/";
pub static SORA_LOGIN_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_LOGIN_URL_STR).expect("URL should parse")
});
pub const SORA_ROOT_URL_STR: &str = "https://sora.com/";
pub static SORA_ROOT_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_ROOT_URL_STR).expect("URL should parse")
});
pub async fn sora_session_login_thread(
app: AppHandle,
app_data_root: AppDataRoot,
sora_creds_manager: SoraCredentialManager
) -> ! {
loop {
for (window_name, webview) in app.webviews() {
if window_name == LOGIN_WINDOW_NAME {
let result = check_login_window(
&webview,
&app_data_root,
&sora_creds_manager,
).await;
if let Err(err) = result {
error!("Error checking login window: {:?}", err);
}
break;
}
}
tokio::time::sleep(std::time::Duration::from_millis(2_000)).await;
}
}
async fn check_login_window(
webview: &Webview,
app_data_root: &AppDataRoot,
sora_credential_manager: &SoraCredentialManager,
) -> AnyhowResult<()> {
clear_browsing_data_on_test_domain(webview)?;
//keep_on_task(webview)?;
extract_cookies_to_file(webview, app_data_root, sora_credential_manager).await?;
//initialize_sora_jwt_bearer_token(app_data_root).await?; // TODO: This only runs once. We need better management.
Ok(())
}
fn keep_on_task(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"auth.openai.com" => {},
"openai.com" => {},
"sora.com" => {},
// Third party SSO
"accounts.google.com" => {},
"accounts.youtube.com" => {},
"login.live.com" => {},
"appleid.apple.com" => {},
_ => {
info!("Non login hostname: {}", hostname);
webview.navigate(SORA_LOGIN_URL.clone())?;
}
}
Ok(())
}
/// This is just so we have a way to clear browsing data.
fn clear_browsing_data_on_test_domain(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"storyteller.ai" => {
info!("Clearing all browsing data...");
webview.clear_all_browsing_data()?;
}
_ => {}
}
Ok(())
}
// TODO(bt,2025-04-07): Heuristic to detect when logged in. Only write when logged in.
async fn extract_cookies_to_file(
webview: &Webview,
app_data_root: &AppDataRoot,
sora_credential_manager: &SoraCredentialManager,
) -> AnyhowResult<()> {
let new_cookies = get_all_sora_cookies_as_string(webview)?.trim().to_string();
if new_cookies.is_empty() {
return Ok(());
}
let session_cookie_presence = has_session_cookie(&new_cookies)
.unwrap_or_else(|err| {
error!("Failed to check for session cookie: {:?}", err);
SessionCookiePresence::MaybePresent
});
info!("Session cookies are: {:?}", session_cookie_presence);
let (should_write_cookies, should_upgrade_session) = match session_cookie_presence {
SessionCookiePresence::Present => (true, true),
SessionCookiePresence::MaybePresent => (true, false),
SessionCookiePresence::Absent => (false, false),
};
if !should_write_cookies {
return Ok(());
}
// TODO(bt): Race conditions ahead.
sora_credential_manager.clear_credentials()?;
let _r = fs::remove_file(app_data_root.get_sora_cookie_file_path());
let _r = fs::remove_file(app_data_root.get_sora_bearer_token_file_path());
let _r = fs::remove_file(app_data_root.get_sora_sentinel_file_path());
let mut new_credentials =
SoraCredentialSet::initialize_with_just_cookies_str(&new_cookies);
if should_upgrade_session {
let _upgraded = maybe_upgrade_or_renew_session(&mut new_credentials).await?;
}
sora_credential_manager.set_credentials(&new_credentials)?;
sora_credential_manager.persist_all_to_disk()?;
Ok(())
}
pub fn read_sora_cookies_from_disk(app_data_root: &AppDataRoot) -> Option<String> {
let cookie_file = app_data_root.get_sora_cookie_file_path();
if !cookie_file.exists() {
return None;
}
match read_to_string(cookie_file) {
Ok(contents) => Some(contents.trim().to_string()),
Err(err) => {
warn!("Failed to read cookie file: {:?}", err);
None
}
}
}
@@ -1,3 +1,6 @@
use crate::events::sendable_event_trait::SendableEvent;
use crate::events::sora::sora_image_generation_complete_event::SoraImageGenerationCompleteEvent;
use crate::events::sora::sora_image_generation_failed_event::SoraImageGenerationFailedEvent;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::data_dir::trait_data_subdir::DataSubdir;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
@@ -18,16 +21,6 @@ use tauri::{AppHandle, Emitter};
use tempdir::TempDir;
use tokens::tokens::media_files::MediaFileToken;
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationComplete {
pub media_file_token: MediaFileToken,
}
#[derive(Debug, Clone, Serialize)]
pub struct SoraImageGenerationFailed {
pub prompt: String,
}
pub async fn sora_task_polling_thread(
app_handle: AppHandle,
app_data_root: AppDataRoot,
@@ -102,9 +95,10 @@ async fn polling_loop(
for task in failed_tasks.iter() {
if sora_task_queue.contains_key(&task.id)? {
app_handle.emit("sora-image-generation-failed", SoraImageGenerationFailed {
let event = SoraImageGenerationFailedEvent {
prompt: task.prompt.clone(),
})?;
};
event.send(&app_handle)?;
}
}
@@ -133,9 +127,11 @@ async fn polling_loop(
info!("Uploaded to API backend: {:?}", result.media_file_token);
app_handle.emit("sora-image-generation-complete", SoraImageGenerationComplete {
media_file_token: result.media_file_token
})?;
let event = SoraImageGenerationCompleteEvent {
media_file_token: result.media_file_token,
};
event.send(&app_handle)?;
}
}
@@ -146,7 +142,7 @@ async fn polling_loop(
sora_task_queue.remove_list(&succeeded_task_ids)?;
tokio::time::sleep(std::time::Duration::from_millis(2_000)).await;
tokio::time::sleep(std::time::Duration::from_millis(6_000)).await;
}
}
@@ -0,0 +1,25 @@
use tauri::{PhysicalSize, Window};
use errors::AnyhowResult;
// A typical height difference when dev tools are open:
// Window inner size = PhysicalSize { width: 3704, height: 732 }
// Window outer size = PhysicalSize { width: 3704, height: 1476 }
const HEIGHT_DIFFERENCE_THRESHOLD: u32 = 300;
/// Get the best window size
pub fn best_window_size_heuristic(window: &Window) -> AnyhowResult<PhysicalSize<u32>> {
// NB: Ordinarily we'd use inner size, as the outer size includes the window decorations.
// However, dev tools contract the inner size, and in the event they're open, we should
// prefer the outer size.
let inner = window.inner_size()?;
let outer = window.outer_size()?;
let height_delta = outer.height.saturating_sub(inner.height);
if height_delta > HEIGHT_DIFFERENCE_THRESHOLD {
// Dev tools are open, use the outer size
Ok(outer)
} else {
// Dev tools are not open, use the inner size
Ok(inner)
}
}
@@ -1 +0,0 @@
pub mod sora_webview_cookies;
@@ -1,26 +0,0 @@
use once_cell::sync::Lazy;
use reqwest::Url;
use tauri::Webview;
use tauri::webview::Cookie;
use errors::AnyhowResult;
const SORA_ROOT_COOKIE_URL_STR: &str = "https://sora.com/";
static SORA_ROOT_COOKIE_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_ROOT_COOKIE_URL_STR).expect("URL should parse")
});
pub fn get_all_sora_cookies(webview: &Webview) -> AnyhowResult<Vec<Cookie>> {
let cookies = webview.cookies_for_url(SORA_ROOT_COOKIE_URL.clone())?;
Ok(cookies)
}
pub fn get_all_sora_cookies_as_string(webview: &Webview) -> AnyhowResult<String> {
let cookies = get_all_sora_cookies(webview)?;
let cookie_string = cookies
.iter()
.map(|cookie| format!("{}={}", cookie.name(), cookie.value()))
.collect::<Vec<String>>()
.join("; ");
Ok(cookie_string)
}
+2 -2
View File
@@ -1,9 +1,9 @@
pub mod best_window_size_heuristic;
pub mod clear_all_webview_cookies;
pub mod cookies;
pub mod get_url_file_extension;
pub mod image;
pub mod large_file_transfer;
pub mod object_store;
pub mod simple_http_download;
pub mod sora;
pub mod webview_unsafe;
pub mod webview_unsafe;
@@ -0,0 +1,4 @@
/// The name of the main application window.
/// This is the Artcraft application UX itself.
pub const MAIN_WINDOW_NAME : &str = "main";
@@ -0,0 +1 @@
pub mod constants;
@@ -0,0 +1,2 @@
pub mod sora_login_window;
pub mod main_window;
@@ -0,0 +1,20 @@
use anyhow::anyhow;
use log::info;
use tauri::Webview;
use errors::AnyhowResult;
// This is just so we have a way to clear browsing data while debugging / building.
pub fn clear_browsing_data_on_test_domain(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"storyteller.ai" => {
info!("Clearing all browsing data...");
webview.clear_all_browsing_data()?;
}
_ => {}
}
Ok(())
}
@@ -0,0 +1,28 @@
use errors::AnyhowResult;
use once_cell::sync::Lazy;
use reqwest::Url;
use tauri::webview::Cookie;
use tauri::WebviewWindow;
const CHAT_GPT_ROOT_COOKIE_URL_STR: &str = "https://chatgpt.com";
static CHAT_GPT_ROOT_COOKIE_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(CHAT_GPT_ROOT_COOKIE_URL_STR).expect("URL should parse")
});
pub fn extract_sora_webview_cookies(webview: &WebviewWindow) -> AnyhowResult<String> {
// NB(bt): Cookies were originally on sora.com, but now they're located on
// the sora.chatgpt.com / *.chatgpt.com domains.
let cookies = get_all_chatgpt_cookies(webview)?;
let cookie_string = cookies
.iter()
.map(|cookie| format!("{}={}", cookie.name(), cookie.value()))
.collect::<Vec<String>>()
.join("; ");
Ok(cookie_string)
}
fn get_all_chatgpt_cookies(webview: &WebviewWindow) -> AnyhowResult<Vec<Cookie>> {
let cookies = webview.cookies_for_url(CHAT_GPT_ROOT_COOKIE_URL.clone())?;
Ok(cookies)
}
@@ -0,0 +1,28 @@
use crate::windows::sora_login_window::open_sora_login_window::SORA_LOGIN_URL;
use anyhow::anyhow;
use errors::AnyhowResult;
use log::info;
use tauri::Webview;
// NB: Probably best not to use this.
pub fn keep_on_login_domains(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"auth.openai.com" => {},
"openai.com" => {},
"sora.com" => {},
// Third party SSO
"accounts.google.com" => {},
"accounts.youtube.com" => {},
"login.live.com" => {},
"appleid.apple.com" => {},
_ => {
info!("Non login hostname: {}", hostname);
webview.navigate(SORA_LOGIN_URL.clone())?;
}
}
Ok(())
}
@@ -0,0 +1,5 @@
pub mod clear_browsing_on_test_domain;
pub mod extract_sora_webview_cookies;
pub mod keep_on_login_domains;
pub mod open_sora_login_window;
pub mod sora_login_thread;
@@ -0,0 +1,84 @@
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::utils::clear_all_webview_cookies::clear_all_webview_cookies;
use crate::windows::sora_login_window::sora_login_thread::sora_login_thread;
use anyhow::anyhow;
use errors::AnyhowResult;
use once_cell::sync::Lazy;
use reqwest::Url;
use std::time::Duration;
use tauri::{AppHandle, Manager, WebviewUrl, WebviewWindowBuilder};
/// Name of the window
pub const LOGIN_WINDOW_NAME: &str = "login_window";
pub const START_URL_STR: &str = "https://google.com";
pub static START_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(START_URL_STR).expect("URL should parse")
});
pub const SORA_HOMEPAGE_URL_STR: &str = "https://sora.com/";
pub static SORA_HOMEPAGE_URL : Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_HOMEPAGE_URL_STR).expect("URL should parse")
});
pub const SORA_LOGIN_URL_STR: &str = "https://chatgpt.com/auth/login?next=%2Fsora%2F";
pub static SORA_LOGIN_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_LOGIN_URL_STR).expect("URL should parse")
});
//pub const SORA_ROOT_URL_STR: &str = "https://sora.com/";
//
//pub static SORA_ROOT_URL: Lazy<Url> = Lazy::new(|| {
// Url::parse(SORA_ROOT_URL_STR).expect("URL should parse")
//});
pub async fn open_sora_login_window(
app: &AppHandle,
app_data_root: &AppDataRoot,
sora_creds_manager: &SoraCredentialManager,
) -> AnyhowResult<()> {
if app.get_window(LOGIN_WINDOW_NAME).is_some() {
return Err(anyhow!("Login window already open"));
}
let url = WebviewUrl::External(START_URL.clone());
let window = WebviewWindowBuilder::new(app, LOGIN_WINDOW_NAME, url)
//.user_agent(openai_sora_client::credentials::USER_AGENT)
.always_on_top(true)
.title("Login to OpenAI's Sora")
.center()
.resizable(true)
.visible(true)
.closable(true)
.min_inner_size(200.0, 800.0)
.focused(true)
.devtools(true)
.build()?;
let webview = window.get_webview(LOGIN_WINDOW_NAME)
.ok_or_else(|| anyhow!("no webview found"))?;
clear_all_webview_cookies(&webview)?;
webview.navigate(SORA_HOMEPAGE_URL.clone())?;
// NB: We're starting to get Cloudflare protection screens. Let's try to avoid.
tokio::time::sleep(Duration::from_millis(100)).await;
webview.navigate(SORA_LOGIN_URL.clone())?;
let app_handle = app.clone();
let app_data_root = app_data_root.clone();
let sora_creds_manager = sora_creds_manager.clone();
let _ = tauri::async_runtime::spawn(async move {
sora_login_thread(app_handle, app_data_root, sora_creds_manager).await;
});
Ok(())
}
@@ -0,0 +1,157 @@
use crate::events::sendable_event_trait::SendableEvent;
use crate::events::sora::sora_login_success_event::SoraLoginSuccessEvent;
use crate::state::data_dir::app_data_root::AppDataRoot;
use crate::state::sora::sora_credential_holder::SoraCredentialHolder;
use crate::state::sora::sora_credential_manager::SoraCredentialManager;
use crate::utils::sora::initialize_sora_jwt_bearer_token::initialize_sora_jwt_bearer_token;
use crate::windows::sora_login_window::extract_sora_webview_cookies::extract_sora_webview_cookies;
use crate::windows::sora_login_window::open_sora_login_window::LOGIN_WINDOW_NAME;
use anyhow::anyhow;
use chrono::{DateTime, NaiveDateTime, TimeDelta};
use errors::AnyhowResult;
use log::{error, info, warn};
use once_cell::sync::Lazy;
use openai_sora_client::creds::sora_credential_set::SoraCredentialSet;
use openai_sora_client::recipes::maybe_upgrade_or_renew_session::maybe_upgrade_or_renew_session;
use openai_sora_client::utils::has_session_cookie::{has_session_cookie, SessionCookiePresence};
use reqwest::Url;
use std::fs;
use std::fs::{read_to_string, OpenOptions};
use std::io::Write;
use std::ops::Sub;
use tauri::{AppHandle, Manager, Webview, WebviewWindow};
pub async fn sora_login_thread(
app: AppHandle,
app_data_root: AppDataRoot,
sora_creds_manager: SoraCredentialManager
) {
loop {
let login_webview_window = match app.get_webview_window(LOGIN_WINDOW_NAME) {
Some(webview) => webview,
None => {
info!("Exit sora login thread.");
return; // NB: Only exit if we don't have the webview.
}
};
let result = check_login_window(
&app,
&login_webview_window,
&app_data_root,
&sora_creds_manager,
).await;
match result {
Err(err) => {
error!("Error checking login window: {:?}", err);
}
Ok(false) => {} // Continue iteration and try again...
Ok(true) => {
info!("Successfully saved cookies from login window. Closing.");
if let Err(err) = login_webview_window.close() {
error!("Error closing login window: {:?}", err);
}
return;
}
}
tokio::time::sleep(std::time::Duration::from_millis(2_000)).await;
}
}
/// Returns true if we can exit.
async fn check_login_window(
app_handle: &AppHandle,
webview_window: &WebviewWindow,
app_data_root: &AppDataRoot,
sora_credential_manager: &SoraCredentialManager,
) -> AnyhowResult<bool> {
/* Login flow looks like this:
1. Start: https://sora.chatgpt.com/
2. Login Start: https://chatgpt.com/auth/login?next=%2Fsora%2F [this is where we start]
3. Login Continue: https://auth.openai.com/log-in
4. SSO / Google Login (etc): https://accounts.google.com/v3/signin/challenge/pwd?[query]
5. Done / Landing: https://sora.chatgpt.com/explore
*/
/*
- Do not use credential manager until the end (we don't load old cookies!)
- Check if we're on the correct domain, if not exit? (or inverse, that we're not in login flow)
*/
let hostname = get_hostname(webview_window)?;
let mut maybe_at_destination = false;
match hostname.as_str() {
"sora.com" | // Old destination
"sora.chatgpt.com" // New destination
=> {
maybe_at_destination = true;
}
// chatgpt.com/auth is also an auth domain
"auth.openai.com" |
"accounts.google.com" |
"accounts.youtube.com" |
"login.live.com" |
"appleid.apple.com"
=> {
// NB: We're in auth flow.
info!("Sora login webview is in auth flow; hostname `{}`.", hostname);
return Ok(false)
}
_ => {}, // We just don't know...
}
let webview_cookies = extract_sora_webview_cookies(webview_window)?.trim().to_string();
let session_cookie_presence = has_session_cookie(&webview_cookies)
.unwrap_or_else(|err| {
error!("Failed to check for session cookie: {:?}", err);
SessionCookiePresence::MaybePresent
});
info!("Sora login webview is at hostname `{}`; cookie status: {:?}.", hostname, session_cookie_presence);
match session_cookie_presence {
SessionCookiePresence::Absent => {
info!("Session cookies are absent.");
return Ok(false);
}
_ => {},
}
let mut new_credentials =
SoraCredentialSet::initialize_with_just_cookies_str(&webview_cookies);
let _upgraded = maybe_upgrade_or_renew_session(&mut new_credentials).await?;
// TODO(bt): Race conditions ahead.
sora_credential_manager.clear_credentials()?;
sora_credential_manager.try_purge_credentials_from_disk();
sora_credential_manager.set_credentials(&new_credentials)?;
sora_credential_manager.persist_all_to_disk()?;
// NB: Event sent to the frontend for the login flow. We shouldn't rely on just this
// alone as it could be brittle if the events aren't caught.
let event = SoraLoginSuccessEvent {};
if let Err(err) = event.send(app_handle) {
error!("Error sending Sora login success event: {:?}", err);
}
Ok(true)
}
fn get_hostname(webview: &WebviewWindow) -> AnyhowResult<String> {
let url = webview.url()?;
let url_hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
Ok(url_hostname)
}
+37 -3
View File
@@ -21,11 +21,45 @@
],
"security": {
"csp": {
"default-src": "*",
"connect-src": "*",
"default-src": [
"'self'",
"'unsafe-eval'",
"'unsafe-inline'",
"blob:",
"data:",
"https://*.fakeyou.com",
"https://*.storyteller.ai",
"https://www.googletagmanager.com",
"ipc://localhost",
"tauri://localhost"
],
"connect-src": [
"'self'",
"'unsafe-eval'",
"'unsafe-inline'",
"blob:",
"data:",
"https://*.fakeyou.com",
"https://*.storyteller.ai",
"https://unpkg.com/detect-gpu@5.0.70/",
"https://www.googletagmanager.com",
"ipc://localhost",
"tauri://localhost"
],
"font-src": "*",
"img-src": "* data: blob: 'unsafe-inline'",
"style-src": "*"
"style-src": [
"'self'",
"'unsafe-eval'",
"'unsafe-inline'",
"blob:",
"data:",
"https://*.fakeyou.com",
"https://*.storyteller.ai",
"https://fonts.googleapis.com",
"ipc://localhost",
"tauri://localhost"
]
}
}
},
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
{"default":{"identifier":"default","description":"enables the default permissions","local":true,"windows":["main"],"permissions":["core:default","log:default","upload:allow-download","upload:default",{"identifier":"http:default","allow":[{"url":"https://*.storyteller.ai"},{"url":"https://*.fakeyou.com"}],"deny":[{"url":"https://private.tauri.app"}]}]}}
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -1,3 +1,4 @@
use std::time::Duration;
use anyhow::anyhow;
use log::{info, warn};
use errors::AnyhowResult;
@@ -7,7 +8,7 @@ use crate::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
use crate::creds::sora_sentinel::SoraSentinel;
use crate::requests::bearer::generate::generate_bearer_with_cookie;
use crate::requests::image_gen::common::{ImageSize, NumImages, SoraImageGenResponse};
use crate::requests::image_gen::image_gen_http_request;
use crate::requests::image_gen::{image_gen_http_request, SoraImageGenError};
use crate::requests::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use crate::requests::sentinel_refresh::generate::token::generate_token;
use crate::sora_error::SoraError;
@@ -18,6 +19,8 @@ pub struct ImageRemixAutoRenewRequest<'a> {
pub image_size: ImageSize,
pub sora_media_tokens: Vec<String>,
pub credentials: &'a SoraCredentialSet,
/// This function can try several different requests. This is the timeout for *individual* requests.
pub request_timeout: Option<Duration>,
}
/// Image request with retry and session auto-renewal.
@@ -29,6 +32,7 @@ pub async fn image_remix_with_session_auto_renew(request: ImageRemixAutoRenewReq
image_size: request.image_size,
sora_media_tokens: request.sora_media_tokens.clone(), // NB: Clone because used again
credentials: CredentialMigrationRef::New(request.credentials),
request_timeout: request.request_timeout,
}).await;
let err = match result {
@@ -43,29 +47,32 @@ pub async fn image_remix_with_session_auto_renew(request: ImageRemixAutoRenewReq
match err {
// We'll fail these requests...
image_gen_http_request::SoraError::TooManyConcurrentTasks(_) => {
SoraImageGenError::TooManyConcurrentTasks(_) => {
return Err(SoraError::TooManyConcurrentTasks);
}
image_gen_http_request::SoraError::GenericError(err) => {
SoraImageGenError::GenericError(err) => {
return Err(SoraError::OtherBadStatus(anyhow!("image remix failed with GenericError: {:?}", err)))
}
image_gen_http_request::SoraError::NetworkError(err) => {
SoraImageGenError::NetworkError(err) => {
// TODO: The underlying type should be a reqwest::Error.
return Err(SoraError::OtherBadStatus(anyhow!("network error: {:?}", err)))
}
SoraImageGenError::UsernameRequired(_err) => {
return Err(SoraError::SoraUsernameNotYetCreated)
}
// We'll retry these requests...
image_gen_http_request::SoraError::SentinelBlock(err) => {
SoraImageGenError::SentinelBlock(err) => {
warn!("Image generation failed due to sentinel block error: {:?}", err);
refresh_sentinel = true;
}
image_gen_http_request::SoraError::TokenExpired(err) => {
SoraImageGenError::TokenExpired(err) => {
warn!("Image generation failed due to token expired error: {:?}", err);
refresh_sentinel = true; // TODO: Not sure what this error is, actually.
refresh_jwt = true;
}
image_gen_http_request::SoraError::InvalidJwt(err) => {
SoraImageGenError::InvalidJwt(err) => {
warn!("Image generation failed due to invalid jwt error: {:?}", err);
refresh_jwt = true;
}
@@ -123,12 +130,13 @@ pub async fn image_remix_with_session_auto_renew(request: ImageRemixAutoRenewReq
image_size: request.image_size,
sora_media_tokens: request.sora_media_tokens,
credentials: CredentialMigrationRef::New(&new_creds),
request_timeout: request.request_timeout,
}).await;
match result {
Ok(response) => Ok((response, Some(new_creds))),
Err(err) => match err {
image_gen_http_request::SoraError::TooManyConcurrentTasks(err) => {
image_gen_http_request::SoraImageGenError::TooManyConcurrentTasks(err) => {
Err(SoraError::TooManyConcurrentTasks)
}
_ => {
@@ -14,12 +14,16 @@ use crate::sora_error::SoraError;
use anyhow::anyhow;
use errors::AnyhowResult;
use log::{error, info, warn};
use std::time::Duration;
pub struct ImageUploadFromBytesAutoRenewRequest<'a> {
pub file_bytes: Vec<u8>,
pub filename: String,
pub mime_type: &'a str,
pub credentials: &'a SoraCredentialSet,
/// This function can try to upload several times. This is the timeout for *individual* requests.
pub request_timeout: Option<Duration>,
}
/// Image upload with retry and session auto-renewal.
@@ -32,6 +36,7 @@ pub async fn image_upload_from_bytes_with_session_auto_renew(
request.file_bytes.clone(), // FIXME(bt): This is horrible, but the client needs to take ownership. :(
request.filename.clone(), // FIXME: Same
CredentialMigrationRef::New(request.credentials),
request.request_timeout,
).await;
let err = match result {
@@ -51,6 +56,7 @@ pub async fn image_upload_from_bytes_with_session_auto_renew(
request.file_bytes,
request.filename,
CredentialMigrationRef::New(&new_creds),
request.request_timeout,
).await?;
Ok((result, Some(new_creds)))
@@ -6,10 +6,14 @@ use crate::requests::upload::upload_media_http_request::{upload_media_http_reque
use crate::sora_error::SoraError;
use log::{info, warn};
use std::path::Path;
use std::time::Duration;
pub struct ImageUploadFromFileAutoRenewRequest<'a, P: AsRef<Path>> {
pub file_path: P,
pub credentials: &'a SoraCredentialSet,
/// This function can try to upload several times. This is the timeout for *individual* requests.
pub request_timeout: Option<Duration>,
}
/// Image upload with retry and session auto-renewal.
@@ -21,10 +25,15 @@ pub async fn image_upload_from_file_with_session_auto_renew<P: AsRef<Path>>(
let result = sora_media_upload_from_file(
request.file_path.as_ref().clone(), // FIXME(bt): This is horrible, but the client needs to take ownership. :(
CredentialMigrationRef::New(request.credentials),
/// This function can try to upload several times. This is the timeout for *individual* requests.
request.request_timeout,
).await;
let err = match result {
Ok(response) => return Ok((response, None)),
// We can't retry some errors.
Err(err @ SoraError::SoraUsernameNotYetCreated) => return Err(err),
// Retry all other errors.
Err(err) => err,
};
@@ -37,6 +46,7 @@ pub async fn image_upload_from_file_with_session_auto_renew<P: AsRef<Path>>(
let result = sora_media_upload_from_file(
request.file_path,
CredentialMigrationRef::New(&new_creds),
request.request_timeout,
).await?;
Ok((result, Some(new_creds)))
@@ -39,11 +39,10 @@ pub async fn list_sora_task_status_with_session_auto_renew(
new_creds = Some(updated_creds);
}
let legacy_creds = new_creds.as_ref()
.unwrap_or_else(|| &args.credentials)
.to_legacy_credentials()?;
let upgraded_creds = new_creds.as_ref()
.unwrap_or_else(|| &args.credentials);
let result = get_image_gen_status(&request, &legacy_creds).await;
let result = get_image_gen_status(&request, &upgraded_creds).await;
let err = match result {
Ok(response) => return Ok((response, new_creds)),
@@ -56,7 +55,7 @@ pub async fn list_sora_task_status_with_session_auto_renew(
info!("Retrying image upload with new credentials...");
let result = get_image_gen_status(&request, &legacy_creds).await?;
let result = get_image_gen_status(&request, &upgraded_creds).await?;
Ok((result, Some(new_creds)))
}
@@ -7,13 +7,14 @@ use chrono::{DateTime, TimeDelta, Utc};
use errors::AnyhowResult;
use log::info;
use std::ops::Sub;
use crate::sora_error::SoraError;
const EXPIRATION_DEADLINE : TimeDelta = TimeDelta::hours(12);
/// Call this at session startup to generate the tokens for the first time.
/// Call this periodically to refresh the JWT as it nears its expiration date.
/// This will not refresh the sentinel token once it's been generated - we can use other flows to accomplish that.
pub async fn maybe_upgrade_or_renew_session(sora_credentials: &mut SoraCredentialSet) -> AnyhowResult<bool> {
pub async fn maybe_upgrade_or_renew_session(sora_credentials: &mut SoraCredentialSet) -> Result<bool, SoraError> {
let mut refresh_jwt = !sora_credentials.jwt_bearer_token.is_some();
let refresh_sentinel = !sora_credentials.sora_sentinel.is_some();
@@ -2,6 +2,7 @@ use crate::credentials::SoraCredentials;
use crate::requests::image_gen::image_gen_status::TaskStatus;
use crate::requests::image_gen::image_gen_status::{get_image_gen_status, StatusRequest, TaskResponse, VideoGenStatusResponse};
use errors::AnyhowResult;
use crate::creds::sora_credential_set::SoraCredentialSet;
pub async fn wait_for_image_gen_status(task_id: &String, credentials: &SoraCredentials, retry_limit: Option<u32>) -> AnyhowResult<TaskResponse> {
let status_request = StatusRequest {
@@ -10,11 +11,13 @@ pub async fn wait_for_image_gen_status(task_id: &String, credentials: &SoraCrede
// before: Some(task_id.clone()),
};
let retry_limit = retry_limit.unwrap_or(10);
let new_creds = SoraCredentialSet::from_legacy_credentials(&credentials)?;
for _ in 0..retry_limit {
tokio::time::sleep(std::time::Duration::from_secs(10)).await;
let status_response = get_image_gen_status(&status_request, credentials).await?;
let status_response = get_image_gen_status(&status_request, &new_creds).await?;
match find_task_response_by_id(&status_response, task_id.clone()) {
Some(task_response) => match TaskStatus::from_str(&task_response.status) {
File diff suppressed because one or more lines are too long
@@ -1,7 +1,9 @@
use crate::creds::credential_migration::CredentialMigrationRef;
use serde_derive::{Deserialize, Serialize};
use thiserror::Error;
use crate::requests::image_gen::image_gen_status::TaskId;
use log::warn;
use serde_derive::{Deserialize, Serialize};
use std::time::Duration;
use thiserror::Error;
const SORA_IMAGE_GEN_URL: &str = "https://sora.com/backend/video_gen";
@@ -9,7 +11,7 @@ const SORA_IMAGE_GEN_URL: &str = "https://sora.com/backend/video_gen";
const USER_AGENT : &str = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/127.0.0.0 Safari/537.36";
#[derive(Error, Debug)]
pub enum SoraError {
pub enum SoraImageGenError {
#[error("Sora token expired: {0}")]
TokenExpired(String),
@@ -27,6 +29,9 @@ pub enum SoraError {
#[error("Network error: {0}")]
NetworkError(String),
#[error("Sora username required: {0}")]
UsernameRequired(String),
}
#[derive(Serialize, Debug)]
@@ -127,6 +132,7 @@ pub enum SoraErrorCode {
InvalidJwt,
TokenExpired,
TooManyConcurrentTasks,
UsernameRequired,
Unknown(String),
}
@@ -147,7 +153,11 @@ impl std::fmt::Display for RawSoraErrorInner {
}
/// Don't expose the internal request implementation as there are only a few "correct" ways to call the API.
pub (crate) async fn image_gen_http_request(sora_request: RawSoraImageGenRequest, credentials: CredentialMigrationRef<'_>) -> Result<RawSoraResponse, SoraError> {
pub (crate) async fn image_gen_http_request(
sora_request: RawSoraImageGenRequest,
credentials: CredentialMigrationRef<'_>,
request_timeout: Option<Duration>,
) -> Result<RawSoraResponse, SoraImageGenError> {
let client = reqwest::Client::new();
let mut cookie;
@@ -161,78 +171,88 @@ pub (crate) async fn image_gen_http_request(sora_request: RawSoraImageGenRequest
// TODO(bt,2025-04-23): We're using a Sora payload error in place of application state error. Surface this differently.
sentinel = creds.sentinel.as_ref()
.map(|sentinel| sentinel.to_string())
.ok_or(SoraError::SentinelBlock("Sentinel is required for image generation.".to_string()))?;
.ok_or(SoraImageGenError::SentinelBlock("Sentinel is required for image generation.".to_string()))?;
}
CredentialMigrationRef::New(creds) => {
cookie = creds.cookies.to_string();
// TODO(bt,2025-04-23): We're using a Sora payload error in place of application state error. Surface this differently.
authorization_header = creds.jwt_bearer_token.as_ref()
.ok_or(SoraError::InvalidJwt("JWT bearer is required for image generation".to_string()))?
.ok_or(SoraImageGenError::InvalidJwt("JWT bearer is required for image generation".to_string()))?
.to_authorization_header_value();
// TODO(bt,2025-04-23): We're using a Sora payload error in place of application state error. Surface this differently.
sentinel = creds.sora_sentinel.as_ref()
.map(|sentinel| sentinel.get_sentinel().to_string())
.ok_or(SoraError::SentinelBlock("Sentinel is required for image generation.".to_string()))?;
.ok_or(SoraImageGenError::SentinelBlock("Sentinel is required for image generation.".to_string()))?;
}
}
let http_request = client.post(SORA_IMAGE_GEN_URL)
let mut http_request = client.post(SORA_IMAGE_GEN_URL)
.header("User-Agent", USER_AGENT)
.header("Cookie", &cookie)
.header("Authorization", &authorization_header)
.header("Content-Type", "application/json")
.header("OpenAI-Sentinel-Token", &sentinel);
if let Some(timeout) = request_timeout {
http_request = http_request.timeout(timeout);
}
let http_request = http_request.json(&sora_request).build()
.map_err(|e| SoraError::NetworkError(e.to_string()))?;
.map_err(|e| SoraImageGenError::NetworkError(e.to_string()))?;
let response = client.execute(http_request)
.await
.map_err(|e| SoraError::NetworkError(e.to_string()))?;
.map_err(|e| SoraImageGenError::NetworkError(e.to_string()))?;
let status = response.status();
let response_body = &response.text().await
.map_err(|e| SoraError::NetworkError(e.to_string()))?;
.map_err(|e| SoraImageGenError::NetworkError(e.to_string()))?;
if status != reqwest::StatusCode::OK {
warn!("Sora image generation failure. Raw response: {:?}", response_body);
let error_response: RawSoraErrorResponse = serde_json::from_str(response_body)
.map_err(|e| SoraError::GenericError(format!("Failed to parse error response: {}", e)))?;
.map_err(|e| SoraImageGenError::GenericError(format!("Failed to parse error response: {}", e)))?;
// Check for specific error codes
if let Some(code) = &error_response.error.code {
match code {
SoraErrorCode::TokenExpired => {
return Err(SoraError::TokenExpired(error_response.error.message));
return Err(SoraImageGenError::TokenExpired(error_response.error.message));
}
SoraErrorCode::SentinelBlock => {
return Err(SoraError::SentinelBlock(error_response.error.message));
return Err(SoraImageGenError::SentinelBlock(error_response.error.message));
}
SoraErrorCode::TooManyConcurrentTasks => {
return Err(SoraError::TooManyConcurrentTasks(error_response.error.message));
return Err(SoraImageGenError::TooManyConcurrentTasks(error_response.error.message));
}
SoraErrorCode::Unknown(_) => {
return Err(SoraError::GenericError(error_response.error.message));
return Err(SoraImageGenError::GenericError(error_response.error.message));
}
SoraErrorCode::InvalidJwt => {
return Err(SoraError::InvalidJwt(error_response.error.message));
return Err(SoraImageGenError::InvalidJwt(error_response.error.message));
}
SoraErrorCode::UsernameRequired => {
return Err(SoraImageGenError::UsernameRequired(error_response.error.message));
}
}
}
return Err(SoraError::GenericError(error_response.error.message));
return Err(SoraImageGenError::GenericError(error_response.error.message));
}
let response = serde_json::from_str(response_body)
.map_err(|e| SoraError::GenericError(format!("Failed to parse success response: {}", e)))?;
.map_err(|e| SoraImageGenError::GenericError(format!("Failed to parse success response: {}", e)))?;
Ok(response)
}
#[cfg(test)]
mod tests {
use errors::AnyhowResult;
use crate::requests::image_gen::image_gen_http_request::RawSoraResponse;
use errors::AnyhowResult;
#[test]
fn deserialize_task_id() -> AnyhowResult<()> {
@@ -6,7 +6,9 @@ use reqwest::Url;
use serde_derive::{Deserialize, Serialize};
use std::fmt::{Debug, Display};
use std::{fs::File, io::Write, path::Path};
use log::debug;
use log::{debug, warn};
use crate::creds::sora_credential_set::SoraCredentialSet;
use crate::creds::sora_jwt_bearer_token::SoraJwtBearerToken;
const SORA_STATUS_URL: &str = "https://sora.com/backend/video_gen";
@@ -171,7 +173,7 @@ pub struct InpaintItem {
pub r#type: String,
pub preset_id: Option<String>,
pub generation_id: Option<String>,
pub upload_media_id: String,
pub upload_media_id: Option<String>,
pub frame_index: i32,
pub source_start_frame: i32,
}
@@ -199,7 +201,16 @@ impl StatusRequest {
}
/// Gets the status of image generation tasks from Sora API
pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &SoraCredentials) -> Result<VideoGenStatusResponse, SoraError> {
pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &SoraCredentialSet) -> Result<VideoGenStatusResponse, SoraError> {
let bearer_header = match credentials.jwt_bearer_token.as_ref() {
Some(bearer) => bearer.to_authorization_header_value(),
None => {
warn!("No JWT bearer for getting image gen status!");
return Err(SoraError::NoBearerTokenAvailable)
},
};
let client = reqwest::Client::new();
let mut url = reqwest::Url::parse(SORA_STATUS_URL)
@@ -216,8 +227,8 @@ pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &
let http_request = client.get(url)
.header("User-Agent", USER_AGENT)
.header("Cookie", &credentials.cookie)
.header("Authorization", credentials.authorization_header_value())
.header("Cookie", credentials.cookies.as_str())
.header("Authorization", bearer_header)
.header("Content-Type", "application/json");
let http_request = http_request.build()?;
@@ -235,9 +246,15 @@ pub async fn get_image_gen_status(status_request: &StatusRequest, credentials: &
return Err(SoraError::AnyhowError(anyhow::anyhow!("The status request failed; status = {:?}, message = {:?}", status, response_body)));
}
let response: VideoGenStatusResponse = serde_json::from_str(response_body)?;
let maybe_response = serde_json::from_str::<VideoGenStatusResponse>(response_body);
Ok(response)
match maybe_response {
Ok(response) => Ok(response),
Err(err) => {
warn!("Failed to parse status response: {:?}", err);
Err(SoraError::JsonErrorWithBody(err, response_body.to_string()))
}
}
}
pub async fn save_generations_to_dir(generations: &[Generation], dir: &str) -> AnyhowResult<()> {
@@ -275,6 +292,7 @@ mod tests {
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
use crate::creds::sora_credential_set::SoraCredentialSet;
#[ignore]
#[tokio::test]
@@ -297,10 +315,12 @@ mod tests {
let bearer = bearer.trim().to_string();
let creds = SoraCredentials { bearer_token: bearer, cookie, sentinel: Some(sentinel) };
let new_creds = SoraCredentialSet::from_legacy_credentials(&creds)?;
// Get the task status for a specific task
// let response = get_image_gen_status(&StatusRequest::new(None, Some("task_01jr9yvpfyetx9r7qvvx38scna".to_string())), &creds).await?;
let response = get_image_gen_status(&StatusRequest::new(Some(50), None), &creds).await?;
let response = get_image_gen_status(&StatusRequest::new(Some(50), None), &new_creds).await?;
println!("task_id: {}", response.task_responses[0].id);
let task_id = response.task_responses[0].id.clone();
@@ -1,5 +1,5 @@
pub (crate) mod image_gen_http_request;
pub use image_gen_http_request::SoraError;
pub use image_gen_http_request::SoraImageGenError;
pub mod common;
pub mod sora_image_gen_remix;
pub mod sora_image_gen_simple;
@@ -1,7 +1,8 @@
use log::info;
use crate::creds::credential_migration::CredentialMigrationRef;
use crate::requests::image_gen::common::{ImageSize, NumImages, SoraImageGenResponse};
use crate::requests::image_gen::image_gen_http_request::{image_gen_http_request, InpaintItem, InpaintItemType, OperationType, RawSoraImageGenRequest, SoraError, VideoGenType};
use crate::requests::image_gen::image_gen_http_request::{image_gen_http_request, InpaintItem, InpaintItemType, OperationType, RawSoraImageGenRequest, SoraImageGenError, VideoGenType};
use log::info;
use std::time::Duration;
pub struct SoraImageGenRemixRequest<'a> {
pub prompt: String,
@@ -9,11 +10,12 @@ pub struct SoraImageGenRemixRequest<'a> {
pub image_size: ImageSize,
pub sora_media_tokens: Vec<String>,
pub credentials: CredentialMigrationRef<'a>,
pub request_timeout: Option<Duration>,
}
/// The "remix" commands let you supply additional images as context.
/// Sora "media tokens" of previously uploaded images must be supplied.
pub async fn sora_image_gen_remix(request: SoraImageGenRemixRequest<'_>) -> Result<SoraImageGenResponse, SoraError> {
pub async fn sora_image_gen_remix(request: SoraImageGenRemixRequest<'_>) -> Result<SoraImageGenResponse, SoraImageGenError> {
let args = RawSoraImageGenRequest {
r#type: VideoGenType::ImageGen,
operation: OperationType::Remix,
@@ -39,7 +41,7 @@ pub async fn sora_image_gen_remix(request: SoraImageGenRemixRequest<'_>) -> Resu
// TODO: Replace this with debug level logging in the future.
info!("Sending remix request: {:?}", args);
let result = image_gen_http_request(args, request.credentials).await?;
let result = image_gen_http_request(args, request.credentials, request.request_timeout).await?;
Ok(SoraImageGenResponse {
task_id: result.id,
@@ -80,6 +82,7 @@ mod tests {
image_size: ImageSize::Square,
sora_media_tokens: vec!["media_01jqyhrz4detyvtzwp2p4j63ad".to_string()],
credentials: CredentialMigrationRef::Legacy(&creds),
request_timeout: None,
}).await?;
println!("task_id: {}", response.task_id);
@@ -1,14 +1,15 @@
use crate::credentials::SoraCredentials;
use crate::creds::credential_migration::CredentialMigrationRef;
use crate::requests::image_gen::common::{ImageSize, NumImages, SoraImageGenResponse};
use crate::requests::image_gen::image_gen_http_request::{image_gen_http_request, OperationType, RawSoraImageGenRequest, VideoGenType};
use errors::AnyhowResult;
use crate::creds::credential_migration::CredentialMigrationRef;
use std::time::Duration;
pub struct SoraImageGenSimpleRequest<'a> {
pub prompt: String,
pub num_images: NumImages,
pub image_size: ImageSize,
pub credentials: CredentialMigrationRef<'a>,
pub request_timeout: Option<Duration>,
}
pub async fn sora_image_gen_simple(args: SoraImageGenSimpleRequest<'_>) -> AnyhowResult<SoraImageGenResponse> {
@@ -24,7 +25,7 @@ pub async fn sora_image_gen_simple(args: SoraImageGenSimpleRequest<'_>) -> Anyho
};
// TODO: Error handling.
let result = image_gen_http_request(sora_request, args.credentials).await?;
let result = image_gen_http_request(sora_request, args.credentials, args.request_timeout).await?;
Ok(SoraImageGenResponse {
task_id: result.id,
@@ -34,12 +35,12 @@ pub async fn sora_image_gen_simple(args: SoraImageGenSimpleRequest<'_>) -> Anyho
#[cfg(test)]
mod tests {
use crate::credentials::SoraCredentials;
use crate::creds::credential_migration::CredentialMigrationRef;
use crate::requests::image_gen::common::{ImageSize, NumImages};
use crate::requests::image_gen::sora_image_gen_simple::{sora_image_gen_simple, SoraImageGenSimpleRequest};
use errors::AnyhowResult;
use std::fs::read_to_string;
use testing::test_file_path::test_file_path;
use crate::creds::credential_migration::CredentialMigrationRef;
#[ignore] // You can manually run "ignore" tests in the IDE, but they won't run in CI.
#[tokio::test]
@@ -64,6 +65,7 @@ mod tests {
num_images: NumImages::One,
image_size: ImageSize::Square,
credentials: CredentialMigrationRef::Legacy(&creds),
request_timeout: None,
}).await?;
println!("task_id: {}", response.task_id);
File diff suppressed because one or more lines are too long
@@ -0,0 +1 @@
pub mod list_media;
@@ -1,5 +1,6 @@
pub mod bearer;
pub mod image_gen;
pub mod job;
pub mod list_media;
pub mod sentinel_refresh;
pub mod upload;
@@ -2,10 +2,16 @@ use crate::creds::credential_migration::CredentialMigrationRef;
use crate::requests::upload::upload_media_http_request::{upload_media_http_request, SoraMediaUploadResponse};
use crate::sora_error::SoraError;
use std::path::PathBuf;
use std::time::Duration;
/// Upload bytes.
/// The underlying reqwest lib needs to own the bytes, so we can't pass as a reference.
pub async fn sora_media_upload_from_bytes(bytes: Vec<u8>, file_name: String, creds: CredentialMigrationRef<'_>) -> Result<SoraMediaUploadResponse, SoraError> {
pub async fn sora_media_upload_from_bytes(
bytes: Vec<u8>,
file_name: String,
creds: CredentialMigrationRef<'_>,
maybe_timeout: Option<Duration>,
) -> Result<SoraMediaUploadResponse, SoraError> {
let file_path = PathBuf::from(&file_name);
// TODO: Read file magic bytes first, then fall back to this.
@@ -20,5 +26,5 @@ pub async fn sora_media_upload_from_bytes(bytes: Vec<u8>, file_name: String, cre
_ => "application/octet-stream",
};
Ok(upload_media_http_request(bytes, file_name, mime_type, creds).await?)
Ok(upload_media_http_request(bytes, file_name, mime_type, creds, maybe_timeout).await?)
}
@@ -2,6 +2,7 @@ use crate::creds::credential_migration::CredentialMigrationRef;
use crate::requests::upload::upload_media_http_request::{upload_media_http_request, SoraMediaUploadResponse};
use crate::sora_error::SoraError;
use std::path::Path;
use std::time::Duration;
use tokio::fs::File;
use tokio::io::AsyncReadExt;
@@ -9,7 +10,11 @@ use tokio::io::AsyncReadExt;
/// There's a better way to implement this.
const INITIAL_BUFFER_SIZE : usize = 1024*1024;
pub async fn sora_media_upload_from_file<P: AsRef<Path>>(file_path: P, creds: CredentialMigrationRef<'_>) -> Result<SoraMediaUploadResponse, SoraError> {
pub async fn sora_media_upload_from_file<P: AsRef<Path>>(
file_path: P,
creds: CredentialMigrationRef<'_>,
maybe_timeout: Option<Duration>,
) -> Result<SoraMediaUploadResponse, SoraError> {
let mut file = File::open(&file_path).await?;
let mut buffer = Vec::with_capacity(INITIAL_BUFFER_SIZE);
file.read_to_end(&mut buffer).await?;
@@ -31,7 +36,7 @@ pub async fn sora_media_upload_from_file<P: AsRef<Path>>(file_path: P, creds: Cr
_ => "application/octet-stream",
};
Ok(upload_media_http_request(buffer, filename, mime_type, creds).await?)
Ok(upload_media_http_request(buffer, filename, mime_type, creds, maybe_timeout).await?)
}
#[cfg(test)]
@@ -63,6 +68,7 @@ mod tests {
let response = sora_media_upload_from_file(
image_path,
CredentialMigrationRef::Legacy(&creds),
None,
).await?;
println!("media: {:?}", response);
@@ -1,3 +1,4 @@
use std::time::Duration;
use crate::credentials::{SoraCredentials, USER_AGENT};
use crate::creds::credential_migration::CredentialMigrationRef;
use crate::sora_error::SoraError;
@@ -69,6 +70,7 @@ pub (crate) async fn upload_media_http_request(
filename: String,
mime_type: &str,
credentials: CredentialMigrationRef<'_>,
maybe_timeout: Option<Duration>,
) -> Result<SoraMediaUploadResponse, SoraError> {
// Create multipart form
@@ -97,11 +99,15 @@ pub (crate) async fn upload_media_http_request(
// Make API request
let client = Client::new();
let request_builder = client.post(SORA_UPLOAD_MEDIA_URL)
let mut request_builder = client.post(SORA_UPLOAD_MEDIA_URL)
.multipart(form)
.header("User-Agent", USER_AGENT)
.header("Cookie", &cookie)
.header("Authorization", &auth_header);
if let Some(timeout) = maybe_timeout {
request_builder = request_builder.timeout(timeout);
}
let response = request_builder.send().await?;
@@ -26,10 +26,27 @@ pub enum SoraError {
/// }
UnauthorizedCookieOrBearerExpired,
/// Failed to generate a bearer token despite the `200 OK` response.
/// This has been generated by sending old/expired session cookies or empty cookies.
FailedToGenerateBearer,
/// Sora requires users to set up an account username before they can generate images.
/// This requires the user to set up that username.
SoraUsernameNotYetCreated,
/// Something is wrong with the JWT bearer token.
/// This error originates on our end as we try to parse the JWT.
LocalJwtClaimsParseError(String),
/// Sora is having issues.
/// Typically served from Cloudflare
/// We preserve the message for debugging.
BadGateway(String),
/// HTTP 524 (Cloudflare-specific)
/// Cloudflare formed a TCP connection to Sora, but no payload was delivered before timeout
/// We preserve the message for debugging
CloudFlareTimeout(String),
/// Another error occurred.
OtherBadStatus(anyhow::Error),
@@ -45,6 +62,10 @@ pub enum SoraError {
/// serde_json::Error, likely from JSON deserialization schema mismatch.
JsonError(serde_json::Error),
/// serde_json::Error, likely from JSON deserialization schema mismatch.
/// Includes the original body.
JsonErrorWithBody(serde_json::Error, String),
}
impl Error for SoraError {}
@@ -55,6 +76,15 @@ impl Display for SoraError {
Self::NoBearerTokenAvailable => {
write!(f, "Unauthorized: no bearer token set")
}
Self::FailedToGenerateBearer => {
write!(f, "Failed to generate bearer. Probably invalid session.")
}
Self::SoraUsernameNotYetCreated => {
write!(f, "Sora username not created. Please set up a username.")
}
Self::LocalJwtClaimsParseError(message) => {
write!(f, "Local JWT claims parse error: {}", message)
}
Self::TooManyConcurrentTasks => {
write!(f, "Too many concurrent tasks. Please wait.")
}
@@ -64,6 +94,9 @@ impl Display for SoraError {
Self::BadGateway(message) => {
write!(f, "Bad Gateway; Sora is likely having issues: {:?}", message)
}
Self::CloudFlareTimeout(message) => {
write!(f, "Cloudflare Timeout (524); Sora is likely having issues: {:?}", message)
}
Self::OtherBadStatus(err) => {
write!(f, "Other error: {}", err)
}
@@ -79,6 +112,9 @@ impl Display for SoraError {
Self::JsonError(err) => {
write!(f, "serde_json error: {}", err)
}
Self::JsonErrorWithBody(err, message) => {
write!(f, "serde_json error: {:?} body: {:?}", err, message)
}
}
}
}
@@ -8,17 +8,61 @@ pub async fn classify_general_http_error(response: reqwest::Response) -> SoraErr
Ok(text) => text,
Err(err) => return SoraError::ReqwestError(err),
};
/*
Old cookie -
Message: {
"error": {
"message": "Your authentication token has expired. Please try signing in again.",
"type": "invalid_request_error",
"param": null,
"code": "token_expired"
}
}
No cookie -
Message: {
"error": {
"message": "Your authentication token has expired. Please try signing in again.",
"type": "invalid_request_error",
"param": null,
"code": "token_expired"
}
}
*/
// TODO: I *think* this is just the bearer token, not the cookie.
let cookie_expired =
message.contains("Your authentication token has expired. Please try signing in again.")
|| message.contains("token_expired");
if cookie_expired {
return SoraError::UnauthorizedCookieOrBearerExpired
return SoraError::UnauthorizedCookieOrBearerExpired;
}
let needs_onboarding =
message.contains("You must onboard before using this service")
|| message.contains("onboarding_required");
if needs_onboarding {
return SoraError::SoraUsernameNotYetCreated;
}
if status.as_u16() == 502 {
return SoraError::BadGateway(message);
let status_code = status.as_u16();
match status_code {
502 => {
return SoraError::BadGateway(message);
}
524 => {
if message.contains("cloudflare") || message.contains("Cloudflare") {
return SoraError::CloudFlareTimeout(message);
}
}
_ => {}, // Fall-through
}
SoraError::OtherBadStatus(anyhow::anyhow!("Upload failed with status {}: {}", status, message))
@@ -4,7 +4,8 @@ use cookie::Cookie;
use once_cell::sync::Lazy;
use errors::AnyhowResult;
pub const SESSION_COOKIE_NAME : &str = "__Secure-authjs.session-token";
pub const SESSION_COOKIE_NAME_OLD : &str = "__Secure-authjs.session-token";
pub const SESSION_COOKIE_NAME_NEW : &str = "__Secure-next-auth.session-token";
/// Cookies Sora.com mints that are not the session cookie. We can ignore these.
/// This list is very likely to change over time.
@@ -51,7 +52,9 @@ pub fn has_session_cookie(cookies_header: &str) -> AnyhowResult<SessionCookiePre
for cookie in cookies {
let cookie= cookie?;
let cookie_name = cookie.name();
if cookie_name == SESSION_COOKIE_NAME {
if cookie_name == SESSION_COOKIE_NAME_NEW {
return Ok(SessionCookiePresence::Present);
} else if cookie_name == SESSION_COOKIE_NAME_NEW {
return Ok(SessionCookiePresence::Present);
} else if !IRRELEVANT_COOKIE_NAMES.contains(cookie_name) {
unknown_cookie_count += 1;
@@ -1,12 +1,13 @@
//! Adapted from 'jwt' crate
use crate::sora_error::SoraError;
use anyhow::anyhow;
use base64::Engine;
use base64::prelude::BASE64_URL_SAFE_NO_PAD;
use base64::Engine;
use chrono::{DateTime, Utc};
use errors::AnyhowResult;
use log::warn;
use serde_json::Value;
use errors::AnyhowResult;
const SEPARATOR: &str = ".";
@@ -19,48 +20,53 @@ pub struct SoraJwtClaims {
pub email_verified: bool,
}
pub fn lightweight_sora_jwt_parse(token: &str) -> AnyhowResult<SoraJwtClaims> {
pub fn lightweight_sora_jwt_parse(token: &str) -> Result<SoraJwtClaims, SoraError> {
let [_header_str, claims_str, _signature_str] = split_components(token)?;
let claims = decode_base64(claims_str)?;
let claims : serde_json::Map<String, Value> = serde_json::from_str(&claims)?;
let claims = decode_base64(claims_str)
.map_err(|err| SoraError::LocalJwtClaimsParseError(
format!("failure to parse claims base64: {:?}", err)))?;
let claims : serde_json::Map<String, Value> = serde_json::from_str(&claims)
.map_err(|err| SoraError::LocalJwtClaimsParseError(
format!("failure to parse claims json: {:?}", err)))?;
let iat = claims.get("iat")
.map(|val| val.as_number())
.flatten()
.ok_or(anyhow!("no iat claim"))?;
.ok_or_else(|| jwt_error("no iat claim"))?;
let iat = iat.as_i64()
.ok_or(anyhow!("iat is not a number"))?;
.ok_or_else(|| jwt_error("iat is not a number"))?;
let iat = DateTime::from_timestamp(iat, 0)
.ok_or(anyhow!("iat is not a valid timestamp"))?;
.ok_or_else(|| jwt_error("iat is not a valid timestamp"))?;
let exp = claims.get("exp")
.map(|val| val.as_number())
.flatten()
.ok_or(anyhow!("no exp claim"))?;
.ok_or_else(|| jwt_error("no exp claim"))?;
let exp = exp.as_i64()
.ok_or(anyhow!("exp is not a number"))?;
.ok_or_else(|| jwt_error("exp is not a number"))?;
let exp = DateTime::from_timestamp(exp, 0)
.ok_or(anyhow!("iat is not a valid timestamp"))?;
.ok_or_else(|| jwt_error("iat is not a valid timestamp"))?;
let user_id = claims.get("https://api.openai.com/auth")
.map(|val| val.get("user_id"))
.flatten()
.map(|val| val.as_str())
.flatten()
.ok_or(anyhow!("no user_id claim"))?;
.ok_or_else(|| jwt_error("no user_id claim"))?;
let profile = claims.get("https://api.openai.com/profile")
.ok_or(anyhow!("no user_id claim"))?;
.ok_or_else(|| jwt_error("no user_id claim"))?;
let email = profile.get("email")
.map(|val| val.as_str())
.flatten()
.ok_or(anyhow!("no email claim"))?;
.ok_or_else(|| jwt_error("no email claim"))?;
let email_verified = profile.get("email_verified")
.map(|val| val.as_bool())
@@ -79,14 +85,19 @@ pub fn lightweight_sora_jwt_parse(token: &str) -> AnyhowResult<SoraJwtClaims> {
})
}
fn split_components(token: &str) -> AnyhowResult<[&str; 3]> {
fn split_components(token: &str) -> Result<[&str; 3], SoraError> {
let mut components = token.split(SEPARATOR);
let header = components.next().ok_or(anyhow!("no header component"))?;
let claims = components.next().ok_or(anyhow!("no claims component"))?;
let signature = components.next().ok_or(anyhow!("no signature component"))?;
let header = components.next()
.ok_or_else(|| jwt_error("no header component"))?;
let claims = components.next()
.ok_or_else(|| jwt_error("no claims component"))?;
let signature = components.next()
.ok_or_else(|| jwt_error("no signature component"))?;
if components.next().is_some() {
return Err(anyhow!("too many components"));
return Err(jwt_error("too many components"));
}
Ok([header, claims, signature])
@@ -98,6 +109,10 @@ fn decode_base64(raw: &str) -> AnyhowResult<String> {
Ok(json_str)
}
fn jwt_error(reason: &str) -> SoraError {
SoraError::LocalJwtClaimsParseError(reason.to_string())
}
#[cfg(test)]
mod tests {
use crate::utils::lightweight_sora_jwt_parse::lightweight_sora_jwt_parse;
@@ -15,7 +15,7 @@ use log::warn;
use log::{debug, error};
use mysql_queries::queries::media_files::get::get_media_file::get_media_file;
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::requests::image_gen::SoraError;
use openai_sora_client::requests::image_gen::SoraImageGenError;
use openai_sora_client::requests::sentinel_refresh::generate::token::generate_token;
use serde::Deserialize;
use serde::Serialize;
@@ -326,7 +326,7 @@ pub async fn enqueue_studio_image_generation_handler(http_request: HttpRequest,
debug!("Sora image gen response: {:?}", response);
if let Err(SoraError::SentinelBlock(msg)) = &response {
if let Err(SoraImageGenError::SentinelBlock(msg)) = &response {
error!("Sora sentinel block, attempting refresh: {}", msg);
match generate_token().await {
@@ -382,16 +382,16 @@ pub async fn enqueue_studio_image_generation_handler(http_request: HttpRequest,
let response = response.map_err(|err| {
match err {
SoraError::TokenExpired(msg) => {
SoraImageGenError::TokenExpired(msg) => {
error!("Sora token expired, needs refresh: {}", msg);
// TODO: Implement token refresh logic here
EnqueueImageGenRequestError::ServerError
},
SoraError::SentinelBlock(msg) => {
SoraImageGenError::SentinelBlock(msg) => {
error!("Sora sentinel block still occurring after refresh: {}", msg);
EnqueueImageGenRequestError::ServerError
},
SoraError::TooManyConcurrentTasks(msg) => {
SoraImageGenError::TooManyConcurrentTasks(msg) => {
error!("Sora too many concurrent tasks: {}", msg);
EnqueueImageGenRequestError::TooManyConcurrentTasks
},
@@ -320,7 +320,7 @@ export const PromptBox = () => {
if (!prompt.trim()) return;
try {
console.log(
"Enqueuing with prompt:",
"(3) Enqueuing with prompt:",
prompt,
"and reference images:",
referenceImages,
+12 -3
View File
@@ -1,11 +1,20 @@
<!doctype html>
<html lang="en">
<head>
<title>ArtCraft</title>
<meta charset="utf-8" />
<title>ArtCraft 3D</title>
<base href="/" />
<meta name="viewport" content="width=device-width, initial-scale=1" />
<!-- Google tag (gtag.js) -->
<script async src="https://www.googletagmanager.com/gtag/js?id=G-Y4Y8KMLPQ7"></script>
<script>
window.dataLayer = window.dataLayer || [];
function gtag(){dataLayer.push(arguments);}
gtag('js', new Date());
gtag('config', 'G-Y4Y8KMLPQ7');
</script>
<base href="/" />
<link rel="icon" type="image/x-icon" href="/favicon.ico" />
<link rel="preconnect" href="https://fonts.googleapis.com" />
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />

Some files were not shown because too many files have changed in this diff Show More