Compare commits

...
Author SHA1 Message Date
CrossPr0duct 3b4afd5f67 Fixed original bug associated with loading the images to ensure that the screen renders the images on addition. 2025-02-23 21:27:01 -05:00
BFlat a6489b3252 color picker ux (#422)
![image](https://github.com/user-attachments/assets/8544977e-5d4c-4ad6-bfaa-a7bc500a3c83)

changes:
- active styling and circular preview
- if paint brush button is active, hovering over the button will open
the color picker automatically and when mouse leaves the picker popover
panel, it closes and confirms. Reduces multiple clicks for a better
experience.
2025-02-23 17:58:33 -05:00
Tanish Manku bbcd64fab4 Hack: Conditionally show color picker button based on mode
This needs further design review. Ideally I'd write a proper separate
component that handles the mode selection as well as color picker/hover
logic but we're moving fast.
2025-02-23 02:53:30 -08:00
Tanish Manku 7f2a082027 Modify ColorPicker to fit toolbar style with props 2025-02-23 02:51:59 -08:00
Tanish Manku 67a5ea44c1 Add paint color signal events 2025-02-23 02:51:41 -08:00
Tanish Manku 54261d86da Make context menu vertical 2025-02-23 01:41:28 -08:00
Tanish Manku ae71762044 Make context menu only appear on right-click 2025-02-23 01:41:15 -08:00
BFlat 20fd9006fe everything restyled (#421)
everything restyled:
- prompt box
- new slider
- color scheme/theme
- moved save button to top right of screen
2025-02-22 15:04:04 +07:00
Tanish Manku 95c4293fe8 Engine: Add placeholder event listeners for modes 2025-02-21 17:05:28 -08:00
Tanish Manku 737edda502 Setup select, paint and eraser modes 2025-02-21 17:04:03 -08:00
Tanish Manku d7092e9492 Replace dividers with rule tags for symmetry 2025-02-21 15:54:51 -08:00
Tanish Manku c121d1ac8c Add paint button to main toolbar 2025-02-21 15:54:35 -08:00
Tanish Manku 40ceb2e08b Add an Eraser button to Toolbar 2025-02-21 15:49:03 -08:00
Tanish Manku fb2ec6baa4 Make main toolbar appear on the left 2025-02-21 15:39:09 -08:00
Michael Chung 6077232cd3 Hide Preview Drag Layer as it is not done. 2025-02-21 17:21:43 -05:00
Michael Chung f013373180 Copy Shift Still needs some UX work but it does work.
Need to store it as a image node.
2025-02-21 00:39:59 -05:00
Michael Chung f0c1fcf516 Got the color changes to work. 2025-02-20 22:34:17 -05:00
Brandon Thomas 9f392de9c5 bump packages to enable windows builds 2025-02-20 18:10:07 -05:00
BFlat f5fb340de9 landing hero update (#419) 2025-02-21 01:49:28 +07:00
Tanish Manku 0392bda03c Forward color picker events to engine (#418) 2025-02-20 11:38:29 -05:00
Michael Chung 1bca478621 Move color picker into the correct toolbar 2025-02-19 23:56:45 -05:00
Michael Chung f9c966003a Fixed issue with shapes not having the context menu component. 2025-02-19 23:18:28 -05:00
Michael Chung 2d8245cd55 Added Shapes.
Removed Extra buttons.
2025-02-19 22:29:31 -05:00
Tanish Manku 9d1d605f5c Add shapes to global toolbar and color picker to contextual (#417)
Shapes scaffolding is fully set up, color picker isn't tied up to the
nodes so it doesn't actually change the color yet
2025-02-19 21:55:06 -05:00
Michael Chung 1e38fbad5c Turns out you need to convert these rasters into images. 2025-02-19 20:33:01 -05:00
Michael Chung 6e03df968d Remove duplicate adds?
Then fixed layering issue by removing useless background layer.
Need to fix shape rendering.
2025-02-19 19:34:06 -05:00
Michael Chung 13e0e4bb02 Merge branch 'main' of github.com:storytold/storyteller-rust 2025-02-19 18:00:26 -05:00
Michael Chung 442c3579e1 Fixed Modal to have the image enter the scene right away.
Added Shape Fixes resizes properly and drags.
Removing Progress Fill.
2025-02-19 17:59:14 -05:00
bflatastic b8a305e271 mobile download button disable landing 2025-02-19 22:58:22 +07:00
BFlat a70061bfd3 realtime product landing (#415) 2025-02-19 20:44:22 +07:00
Brandon Thomas e7a392b8e2 add cfg (#416) 2025-02-19 00:21:24 -05:00
Michael Chung 9147da22f2 ShapeNodes work need to fix transformations. 2025-02-19 00:09:47 -05:00
Brandon Thomas 29c2df3ee6 strength, negative prompt 2025-02-18 21:42:22 -05:00
Brandon Thomas 1db4278712 cleanup 2025-02-18 02:19:38 -05:00
Brandon Thomas 5ee910ad07 pass prompt from frontend 2025-02-18 01:57:37 -05:00
Brandon ThomasandKasisnu ff9d2c6b34 realtime + speedup (#414)
Co-authored-by: Kasisnu <5044120+kasisnu@users.noreply.github.com>
2025-02-18 01:29:56 -05:00
Michael Chung 462dcd7645 Removed buttons that didn't need to be there.
Wired up prompt and strength sliders and save image.
Removed some more code.
2025-02-14 17:35:48 -05:00
Michael Chung d1617f88a4 Fixed UI Positioning.
Tied events to the system.
Should accept realtime inputs on changes.
2025-02-14 17:10:14 -05:00
ArEnScandTanish c2e57d0815 Prompt Box and Strength (#412)
Co-authored-by: Tanish <tanish2k09.dev@gmail.com>
2025-02-14 15:44:15 -05:00
97 changed files with 3838 additions and 630 deletions
+2
View File
@@ -31,6 +31,7 @@
<w>hifigan</w>
<w>hparams</w>
<w>huggingface</w>
<w>kasisnu</w>
<w>lipsync</w>
<w>mocap</w>
<w>mochi</w>
@@ -47,6 +48,7 @@
<w>techcrunch</w>
<w>transcoded</w>
<w>transcodings</w>
<w>unet</w>
<w>unpermitted</w>
<w>upserting</w>
<w>upserts</w>
Generated
+967 -78
View File
File diff suppressed because it is too large Load Diff
+7
View File
@@ -0,0 +1,7 @@
image_width: 512
image_height: 512
#scheduler_steps: 15
# NB(bt): It looks like we can get down to 2 sampling steps, perhaps ?
scheduler_steps: 2
scheduler_samples: 1
cfg_scale: 1.1
+29 -7
View File
@@ -18,11 +18,33 @@ crate-type = ["staticlib", "cdylib", "rlib"]
tauri-build = { version = "2.0.4", features = [] }
[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
log = "0.4"
tauri = { version = "2.2.4", features = [] }
tauri-plugin-log = "2.0.0-rc"
once_cell = "1.20.3"
image = "0.25.5"
# Internal
filesys = { path = "../../lib/files/filesys" }
memory_caching = { path = "../../lib/caching/memory_caching" }
# Bypass cudarc issues on Windows https://github.com/coreylowman/cudarc/issues/332
cudarc = "0.13.7"
# External
anyhow = "1.0"
bytes = "1.10"
base64 = "0.22.1"
candle-core = { version = "0.8.2", features = ["cuda"] }
candle-nn = { version = "0.8.2", features = ["cuda"] }
candle-transformers = { version = "0.8.2", features = ["cuda"] }
hf-hub = "0.3.2"
image = "0.25.5"
serde_yaml = "0.9"
log = "0.4"
once_cell = "1.20.3"
rand = "0.8"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
tauri = { version = "2.2.4", features = [] }
tauri-plugin-log = "2"
tokenizers = "0.15.2"
[features]
default = ["cuda"]
cuda = ["candle-core/cuda", "candle-nn/cuda", "candle-transformers/cuda"]
@@ -6,6 +6,7 @@
"main"
],
"permissions": [
"core:default"
"core:default",
"log:default"
]
}
}
@@ -1,11 +1,11 @@
use std::io::Cursor;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use base64::prelude::{BASE64_STANDARD, BASE64_URL_SAFE};
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::ImageReader;
use std::io::Cursor;
#[tauri::command]
pub fn infer_image(image: &str) -> Result<String, String> {
pub fn flip_image(image: &str) -> Result<String, String> {
println!("infer_image called; processing image...");
let bytes = BASE64_STANDARD.decode(image)
@@ -18,17 +18,19 @@ pub fn infer_image(image: &str) -> Result<String, String> {
.map_err(|err| format!("Image decode error: {}", err))?;
let rotated_image = image.rotate180();
let mut buffer = Vec::new(); // TODO: Preallocate, better data structure
let mut writer = Cursor::new(&mut buffer);
let encoder = PngEncoder::new_with_quality
(&mut writer, CompressionType::Fast, FilterType::Adaptive);
let encoder = PngEncoder::new_with_quality(&mut writer, CompressionType::Fast, FilterType::Adaptive);
rotated_image.write_with_encoder(encoder)
.map_err(|err| format!("Image encoding error: {}", err))?;
let encoded = BASE64_STANDARD.encode(&buffer);
println!("Encoded: {:?}", encoded.split_at(10).0);
Ok(encoded)
}
@@ -0,0 +1,117 @@
use crate::ml::model_cache::ModelCache;
use crate::ml::prompt_cache::PromptCache;
use crate::ml::stable_diffusion::stable_diffusion_pipeline::{stable_diffusion_pipeline, Args};
use crate::state::app_config::AppConfig;
use base64::{engine::general_purpose::STANDARD as BASE64_STANDARD, Engine};
use bytes::BytesMut;
use image::imageops::FilterType;
use image::{DynamicImage, ImageFormat, ImageReader};
use log::{error, info};
use std::io::Cursor;
use std::path::PathBuf;
use tauri::State;
const PROMPT_FILENAME : &str = "prompt.txt";
const PROMPT: &str = "A beautiful landscape with mountains and a lake";
const NEGATIVE_PROMPT: &str = "bad quality, bad faces, poor quality, blurry faces, watermark";
/// This handler takes an image (as a base64 encoded string) and a prompt and returns
/// an image (as a base64-encoded string).
#[tauri::command]
pub fn infer_image(
image: &str,
prompt: Option<String>,
strength: Option<u8>,
model_config: State<AppConfig>,
model_cache: State<ModelCache>,
prompt_cache: State<PromptCache>,
) -> Result<String, String> {
let prompt = get_prompt_or_fallback(prompt);
info!("Prompt: {}", prompt);
let image = hydrate_base64_image(image)
.map_err(|err| format!("Couldn't hydrate image from base64: {}", err))?;
let result = do_infer_image(&prompt, image, strength, &model_config, &model_cache, prompt_cache);
if let Err(err) = result.as_deref() {
error!("There was an error: {:?}", err);
}
result
}
fn do_infer_image(
prompt: &str,
image: DynamicImage,
strength: Option<u8>,
config: &AppConfig,
model_cache: &ModelCache,
prompt_cache: State<PromptCache>,
) -> Result<String, String> {
println!("infer_image called; generating image with SDXL Turbo...");
let args = Args {
image: &image,
prompt: prompt.to_string(),
uncond_prompt: NEGATIVE_PROMPT.to_string(),
//guidance_scale: Some(0.0),
model_cache,
configs: config,
prompt_cache: &prompt_cache,
i2i_strength: strength,
cfg_scale: config.cfg_scale,
};
match stable_diffusion_pipeline(args) {
Ok(image) => {
let mut bytes = Vec::with_capacity(1024*1024);
image.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)
.map_err(|err| format!("failure to encode image: {:?}", err))?;
let bytes = BASE64_STANDARD.encode(bytes);
Ok(bytes)
},
Err(e) => Err(format!("Failed to generate image: {}", e)),
}
}
fn get_prompt_or_fallback(user_prompt: Option<String>) -> String {
let user_prompt = user_prompt.map(|prompt| prompt.trim().to_string())
.filter(|prompt| !prompt.is_empty());
if let Some(prompt) = user_prompt {
return prompt;
}
let prompt_file = PathBuf::from(PROMPT_FILENAME)
.canonicalize()
.unwrap_or_else(|_| PathBuf::from(PROMPT_FILENAME));
std::fs::read_to_string(&prompt_file)
.map_err(|err| format!("Failed to read prompt file: {}", err))
.unwrap_or_else(|_| {
error!("Failed to read prompt file: {:?}", prompt_file);
PROMPT.to_string()
})
.trim()
.to_string()
}
fn hydrate_base64_image(base64_image: &str) -> anyhow::Result<DynamicImage> {
let bytes = BASE64_STANDARD.decode(base64_image)?;
let image = ImageReader::new(Cursor::new(bytes))
.with_guessed_format()?
.decode()?;
// TODO(bt,2025-02-17): Running out of vram with full image buffer size
let image = image.resize(512, 512, FilterType::CatmullRom);
Ok(image)
}
@@ -0,0 +1,3 @@
pub mod flip_image;
pub mod image_endpoint;
pub mod test_counter;
@@ -0,0 +1,24 @@
use std::sync::{Arc, RwLock};
use once_cell::sync::Lazy;
static COUNTER : Lazy<Arc<RwLock<u64>>> = Lazy::new(|| Arc::new(RwLock::new(0)));
#[tauri::command]
pub fn test_counter() -> u64 {
let value : u64;
{
match COUNTER.write() {
Ok(mut counter) => {
*counter += 1;
value = *counter;
},
Err(_e) => {
value = 0;
}
}
}
println!("I was invoked from JavaScript! {:?}", value);
value
}
+46 -41
View File
@@ -1,55 +1,60 @@
mod image_endpoint;
pub mod endpoints;
pub mod ml;
pub mod state;
use std::sync::{Arc, RwLock};
use once_cell::sync::Lazy;
use image_endpoint::infer_image;
use crate::ml::model_cache::ModelCache;
use crate::ml::prompt_cache::PromptCache;
use crate::state::app_config::AppConfig;
use endpoints::image_endpoint::infer_image;
use endpoints::test_counter::test_counter;
use tauri_plugin_log::Target;
use tauri_plugin_log::TargetKind;
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
// NB: Tauri wants to install the logger itself, so we can't rely on the logger crate
// until the tauri runtime begins.
println!("Loading model config...");
let config = AppConfig::init()
.expect("config should load");
let prompt_cache = PromptCache::with_capacity(8)
.expect("prompt cache should load");
println!("Creating model cache...");
let model_cache = ModelCache::new(
config.device.clone(),
config.dtype,
config.sd_version.clone(),
config.sd_config.clone()
).expect("Model cache should create");
println!("Initializing backend runtime...");
tauri::Builder::default()
.plugin(tauri_plugin_log::Builder::new()
.level(log::LevelFilter::Info)
.targets(vec![Target::new(TargetKind::Stdout)])
.build())
.setup(|app| {
if cfg!(debug_assertions) {
app.handle().plugin(
tauri_plugin_log::Builder::default()
.level(log::LevelFilter::Info)
.build(),
)?;
}
//if cfg!(debug_assertions) {
// app.handle().plugin(
// tauri_plugin_log::Builder::default()
// .level(log::LevelFilter::Info)
// .build(),
// )?;
//}
Ok(())
})
.manage(config)
.manage(prompt_cache)
.manage(model_cache)
.invoke_handler(tauri::generate_handler![
my_custom_command,
test_round_trip,
test_counter,
infer_image,
])
.run(tauri::generate_context!())
.expect("error while running tauri application");
}
#[tauri::command]
fn my_custom_command() {
println!("I was invoked from JavaScript!");
}
static COUNTER : Lazy<Arc<RwLock<u64>>> = Lazy::new(|| Arc::new(RwLock::new(0)));
#[tauri::command]
fn test_round_trip() -> u64 {
let value : u64;
{
match COUNTER.write() {
Ok(mut counter) => {
*counter += 1;
value = *counter;
},
Err(_e) => {
value = 0;
}
}
}
println!("I was invoked from JavaScript! {:?}", value);
value
}
@@ -0,0 +1,17 @@
use anyhow::Result;
use candle_core::{DType, Device, Tensor};
use image::DynamicImage;
pub fn dynamic_image_to_tensor(image: &DynamicImage, device: &Device, datatype: DType) -> Result<Tensor> {
let (height, width) = (image.height() as usize, image.width() as usize);
let height = height - height % 32;
let width = width - width % 32;
let image = image.resize_to_fill(width as u32, height as u32, image::imageops::FilterType::CatmullRom);
let image = image.to_rgb8();
let image = image.into_raw();
Ok(Tensor::from_vec(image, (height, width, 3), device)?
.permute((2, 0, 1))?
.to_dtype(datatype)?
.affine(2. / 255., -1.)?
.unsqueeze(0)?)
}
@@ -0,0 +1,28 @@
use candle_core::{DType, Device, Tensor};
pub fn load_image_file_to_tensor<T: AsRef<std::path::Path>>(path: T, device: &Device) -> anyhow::Result<Tensor> {
let img = image::ImageReader::open(path)?.decode()?;
let (height, width) = (img.height() as usize, img.width() as usize);
println!("INPUT IMAGE WIDTH AND HEIGHT: {}x{}", width, height);
// TODO: Downscale from 1024 -- YES! THIS HACK DOES THE TRICK OF GETTING THE TENSOR SHAPE TO [1,4,64,64] !!
let height = height / 2;
let width = width / 2;
let height = height - height % 32;
let width = width - width % 32;
let img = img.resize_to_fill(
width as u32,
height as u32,
image::imageops::FilterType::CatmullRom,
);
println!("RESIZED IMAGE WIDTH AND HEIGHT: {}x{}", width, height);
let img = img.to_rgb8();
let img = img.into_raw();
let img = Tensor::from_vec(img, (height, width, 3), device)?
.permute((2, 0, 1))?
.to_dtype(DType::F32)?
.affine(2. / 255., -1.)?
.unsqueeze(0)?;
Ok(img)
}
@@ -0,0 +1,18 @@
use anyhow::Result;
use candle_core::{DType, Device, Tensor};
pub fn load_image_file_to_tensor_2<T: AsRef<std::path::Path>>(path: T) -> Result<Tensor> {
let img = image::ImageReader::open(path)?.decode()?;
let (height, width) = (img.height() as usize, img.width() as usize);
let height = height - height % 32;
let width = width - width % 32;
let img = img.resize_to_fill(width as u32, height as u32, image::imageops::FilterType::CatmullRom);
let img = img.to_rgb8();
let img = img.into_raw();
let img = Tensor::from_vec(img, (height, width, 3), &Device::Cpu)?
.permute((2, 0, 1))?
.to_dtype(DType::F32)?
.affine(2. / 255., -1.)?
.unsqueeze(0)?;
Ok(img)
}
@@ -0,0 +1,5 @@
pub mod dynamic_image_to_tensor;
pub mod load_image_file_to_tensor;
pub mod load_image_file_to_tensor_2;
pub mod save_image_from_tensor;
pub mod tensor_to_image_buffer;
@@ -0,0 +1,28 @@
use anyhow::{Error as E, Result};
use candle_core::Tensor;
use image::{ImageBuffer, Rgb};
pub fn save_image_from_tensor<P: AsRef<std::path::Path>>(img: &Tensor, p: P) -> Result<()> {
println!("Saving image to {:?}", p.as_ref());
let p = p.as_ref();
let (channel, height, width) = img.dims3()?;
println!("Image dimensions: {}x{} with {} channels", width, height, channel);
if channel != 3 {
anyhow::bail!("save_image expects an input of shape (3, height, width)")
}
let img = img.permute((1, 2, 0))?.flatten_all()?;
let pixels = img.to_vec1::<u8>()?;
println!("Converting tensor to image buffer...");
let image: ImageBuffer<Rgb<u8>, Vec<u8>> =
match ImageBuffer::from_raw(width as u32, height as u32, pixels) {
Some(image) => image,
None => anyhow::bail!("error saving image {p:?}"),
};
println!("Successfully created image buffer");
image.save(p).map_err(E::from)?;
println!("Successfully saved image to disk");
Ok(())
}
@@ -0,0 +1,18 @@
use candle_core::Tensor;
use image::{ImageBuffer, Rgb};
pub type RgbImage = ImageBuffer<Rgb<u8>, Vec<u8>>;
pub fn tensor_to_image_buffer(img: &Tensor) -> anyhow::Result<RgbImage> {
let (channel, height, width) = img.dims3()?;
if channel != 3 {
anyhow::bail!("save_image expects an input of shape (3, height, width)")
}
let img = img.permute((1, 2, 0))?.flatten_all()?;
let pixels = img.to_vec1::<u8>()?;
ImageBuffer::from_raw(width as u32, height as u32, pixels)
.ok_or_else(|| anyhow::anyhow!("error loading image from raw tensor"))
}
+6
View File
@@ -0,0 +1,6 @@
pub mod image;
pub mod model_cache;
pub mod model_file;
pub mod models;
pub mod prompt_cache;
pub mod stable_diffusion;
@@ -0,0 +1,118 @@
use crate::ml::models::unet_model::UNetModel;
use crate::ml::model_file::StableDiffusionVersion;
use anyhow::anyhow;
use candle_core::{DType, Device, Tensor};
use candle_transformers::models::stable_diffusion::StableDiffusionConfig;
use hf_hub::api::sync::Api;
use std::sync::{Arc, RwLock};
use candle_transformers::models::stable_diffusion::vae::DiagonalGaussianDistribution;
use crate::ml::models::lazy_load_vae_model::LazyLoadVaeModel;
// TODO: This data structure is gross. Generalize the locking and loading semantics for reuse.
// Simple registry for now. We can build complex machinery that aids in
// VRAM utilization, disk space saving, intelligent scheduling, etc. in the future.
pub struct ModelCache {
// TODO(bt,2025-02-14): Support for more than one device in the future.
device: Device,
dtype: DType,
hf_api: Api,
sd_config: StableDiffusionConfig,
sd_version: StableDiffusionVersion,
unet: Arc<RwLock<Option<UNetModel>>>,
lazy_vae: LazyLoadVaeModel,
}
impl ModelCache {
pub fn new(
device: Device,
dtype: DType,
sd_version: StableDiffusionVersion,
sd_config: StableDiffusionConfig,
) -> anyhow::Result<Self> {
let api = Api::new()?;
Ok(Self {
device: device.clone(),
dtype,
hf_api: api.clone(),
unet: Arc::new(RwLock::new(None)),
lazy_vae: LazyLoadVaeModel::new(
&sd_config,
&sd_version,
&api,
&device,
dtype,
)?,
sd_config,
sd_version,
})
}
pub fn vae_encode(&self, xs: &Tensor) -> anyhow::Result<DiagonalGaussianDistribution> {
self.lazy_vae.encode(xs)
}
pub fn vae_decode(&self, xs: &Tensor) -> anyhow::Result<Tensor> {
self.lazy_vae.decode(xs)
}
pub fn unet_inference(&self, latent_model_input: &Tensor, timestep: f64, text_embeddings: &Tensor) -> anyhow::Result<Tensor> {
println!("unet_inference");
match self.try_unet_inference(latent_model_input, timestep, text_embeddings) {
Err(err) => Err(err),
Ok(Some(result)) => Ok(result),
Ok(None) => {
// Model wasn't previously loaded
self.load_unet()?;
self.try_unet_inference(latent_model_input, timestep, text_embeddings)?
.ok_or(anyhow!("model was not previously loaded"))
}
}
}
fn load_unet(&self) -> anyhow::Result<()> {
println!("load_unet");
match self.unet.write() {
Err(err) => return Err(anyhow!("lock error: {:?}", err)),
Ok(mut maybe_model) => {
match &*maybe_model {
Some(_model) => {} // Fall through
None => {
let repo = self.sd_version.repo();
println!("Downloading UNET model files from: {} ...", repo);
let unet_file = self.hf_api.model(repo.to_string())
.get("unet/diffusion_pytorch_model.safetensors")
.map_err(|err| anyhow!("error fetching model: {:?}", err))?;
println!("Building UNET model... (3)");
let unet = UNetModel::new(&self.sd_config, unet_file, &self.device, self.dtype)
.map_err(|err| anyhow!("error initializing unet model: {:?}", err))?;
println!("Built UNET...");
*maybe_model = Some(unet);
}
}
}
}
Ok(())
}
pub fn try_unet_inference(&self, latent_model_input: &Tensor, timestep: f64, text_embeddings: &Tensor) -> anyhow::Result<Option<Tensor>> {
println!("try_unet_inference");
match self.unet.read() {
Ok(mut model) => {
match &*model {
None => Ok(None), // Model is not yet loaded
Some(model) => {
Ok(Some(model.inference(latent_model_input, timestep, text_embeddings)?))
}
}
}
Err(err) => Err(anyhow!("lock error: {:?}", err)),
}
}
}
@@ -0,0 +1,83 @@
#[cfg(feature = "accelerate")]
extern crate accelerate_src;
#[cfg(feature = "mkl")]
extern crate intel_mkl_src;
use candle_transformers::models::stable_diffusion;
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum ModelFile {
Tokenizer,
Tokenizer2,
Clip,
Clip2,
Unet,
Vae,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum StableDiffusionVersion {
V1_5,
V1_5Inpaint,
V2_1,
V2Inpaint,
Xl,
XlInpaint,
Turbo,
}
impl StableDiffusionVersion {
pub fn repo(&self) -> &'static str {
match self {
Self::XlInpaint => "diffusers/stable-diffusion-xl-1.0-inpainting-0.1",
Self::Xl => "stabilityai/stable-diffusion-xl-base-1.0",
Self::V2Inpaint => "stabilityai/stable-diffusion-2-inpainting",
Self::V2_1 => "stabilityai/stable-diffusion-2-1",
Self::V1_5 => "runwayml/stable-diffusion-v1-5",
Self::V1_5Inpaint => "stable-diffusion-v1-5/stable-diffusion-inpainting",
Self::Turbo => "stabilityai/sdxl-turbo",
}
}
}
impl ModelFile {
pub fn get(
&self,
filename: Option<String>,
version: StableDiffusionVersion,
use_f16: bool,
) -> anyhow::Result<std::path::PathBuf> {
use hf_hub::api::sync::Api;
match filename {
Some(filename) => Ok(std::path::PathBuf::from(filename)),
None => {
let (repo, path) = match self {
Self::Tokenizer => {
let tokenizer_repo = match version {
StableDiffusionVersion::V1_5
| StableDiffusionVersion::V2_1
| StableDiffusionVersion::V1_5Inpaint
| StableDiffusionVersion::V2Inpaint => "openai/clip-vit-base-patch32",
StableDiffusionVersion::Xl
| StableDiffusionVersion::XlInpaint
| StableDiffusionVersion::Turbo => {
"openai/clip-vit-large-patch14"
}
};
(tokenizer_repo, "tokenizer.json")
}
Self::Tokenizer2 => {
("laion/CLIP-ViT-bigG-14-laion2B-39B-b160k", "tokenizer.json")
}
Self::Clip => (version.repo(), "text_encoder/model.safetensors"),
Self::Clip2 => (version.repo(), "text_encoder_2/model.safetensors"),
Self::Unet => (version.repo(), "unet/diffusion_pytorch_model.safetensors"),
Self::Vae => (version.repo(), "vae/diffusion_pytorch_model.safetensors"),
};
let filename = Api::new()?.model(repo.to_string()).get(path)?;
Ok(filename)
}
}
}
}
@@ -0,0 +1,130 @@
use crate::ml::model_file::StableDiffusionVersion;
use anyhow::anyhow;
use candle_core::{DType, Device, Tensor};
use candle_transformers::models::stable_diffusion::vae::{AutoEncoderKL, DiagonalGaussianDistribution};
use candle_transformers::models::stable_diffusion::StableDiffusionConfig;
use hf_hub::api::sync::Api;
use std::sync::{Arc, RwLock};
pub struct LazyLoadVaeModel {
lazy_model: Arc<RwLock<Option<AutoEncoderKL>>>,
model_configs: VaeConfigs,
}
// TODO: Maybe we pass these via a central config read at startup, passed as an Arc<ConfigT>
pub struct VaeConfigs {
sd_config: StableDiffusionConfig,
sd_version: StableDiffusionVersion,
device: Device,
dtype: DType,
hf_api: Api,
}
impl LazyLoadVaeModel {
pub fn new(
sd_config: &StableDiffusionConfig,
sd_version: &StableDiffusionVersion,
hf_api: &Api,
device: &Device,
dtype: DType,
) -> anyhow::Result<Self> {
println!("building vae model... (1)");
Ok(Self {
lazy_model: Arc::new(RwLock::new(None)),
model_configs: VaeConfigs {
sd_config: sd_config.clone(),
sd_version: sd_version.clone(),
device: device.clone(),
hf_api: hf_api.clone(),
dtype
}
})
}
pub fn encode(&self, xs: &Tensor) -> anyhow::Result<DiagonalGaussianDistribution> {
println!("vae encode...");
match self.try_encode(xs) {
Err(err) => Err(err),
Ok(Some(result)) => Ok(result),
Ok(None) => {
// Model wasn't previously loaded
self.load_model()?;
self.try_encode(xs)?
.ok_or(anyhow!("model was not previously loaded"))
}
}
}
pub fn decode(&self, xs: &Tensor) -> anyhow::Result<Tensor> {
println!("vae decode...");
match self.try_decode(xs) {
Err(err) => Err(err),
Ok(Some(result)) => Ok(result),
Ok(None) => {
// Model wasn't previously loaded
self.load_model()?;
self.try_decode(xs)?
.ok_or(anyhow!("model was not previously loaded"))
}
}
}
fn try_encode(&self, xs: &Tensor) -> anyhow::Result<Option<DiagonalGaussianDistribution>> {
match self.lazy_model.read() {
Err(err) => Err(anyhow!("lock error: {:?}", err)),
Ok(model) => {
match &*model {
None => Ok(None), // Model is not yet loaded
Some(model) => {
Ok(Some(model.encode(xs)?))
}
}
}
}
}
fn try_decode(&self, xs: &Tensor) -> anyhow::Result<Option<Tensor>> {
match self.lazy_model.read() {
Err(err) => Err(anyhow!("lock error: {:?}", err)),
Ok(model) => {
match &*model {
None => Ok(None), // Model is not yet loaded
Some(model) => {
Ok(Some(model.decode(xs)?))
}
}
}
}
}
fn load_model(&self) -> anyhow::Result<()> {
println!("load vae");
match self.lazy_model.write() {
Err(err) => return Err(anyhow!("lock error: {:?}", err)),
Ok(mut maybe_model) => {
match &*maybe_model {
Some(_model) => {} // Already loaded; fall through
None => {
let repo = self.model_configs.sd_version.repo();
println!("Building VAE model from : {:?} ... (3)", repo);
let vae_file = self.model_configs.hf_api.model(repo.to_string())
.get("vae/diffusion_pytorch_model.safetensors")?;
println!("Building VAE model from file {:?}...", &vae_file);
let vae = self.model_configs
.sd_config
.build_vae(vae_file, &self.model_configs.device, self.model_configs.dtype)?;
println!("Built VAE...");
*maybe_model = Some(vae);
}
}
}
}
Ok(())
}
}
@@ -0,0 +1,2 @@
pub mod unet_model;
pub mod lazy_load_vae_model;
@@ -0,0 +1,35 @@
use candle_core::{DType, Device, Tensor};
use candle_transformers::models::stable_diffusion::unet_2d::UNet2DConditionModel;
use candle_transformers::models::stable_diffusion::StableDiffusionConfig;
use std::path::Path;
pub struct UNetModel {
model: UNet2DConditionModel,
}
impl UNetModel {
pub fn new<P: AsRef<Path>>(
sd_config: &StableDiffusionConfig,
unet_weights_file: P,
device: &Device,
dtype: DType,
) -> anyhow::Result<Self> {
println!("building unet model... (1)");
let model = sd_config.build_unet(
unet_weights_file,
&device,
4,
false,
dtype
)?;
Ok(Self {
model,
})
}
pub fn inference(&self, latent_model_input: &Tensor, timestep: f64, text_embeddings: &Tensor) -> anyhow::Result<Tensor> {
Ok(self.model.forward(&latent_model_input, timestep, &text_embeddings)?)
}
}
@@ -0,0 +1,22 @@
use candle_core::Tensor;
use memory_caching::arc_sieve::ArcSieve;
pub struct PromptCache {
cache: ArcSieve<String, Tensor>,
}
impl PromptCache {
pub fn with_capacity(capacity: usize) -> anyhow::Result<Self> {
Ok(Self {
cache: ArcSieve::with_capacity(capacity)?,
})
}
pub fn get_copy(&self, key: &String) -> anyhow::Result<Option<Tensor>> {
self.cache.get_copy(key)
}
pub fn store_copy(&self, key: &String, value: &Tensor) -> anyhow::Result<()> {
self.cache.store_copy(key, value)
}
}
@@ -0,0 +1,13 @@
use crate::ml::model_file::StableDiffusionVersion;
pub fn get_vae_scale(sd_version: StableDiffusionVersion) -> f64 {
match sd_version {
StableDiffusionVersion::V1_5
| StableDiffusionVersion::V1_5Inpaint
| StableDiffusionVersion::V2_1
| StableDiffusionVersion::V2Inpaint
| StableDiffusionVersion::XlInpaint
| StableDiffusionVersion::Xl => 0.18215,
StableDiffusionVersion::Turbo => 0.13025,
}
}
@@ -0,0 +1,153 @@
#[cfg(feature = "accelerate")]
extern crate accelerate_src;
#[cfg(feature = "mkl")]
extern crate intel_mkl_src;
use candle_transformers::models::stable_diffusion;
use crate::ml::model_file::{ModelFile, StableDiffusionVersion};
use anyhow::Error as E;
use candle_core::{DType, Device, Module, Tensor, D};
use log::info;
use tokenizers::Tokenizer;
#[allow(clippy::too_many_arguments)]
pub fn infer_clip_text_embeddings(
prompt: &str,
uncond_prompt: &str,
tokenizer: Option<String>,
clip_weights: Option<String>,
clip2_weights: Option<String>,
sd_version: StableDiffusionVersion,
sd_config: &stable_diffusion::StableDiffusionConfig,
use_f16: bool,
device: &Device,
dtype: DType,
use_guide_scale: bool,
) -> anyhow::Result<Tensor> {
info!("Preparing text embeddings...");
let which = match sd_version {
StableDiffusionVersion::Xl
| StableDiffusionVersion::XlInpaint
| StableDiffusionVersion::Turbo => vec![true, false],
_ => vec![true],
};
let text_embeddings = which
.iter()
.map(|first| {
do_infer_clip_text_embeddings(
&prompt,
&uncond_prompt,
None, // tokenizer
None, // clip_weights
None, // clip2_weights
sd_version,
&sd_config,
false, // use_f16
&device,
dtype,
use_guide_scale,
*first,
)
})
.collect::<anyhow::Result<Vec<_>>>()?;
let text_embeddings = Tensor::cat(&text_embeddings, D::Minus1)?;
Ok(text_embeddings)
}
#[allow(clippy::too_many_arguments)]
fn do_infer_clip_text_embeddings(
prompt: &str,
uncond_prompt: &str,
tokenizer: Option<String>,
clip_weights: Option<String>,
clip2_weights: Option<String>,
sd_version: StableDiffusionVersion,
sd_config: &stable_diffusion::StableDiffusionConfig,
use_f16: bool,
device: &Device,
dtype: DType,
use_guide_scale: bool,
first: bool,
) -> anyhow::Result<Tensor> {
let tokenizer_file = if first {
ModelFile::Tokenizer
} else {
ModelFile::Tokenizer2
};
let tokenizer = tokenizer_file.get(tokenizer, sd_version, use_f16)?;
let tokenizer = Tokenizer::from_file(tokenizer).map_err(E::msg)?;
let pad_id = match &sd_config.clip.pad_with {
Some(padding) => *tokenizer.get_vocab(true).get(padding.as_str()).unwrap(),
None => *tokenizer.get_vocab(true).get("<|endoftext|>").unwrap(),
};
println!("Running with prompt \"{prompt}\".");
let mut tokens = tokenizer
.encode(prompt, true)
.map_err(E::msg)?
.get_ids()
.to_vec();
if tokens.len() > sd_config.clip.max_position_embeddings {
anyhow::bail!(
"the prompt is too long, {} > max-tokens ({})",
tokens.len(),
sd_config.clip.max_position_embeddings
)
}
while tokens.len() < sd_config.clip.max_position_embeddings {
tokens.push(pad_id)
}
let tokens = Tensor::new(tokens.as_slice(), device)?.unsqueeze(0)?;
println!("Building the Clip transformer.");
let clip_weights_file = if first {
ModelFile::Clip
} else {
ModelFile::Clip2
};
let clip_weights = if first {
clip_weights_file.get(clip_weights, sd_version, use_f16)?
} else {
clip_weights_file.get(clip2_weights, sd_version, use_f16)?
};
let clip_config = if first {
&sd_config.clip
} else {
sd_config.clip2.as_ref().unwrap()
};
let text_model = stable_diffusion::build_clip_transformer(clip_config, clip_weights, device, DType::F32)?;
let text_embeddings = text_model.forward(&tokens)?;
let text_embeddings = if use_guide_scale {
let mut uncond_tokens = tokenizer
.encode(uncond_prompt, true)
.map_err(E::msg)?
.get_ids()
.to_vec();
if uncond_tokens.len() > sd_config.clip.max_position_embeddings {
anyhow::bail!(
"the negative prompt is too long, {} > max-tokens ({})",
uncond_tokens.len(),
sd_config.clip.max_position_embeddings
)
}
while uncond_tokens.len() < sd_config.clip.max_position_embeddings {
uncond_tokens.push(pad_id)
}
let uncond_tokens = Tensor::new(uncond_tokens.as_slice(), device)?.unsqueeze(0)?;
let uncond_embeddings = text_model.forward(&uncond_tokens)?;
Tensor::cat(&[uncond_embeddings, text_embeddings], 0)?.to_dtype(dtype)?
} else {
text_embeddings.to_dtype(dtype)?
};
Ok(text_embeddings)
}
@@ -0,0 +1,3 @@
pub mod get_vae_scale;
pub mod infer_clip_text_embeddings;
pub mod stable_diffusion_pipeline;
@@ -0,0 +1,199 @@
use std::sync::{RwLock, RwLockReadGuard, RwLockWriteGuard};
use crate::ml::image::dynamic_image_to_tensor::dynamic_image_to_tensor;
use crate::ml::image::tensor_to_image_buffer::{tensor_to_image_buffer, RgbImage};
use crate::ml::model_cache::ModelCache;
use crate::ml::model_file::StableDiffusionVersion;
use crate::ml::prompt_cache::PromptCache;
use crate::ml::stable_diffusion::get_vae_scale::get_vae_scale;
use crate::ml::stable_diffusion::infer_clip_text_embeddings::infer_clip_text_embeddings;
use crate::state::app_config::AppConfig;
use anyhow::{anyhow, Error as E, Result};
use candle_core::{DType, IndexOp, Tensor, D};
use candle_transformers::models::stable_diffusion::vae::DiagonalGaussianDistribution;
use image::DynamicImage;
use log::info;
use rand::Rng;
pub struct Args<'a> {
pub image: &'a DynamicImage,
pub prompt: String,
pub uncond_prompt: String,
pub cfg_scale: Option<f64>,
pub i2i_strength: Option<u8>,
pub configs: &'a AppConfig,
pub model_cache: &'a ModelCache,
pub prompt_cache: &'a PromptCache,
}
pub fn stable_diffusion_pipeline(args: Args<'_>) -> Result<RgbImage> {
println!("Starting image generation with the following configuration:");
println!(" Model: {:?}", args.configs.sd_version);
println!(" Prompt: {}", args.prompt);
println!(" Steps: {}", args.configs.scheduler_steps);
println!(" Device: {:?}", args.configs.device);
println!("Model dimensions: {}x{}", args.configs.sd_config.width, args.configs.sd_config.height);
// TODO(bt,2025-02-18): The scheduler is `EulerAncestralDiscreteScheduler`, but we may want to port an LCM scheduler.
// This is a target for performance improvement
// See: https://github.com/huggingface/candle/issues/1331
let mut scheduler = args.configs.sd_config.build_scheduler(
args.configs.scheduler_steps)?;
let seed = args.configs.seed.unwrap_or_else(|| rand::thread_rng().gen());
args.configs.device.set_seed(seed)?;
println!("Using seed: {}", seed);
let guidance_scale = match args.cfg_scale {
Some(guidance_scale) => guidance_scale,
None => match args.configs.sd_version {
StableDiffusionVersion::V1_5
| StableDiffusionVersion::V2_1
| StableDiffusionVersion::Xl => 7.5,
StableDiffusionVersion::Turbo => 0.,
_ => 0., // NB: Not sure what the other model families should use, so sticking with "0"
},
};
let use_guide_scale = guidance_scale > 1.0;
println!("Using guide scale: {}", use_guide_scale);
info!("Checking if prompt is cached");
let maybe_cached = args.prompt_cache.get_copy(&args.prompt)?;
let mut text_embeddings = if let Some(tensor) = maybe_cached {
tensor
} else {
info!("Prompt is NOT cached! Calculating embedding...");
let tensor = infer_clip_text_embeddings(
&args.prompt,
&args.uncond_prompt,
None, // tokenizer
None, // clip_weights
None, // clip2_weights
args.configs.sd_version,
&args.configs.sd_config,
false, // use_f16
&args.configs.device,
args.configs.dtype,
use_guide_scale,
)?;
args.prompt_cache.store_copy(&args.prompt, &tensor)?;
tensor
};
println!("Text embeddings shape: {:?}", text_embeddings.shape());
println!("Loading input image into tensor...");
let input_image = dynamic_image_to_tensor(
args.image,
&args.configs.device,
args.configs.dtype)?;
println!("Reference image shape: {:?}", input_image.shape());
let vae_scale = get_vae_scale(args.configs.sd_version);
let init_latent_dist : DiagonalGaussianDistribution = args.model_cache.vae_encode(&input_image)?;
// TODO(bt,2025-02-18): This takes a little bit to generate the sample.
// This is a target for performance improvement
println!("Generating latents from input image...");
let latents = (init_latent_dist.sample()? * vae_scale)?;
//.to_device(&args.configs.device)?;
println!("Initial latents shape: {:?}", latents.shape());
println!("Calculating start step for diffusion process...");
let img2img_strength = match args.i2i_strength {
None => 0.75f64,
Some(strength) => (strength as f64) / 100.0f64,
};
let t_start = {
let start = args.configs.scheduler_steps - (args.configs.scheduler_steps as f64 * img2img_strength) as usize;
println!("Starting from step {} of {} (strength: {})", start, args.configs.scheduler_steps, img2img_strength);
start
};
let latents = if t_start < scheduler.timesteps().len() {
println!("Adding noise to latents...");
let noise = latents.randn_like(0f64, 1f64)?;
scheduler.add_noise(&latents, noise, scheduler.timesteps()[t_start])?
} else {
println!("Using latents directly (no noise addition needed)");
latents
};
println!("Latents initialized successfully");
let timesteps: Vec<_> = scheduler.timesteps().iter().copied().collect();
// Initial noise shape: [1, 4, 64, 64]
println!("Initial noise shape: {:?}", latents.shape());
println!("Starting diffusion process...");
let mut latents = latents;
for (timestep_index, &timestep) in timesteps.iter().enumerate() {
if timestep_index < t_start {
continue;
}
println!("Processing step {}/{} @ timestamp = {}", timestep_index + 1, timesteps.len(), timestep);
let latent_model_input = if use_guide_scale {
Tensor::cat(&[&latents, &latents], 0)?
} else {
latents.clone() // TODO(bt,2025-08-18): Do we have to clone the tensor? `scale_model_input` requires ownership
};
let latent_model_input = scheduler.scale_model_input(latent_model_input, timestep)?;
let mut noise_pred = match args.model_cache.unet_inference(&latent_model_input, timestep as f64, &text_embeddings) {
Ok(pred) => pred,
Err(e) => {
println!("UNet inference failed with error: {}", e);
return Err(anyhow::anyhow!("UNet inference failed: {}", e));
},
};
if use_guide_scale {
let chunks = noise_pred.chunk(2, 0)?;
let (noise_pred_uncond, noise_pred_text) = (&chunks[0], &chunks[1]);
noise_pred = (noise_pred_uncond + (guidance_scale * (noise_pred_text - noise_pred_uncond)?)?)?;
}
latents = scheduler.step(&noise_pred, timestep, &latents)?;
}
println!("Diffusion process completed, decoding image...");
let image = args.model_cache.vae_decode(&(latents / vae_scale)?)?;
println!("VAE decode completed");
println!("Scaling image back");
let image = ((image / 2.)? + 0.5)?;
// TODO(bt,2025-08-18): This normalization is slow.
// This is a target for performance improvement
println!("Normalized image values");
let image = (image.clamp(0f32, 1.)? * 255.)?;
println!("Convert to int8");
let image = image.to_dtype(DType::U8)?;
println!("Converted to 8-bit format");
let image = tensor_to_image_buffer(&image.i(0)?)?;
Ok(image)
}
@@ -0,0 +1,96 @@
use crate::ml::model_file::StableDiffusionVersion;
use crate::state::yaml_config::YamlConfig;
use candle_core::{DType, Device};
use candle_transformers::models::stable_diffusion::StableDiffusionConfig;
use filesys::recursively_find_file_by_name::recursively_find_file_by_name;
use hf_hub::api::sync::Api;
use log::info;
const CONFIG_FILENAME : &str = "app_config.yaml";
const DEFAULT_SD_IMAGE_WIDTH: usize = 512;
const DEFAULT_SD_IMAGE_HEIGHT: usize = 512;
const DEFAULT_SD_NUMERIC_DATATYPE: DType = DType::F32;
const SD_VERSION: StableDiffusionVersion = StableDiffusionVersion::Turbo;
/// A central place to configure app-wide details.
pub struct AppConfig {
/// We only support one device type currently, and we can't multiplex
/// over several devices yet.
pub device: Device,
/// The core numeric type to use for models.
pub dtype: DType,
pub sd_version: StableDiffusionVersion,
pub sd_config: StableDiffusionConfig,
pub image_height: usize,
pub image_width: usize,
pub scheduler_steps: usize,
pub scheduler_samples: usize,
pub seed: Option<u64>,
pub cfg_scale: Option<f64>,
/// Probably shouldn't be here.
pub hf_api: Api,
}
impl AppConfig {
pub fn init() -> anyhow::Result<Self> {
println!("Configuration filename: {}", CONFIG_FILENAME);
let yaml_configs = load_yaml_configs();
println!("Configs: {:?}", yaml_configs);
let device = Device::new_cuda(0)
.unwrap_or_else(|e| {
info!("CUDA not available ({}), falling back to CPU", e);
Device::Cpu
});
let sd_config = match SD_VERSION {
StableDiffusionVersion::Turbo => {
StableDiffusionConfig::sdxl_turbo(None, Some(DEFAULT_SD_IMAGE_HEIGHT), Some(DEFAULT_SD_IMAGE_WIDTH))
}
_ => {
StableDiffusionConfig::v2_1(None, Some(DEFAULT_SD_IMAGE_HEIGHT), Some(DEFAULT_SD_IMAGE_WIDTH))
}
};
let hf_api = Api::new()?;
Ok(Self {
device,
dtype: DEFAULT_SD_NUMERIC_DATATYPE,
sd_version: SD_VERSION,
sd_config,
image_height: yaml_configs.image_height.unwrap_or(DEFAULT_SD_IMAGE_HEIGHT),
image_width: yaml_configs.image_width.unwrap_or(DEFAULT_SD_IMAGE_WIDTH),
scheduler_steps: yaml_configs.scheduler_steps.unwrap_or(1),
scheduler_samples: yaml_configs.scheduler_samples.unwrap_or(15),
seed: yaml_configs.seed,
cfg_scale: yaml_configs.cfg_scale,
hf_api,
})
}
}
fn load_yaml_configs() -> YamlConfig {
let maybe_path = recursively_find_file_by_name(CONFIG_FILENAME, "../../../", 5)
.ok()
.flatten();
if let Some(path) = maybe_path {
let path = path.canonicalize().unwrap_or(path);
println!("Attempting to load configs from: {:?}", &path);
if let Ok(config) = YamlConfig::read_from_file(path) {
return config;
}
}
println!("Loading default configs.");
YamlConfig::default()
}
@@ -0,0 +1,2 @@
pub mod yaml_config;
pub mod app_config;
@@ -0,0 +1,25 @@
use std::fs::File;
use std::io::BufReader;
use std::path::Path;
/// Configuration read or derived from a YAML config file at startup
/// This is not meant to contain any runtime config values.
/// This may not be exhaustive.
#[derive(serde::Deserialize, serde::Serialize, Clone, Debug, Default)]
pub struct YamlConfig {
pub image_height: Option<usize>,
pub image_width: Option<usize>,
pub scheduler_steps: Option<usize>,
pub scheduler_samples: Option<usize>,
pub seed: Option<u64>,
pub cfg_scale: Option<f64>,
}
impl YamlConfig {
pub fn read_from_file<P: AsRef<Path>>(path: P) -> anyhow::Result<Self> {
let file = File::open(path)?;
let reader = BufReader::new(file);
let config = serde_yaml::from_reader(reader)?;
Ok(config)
}
}
+4 -4
View File
@@ -1,6 +1,6 @@
{
"$schema": "https://schema.tauri.app/config/2.0.0-rc",
"productName": "Storyteller Desktop App",
"productName": "Realtime Desktop App",
"version": "0.1.0",
"identifier": "com.unique.value.thing.why",
"build": {
@@ -12,9 +12,9 @@
"app": {
"windows": [
{
"title": "Storyteller Desktop App Title Goes Here",
"width": 800,
"height": 600,
"title": "Realtime Desktop App Title Goes Here",
"width": 1200,
"height": 800,
"resizable": true,
"fullscreen": false
}
+1
View File
@@ -31,4 +31,5 @@ pub mod file_size;
pub mod filename_concat;
pub mod is_filesystem_full_error;
pub mod path_to_string;
pub mod recursively_find_file_by_name;
pub mod rename_across_devices;
@@ -0,0 +1,56 @@
use std::path::{Path, PathBuf};
use walkdir::WalkDir;
use errors::AnyhowResult;
pub fn recursively_find_file_by_name<P: AsRef<Path>, Q: AsRef<Path>>(
search_filename: P,
search_root: Q,
max_depth: usize,
) -> AnyhowResult<Option<PathBuf>> {
let walker = WalkDir::new(search_root)
.max_depth(max_depth);
for entry in walker {
let entry = entry?;
if entry.path().is_dir() {
let check_path = entry.path().join(search_filename.as_ref());
if check_path.exists() && check_path.is_file() {
return Ok(Some(check_path));
}
}
}
Ok(None)
}
#[cfg(test)]
mod tests {
use testing::test_file_path::test_file_path;
use crate::recursively_find_file_by_name::recursively_find_file_by_name;
#[test]
fn test_path_exists() {
let test_data_path = test_file_path("test_data/").expect("should work");
let result = recursively_find_file_by_name(
"super_mario_rpg_beware_the_forests_mushrooms.mp3",
test_data_path,
5
);
let result = result.expect("should be ok");
let result = result.expect("should exist");
assert!(result.ends_with("test_data/audio/mp3/super_mario_rpg_beware_the_forests_mushrooms.mp3"));
}
#[test]
fn test_path_does_not_exist() {
let test_data_path = test_file_path("test_data/").expect("should work");
let result = recursively_find_file_by_name(
"fake_name.foo",
test_data_path,
5
);
let result = result.expect("should be ok");
assert!(result.is_none());
}
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 3.5 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.7 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 1.5 MiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 27 KiB

+118 -107
View File
@@ -1,9 +1,7 @@
import Konva from "konva";
import { RenderEngine } from "./RenderingPrimitives/RenderEngine";
import { ResponseType } from "./WorkerPrimitives/SharedWorkerBase";
import { uiAccess, uiEvents } from "~/signals";
import { ShapeNode } from "./Nodes";
import { UndoStackManager } from "./UndoRedo";
import { CommandManager, MatteBox, SceneManager } from "./EngineUtitlities";
@@ -20,22 +18,17 @@ import {
import { ImageNode, VideoNode, TextNode } from "./Nodes";
import { EngineOptions, TextNodeData, VideoNodeData } from "./types";
import { SharedWorkerResponse } from "./WorkerPrimitives/SharedWorkerBase";
import {
DiffusionSharedWorkerProgressData,
DiffusionSharedWorkerResponseData,
} from "./SharedWorkers/Diffusion/DiffusionSharedWorker";
import { AppModes, VideoResolutions } from "./constants";
import { ToolbarMainButtonNames } from "~/components/features/ToolbarMain/enum";
import { ToolbarNodeButtonNames } from "~/components/features/ToolbarNode/enums";
import { NavigateFunction } from "react-router-dom";
import { LoadingVideosProvider } from "./EngineUtitlities/LoadingVideosProvider";
import { MediaFile } from "~/Classes/ApiManager/models/MediaFile";
import { VideoExtractionHandler } from "./EngineUtitlities/VideoExtractionHandler/VideoExtractionHandler";
import { RealTimeDrawEngine } from "./RenderingPrimitives/RealTimeDrawEngine";
import { NodeColor } from "~/signals/uiEvents/toolbarNode";
import { ShapeType } from "./Nodes";
// for testing loading files from system
// import { FileUtilities } from "./FileUtilities/FileUtilities";
@@ -61,7 +54,6 @@ export class Engine {
private nodeIsolationLayer: Konva.Layer;
private uiLayer: Konva.Layer;
private previewLayer: Konva.Layer;
private renderEngine: RenderEngine;
private offScreenCanvas: OffscreenCanvas;
private realTimeDrawEngine: RealTimeDrawEngine;
private nodesManager: NodesManager;
@@ -127,9 +119,6 @@ export class Engine {
mediaLayerRef: this.mediaLayer,
bgLayerRef: this.bgLayer,
offScreenCanvas: this.offScreenCanvas,
previewLayerRef: this.previewLayer,
onRenderingSystemMessageRecieved:
this.onRenderingSystemReceived.bind(this),
});
// Collection of all Nodes
@@ -191,78 +180,6 @@ export class Engine {
this.setAppMode(AppModes.SELECT);
}
// TODO write code to show error and retry.
onRenderingSystemReceived(
response: SharedWorkerResponse<
DiffusionSharedWorkerResponseData | MediaFile,
DiffusionSharedWorkerProgressData
>,
) {
if (!response.data) {
// throw error to retry
uiAccess.dialogError.show({
title: "Generation Error",
message: response.data?.toString(),
});
uiAccess.toolbarMain.loadingBar.hide();
this.setAppMode(AppModes.SELECT);
return;
}
if (response.responseType === ResponseType.error) {
console.log("Error Data?");
console.log(response.data, response);
uiAccess.dialogError.show({
title: "Generation Error Try again.",
message: response.data?.toString(),
});
uiAccess.toolbarMain.loadingBar.hide();
this.setAppMode(AppModes.SELECT);
return;
}
if (response.responseType === ResponseType.result) {
console.log(response.data);
const data = response.data;
// create video node here.
// choose it to be the size of the rendering output, this case its mobile. (1560, 400)
if (typeof data === "string" || data === undefined) {
return;
}
if ("videoUrl" in data) {
console.log("Engine got stylized video: " + data.videoUrl);
this.addVideo({ mediaFileUrl: data.videoUrl });
} else if ("media_links" in data) {
console.log("Engine got rendered video: " + data.media_links.cdn_url);
downloadURI(data.media_links.cdn_url, "Download Video");
}
// hide the loader
//this.realTimeDrawEngine.videoLoadingCanvas.kNode.hide();
uiAccess.toolbarMain.loadingBar.hide();
this.setAppMode(AppModes.SELECT);
return;
}
if (response.responseType === ResponseType.progress) {
const data = response.data as DiffusionSharedWorkerProgressData;
// TODO wil fix this ?!?! parameter issue
//this.realTimeDrawEngine.videoLoadingCanvas.kNode.show();
uiAccess.toolbarMain.loadingBar.update({
message: "Rendering Frames...",
progress: data.progress * 100,
isShowing: true,
});
// console.log(response);
// if (response.data.zipBlob) {
// FileUtilities.downloadBlobZip(response.data.zipBlob);
// }
return;
}
}
private setAppMode(newAppMode: AppModes) {
this.appMode = newAppMode;
switch (this.appMode) {
@@ -441,9 +358,23 @@ export class Engine {
// Listen to Toolbar Main
uiEvents.toolbarMain.UNDO.onClick(() => this.undoStackManager.undo());
uiEvents.toolbarMain.REDO.onClick(() => this.undoStackManager.redo());
uiEvents.toolbarMain.SAVE.onClick(async (/*event*/) => {
this.sceneManager.saveScene();
await this.realTimeDrawEngine.saveOutput();
});
uiEvents.toolbarMain.SELECT.onClick(() => {
console.log("Toolbar Main >> Select");
});
uiEvents.toolbarMain.PAINT.onClick(() => {
console.log("Toolbar Main >> Paint");
});
uiEvents.toolbarMain.ERASER.onClick(() => {
console.log("Toolbar Main >> Eraser");
});
uiEvents.toolbarMain.PREVIEW.onClick(async () => {
console.log("Toolbar Main >> Preview");
this.setAppMode(AppModes.PREVIEW);
@@ -472,10 +403,6 @@ export class Engine {
this.addImage(image);
});
uiEvents.onGetStagedVideo((videoData) => {
console.log("Engine got user video: " + videoData.mediaFileUrl);
this.addVideo(videoData);
});
uiEvents.onAddTextToEngine((textdata) => {
this.addText(textdata);
});
@@ -506,21 +433,41 @@ export class Engine {
}
}
});
uiEvents.aiStylize.onRequest(async (data) => {
console.log("Engine heard AI Stylize request: ", data);
try {
this.setAppMode(AppModes.RENDERING);
//await this.realTimeDrawEngine.startProcessing(data);
} catch (error) {
// throw error to retry
uiAccess.dialogError.show({
title: "Generation Error",
message: error?.toString() || "Unknown Error",
});
this.setAppMode(AppModes.SELECT);
uiEvents.promptEvents.onPromptStrengthChanged((strength) => {
this.realTimeDrawEngine.currentStrength = strength;
});
uiEvents.promptEvents.onPromptTextChanged((prompt) => {
this.realTimeDrawEngine.currentPrompt = prompt;
});
uiEvents.onAddShapeToEngine((shapeData) => {
switch (shapeData.shape) {
case "circle":
this.addShape(ShapeType.CIRCLE, 100);
break;
case "square":
this.addShape(ShapeType.SQUARE, 100);
break;
case "triangle":
this.addShape(ShapeType.TRIANGLE, 100);
break;
}
});
uiEvents.toolbarNode.color.onConfirmChanged((nodeColor) => {
console.log("Color change event triggered with:", nodeColor);
if (!nodeColor) {
console.warn("nodeColor is undefined or null");
return;
}
if (!nodeColor.kNodeId) {
console.warn("nodeColor.kNodeId is undefined or null");
return;
}
this.changeNodeColor(nodeColor);
});
}
disableAllButtons() {
@@ -584,9 +531,9 @@ export class Engine {
x: 10,
y: 80,
text: "",
fontSize: 24,
fontSize: 18,
fontFamily: "Source Sans 3",
fill: "black",
fill: "white",
});
const anim = new Konva.Animation((frame) => {
if (frame && import.meta.env.DEV) {
@@ -603,6 +550,42 @@ export class Engine {
this.addKeyboardShortcuts();
}
private changeNodeColor(nodeColor: NodeColor) {
console.log("Changing node color:", nodeColor);
// Find the node that is selected with the id
const idSelector = "#" + nodeColor.kNodeId;
const selectedNode = this.mediaLayer.findOne(idSelector);
console.log("Selected node:", selectedNode);
console.log("ID selector:", idSelector);
console.log("Media layer:", this.mediaLayer);
// TODO: Change the color of the node
// Turns out shapes are image nodes... how do we change the image's fill??
if (selectedNode) {
// Get the shape node from the real time drawing engine's image nodes
const shapeNode = this.realTimeDrawEngine.findImageNodeById(
nodeColor.kNodeId,
);
if (shapeNode && shapeNode instanceof ShapeNode) {
// Create new shape node with updated color
const newShapeNode = new ShapeNode({
canvasPosition: this.realTimeDrawEngine.captureCanvas.position(),
canvasSize: this.realTimeDrawEngine.captureCanvas.size(),
shapeType: shapeNode.shapeType,
size: shapeNode.size,
color: nodeColor.color,
mediaLayerRef: this.mediaLayer,
selectionManagerRef: this.selectionManager,
});
newShapeNode.kNode.position(shapeNode.kNode.position());
newShapeNode.kNode.zIndex(shapeNode.kNode.zIndex());
// Remove old node and add new one
shapeNode.kNode.destroy();
this.commandManager.createNode(newShapeNode);
}
}
}
public addText(textNodeData: TextNodeData) {
const textNode = new TextNode({
textNodeData: textNodeData,
@@ -614,6 +597,30 @@ export class Engine {
this.commandManager.createNode(textNode);
}
public addShape(type: ShapeType, size: number, color?: string) {
const shapeNode = new ShapeNode({
canvasPosition: this.realTimeDrawEngine.captureCanvas.position(),
canvasSize: this.realTimeDrawEngine.captureCanvas.size(),
shapeType: type,
size: size,
color: color,
mediaLayerRef: this.mediaLayer,
selectionManagerRef: this.selectionManager,
loaded: async () => {
await this.realTimeDrawEngine.render();
},
});
shapeNode.kNode.zIndex(1); // Replace desiredZIndex with a number
this.commandManager.createNode(shapeNode);
console.debug("Added shapenode:", shapeNode);
console.debug("Added node's ID:", shapeNode.kNode._id);
this.realTimeDrawEngine.addNodes(shapeNode);
}
public addImage(imageFile: File) {
const imageNode = new ImageNode({
mediaLayerRef: this.mediaLayer,
@@ -621,9 +628,13 @@ export class Engine {
canvasSize: this.realTimeDrawEngine.captureCanvas.size(),
imageFile: imageFile,
selectionManagerRef: this.selectionManager,
loaded: async () => {
await this.realTimeDrawEngine.render();
},
});
imageNode.kNode.zIndex(1);
this.commandManager.createNode(imageNode);
this.realTimeDrawEngine.addNodes(imageNode);
}
@@ -1,5 +1,5 @@
import Konva from "konva";
import { ImageNode, TextNode, VideoNode } from "../Nodes";
import { ImageNode, ShapeNode, TextNode, VideoNode } from "../Nodes";
import { MediaNode, Position, RGBColor, Transformation } from "../types";
import {
AddChromaKey,
@@ -48,7 +48,7 @@ export class CommandManager {
this.undoStackManagerRef = engineRefs.undoStackManagerRef;
}
createNode(node: VideoNode | ImageNode | TextNode) {
createNode(node: VideoNode | ImageNode | TextNode | ShapeNode) {
const command = new CreateCommand({
nodes: new Set<MediaNode>([node]),
mediaLayerRef: this.mediaLayerRef,
@@ -58,6 +58,10 @@ export class CommandManager {
renderEngineRef: this.renderEngineRef,
});
this.undoStackManagerRef.executeCommand(command);
// Set the kNode's manual id to same as kNode's internal _id
// This helps identify the kNode in the Konva layer with find
node.kNode.id(node.kNode._id.toString());
}
deleteNodes() {
const nodes = this.selectionManagerRef.getSelectedNodes();
@@ -29,7 +29,7 @@ export abstract class BaseNode {
}: {
selectionManagerRef: SelectionManager;
mediaLayerRef: Konva.Layer;
kNode: Konva.Image | Konva.Group;
kNode: Konva.Image | Konva.Group | Konva.Shape;
}) {
// console.log("Node constructed");
// this.uuid = uuidv4();
@@ -67,8 +67,11 @@ export abstract class BaseNode {
}
if (this.kNode instanceof Konva.Group) {
const wrapperRect = this.kNode.findOne(".wrapper") as Konva.Rect;
wrapperRect.stroke(primaryOrange);
wrapperRect.strokeWidth(highlightStrokeWidth);
console.log("Wrapper rect:", wrapperRect);
if (wrapperRect != null) {
wrapperRect.stroke(primaryOrange);
wrapperRect.strokeWidth(highlightStrokeWidth);
}
return;
}
if (import.meta.env.DEV) {
@@ -86,7 +89,11 @@ export abstract class BaseNode {
}
if (this.kNode instanceof Konva.Group) {
const wrapperRect = this.kNode.findOne(".wrapper") as Konva.Rect;
wrapperRect.strokeWidth(0);
console.log("Wrapper rect:", wrapperRect);
if (wrapperRect != null) {
// Issue investigate
wrapperRect.strokeWidth(0);
}
return;
}
if (import.meta.env.DEV) {
@@ -131,7 +138,7 @@ export abstract class BaseNode {
// console.log("handle select");
if (!isMultiSelect) {
// clear selection if not multislect
//console.log("No Shift >> no multiselect");
console.log("No Shift >> no multiselect");
this.selectionManagerRef.clearSelection();
}
if (this.selectionManagerRef.isNodeSelected(this)) {
@@ -144,8 +151,25 @@ export abstract class BaseNode {
}
};
const handleRightClick = () => {
this.selectionManagerRef.clearSelection();
this.selectionManagerRef.selectNode(this, undefined, true);
}
this.kNode.on("contextmenu", (e) => {
console.log("CONTEXT MENU", e);
e.evt.preventDefault();
handleRightClick();
});
this.kNode.on("mousedown", (e) => {
// console.log("MOUSE DOWN");
// https://developer.mozilla.org/en-US/docs/Web/API/UIEvent/which
// 3: Right mouse button
// If right click, let context event handle it
if (e.evt.button === 3) {
return;
}
// Selection of Node
if (!this.selectionManagerRef.isNodeSelected(this)) {
this.isSelecting = true;
@@ -155,7 +179,13 @@ export abstract class BaseNode {
});
this.kNode.on("mouseup", (e) => {
// console.log("MOUSE UP");
// https://developer.mozilla.org/en-US/docs/Web/API/UIEvent/which
// 3: Right mouse button
// If right click, let context event handle it
if (e.evt.button === 3) {
return;
}
// just coming out of dragging or selecting mode
// no need to handle select
if (this.selectionManagerRef.isDragging() || this.isSelecting) {
@@ -17,12 +17,13 @@ interface ImageNodeContructor {
transform?: TransformationData;
mediaLayerRef: Konva.Layer;
selectionManagerRef: SelectionManager;
loaded: () => Promise<void>;
}
export class ImageNode extends NetworkedNode {
public kNode: Konva.Image;
public imageSize?: Size;
private loaded!: () => Promise<void>;
constructor({
canvasPosition,
canvasSize,
@@ -32,6 +33,7 @@ export class ImageNode extends NetworkedNode {
transform: existingTransform,
mediaLayerRef,
selectionManagerRef,
loaded,
}: ImageNodeContructor) {
// kNodes need to be created first to guaruntee
// that it is not undefined in parent's context
@@ -53,6 +55,7 @@ export class ImageNode extends NetworkedNode {
localFile: imageFile,
});
this.kNode = kNode;
this.loaded = loaded;
// this.imageSize = minNodeSize;
this.mediaLayerRef.add(this.kNode);
@@ -83,8 +86,10 @@ export class ImageNode extends NetworkedNode {
}) {
const imageComponent = new Image();
imageComponent.crossOrigin = "anonymous";
// this.setProgress({ progress: 100, status: UploadStatus.SUCCESS });
imageComponent.onload = () => {
this.setProgress({ progress: 0, status: UploadStatus.FILE_STAGED });
//this.setProgress({ progress: 0, status: UploadStatus.FILE_STAGED });
this.imageSize = {
width: imageComponent.width,
height: imageComponent.height,
@@ -105,33 +110,34 @@ export class ImageNode extends NetworkedNode {
this.kNode.fill(transparent);
this.listenToBaseKNode();
this.mediaLayerRef.draw();
this.uploadImage(imageFile);
//this.uploadImage(imageFile);
this.loaded();
};
imageComponent.onerror = () => {
this.setProgress({ progress: 0, status: UploadStatus.ERROR_ON_FILE });
//this.setProgress({ progress: 0, status: UploadStatus.ERROR_ON_FILE });
};
imageComponent.src = URL.createObjectURL(imageFile);
}
private async loadImageFromUrl(mediaFileUrl: string) {
this.setProgress({ progress: 75, status: UploadStatus.LOADING });
//this.setProgress({ progress: 75, status: UploadStatus.LOADING });
const newImage = new Image();
newImage.crossOrigin = "anonymous";
newImage.onerror = () => {
this.setProgress({ progress: 90, status: UploadStatus.ERROR_ON_LOAD });
//this.setProgress({ progress: 90, status: UploadStatus.ERROR_ON_LOAD });
};
newImage.onload = () => {
// console.log("network image", newImage);
this.kNode.image(newImage);
this.kNode.draw();
this.setProgress({ progress: 100, status: UploadStatus.SUCCESS });
//this.setProgress({ progress: 100, status: UploadStatus.SUCCESS });
};
newImage.src = mediaFileUrl;
this.listenToBaseKNode();
}
private async uploadImage(imageFile: File) {
this.setProgress({ progress: 10, status: UploadStatus.UPLOADING });
//this.setProgress({ progress: 10, status: UploadStatus.UPLOADING });
const mediaUploadApi = new MediaUploadApi();
const uploadResponse = await mediaUploadApi.UploadImage({
@@ -149,7 +155,7 @@ export class ImageNode extends NetworkedNode {
}
private async retreiveImage(mediaFileToken: string) {
this.setProgress({ progress: 50, status: UploadStatus.RETREIVING });
//this.setProgress({ progress: 50, status: UploadStatus.RETREIVING });
const mediaFileApi = new MediaFilesApi();
const mediaFileResponse = await mediaFileApi.GetMediaFileByToken({
mediaFileToken: mediaFileToken,
@@ -0,0 +1,206 @@
import Konva from "konva";
import { SelectionManager } from "../NodesManagers";
import { Position, Size, NodeData, TransformationData } from "../types";
import { NodeType } from "./constants";
import { NodeUtilities } from "./NodeUtilities";
import { NetworkedNode } from "./NetworkedNode";
export enum ShapeType {
CIRCLE = "circle",
SQUARE = "square",
TRIANGLE = "triangle",
}
interface ShapeNodeConstructor {
canvasPosition: Position;
canvasSize: Size;
shapeType: ShapeType;
size: number;
color?: string; // Hex color string
transform?: TransformationData;
mediaLayerRef: Konva.Layer;
selectionManagerRef: SelectionManager;
loaded: () => Promise<void>;
}
export class ShapeNode extends NetworkedNode {
retry(): void {
throw new Error("Method not implemented.");
}
public shapeType: ShapeType;
public size: number;
constructor({
canvasPosition,
canvasSize,
shapeType,
size,
color = "#2D81FF", // Default red if no color provided
transform: existingTransform,
mediaLayerRef,
selectionManagerRef,
loaded,
}: ShapeNodeConstructor) {
// Create the actual shape inside the group
const transform = NodeUtilities.getInitialTransform({
existingTransform,
canvasPosition,
canvasSize,
});
let shape = null;
switch (shapeType) {
case ShapeType.CIRCLE:
const circleShape = new Konva.Circle({
radius: 50,
...transform,
fill: color,
strokeScaleEnabled: false,
draggable: true,
});
// Convert circle to image
const dataURL = circleShape.toDataURL();
const imageObj = new Image();
imageObj.src = dataURL;
shape = new Konva.Image({
image: imageObj,
x: circleShape.x(),
y: circleShape.y(),
width: circleShape.width(),
height: circleShape.height(),
...transform,
fill: "transparent",
strokeScaleEnabled: false,
draggable: true,
});
imageObj.onload = () => {
this.mediaLayerRef.draw();
loaded();
};
// Clean up the temporary circle
circleShape.destroy();
break;
case ShapeType.SQUARE:
const squareShape = new Konva.Rect({
width: 100,
height: 100,
...transform,
fill: color,
strokeScaleEnabled: false,
draggable: true,
});
// Convert square to image
const squareDataURL = squareShape.toDataURL();
const squareImageObj = new Image();
squareImageObj.src = squareDataURL;
shape = new Konva.Image({
image: squareImageObj,
x: squareShape.x(),
y: squareShape.y(),
width: squareShape.width(),
height: squareShape.height(),
...transform,
fill: "transparent",
strokeScaleEnabled: false,
draggable: true,
});
squareImageObj.onload = () => {
this.mediaLayerRef.draw();
loaded()
};
// Clean up the temporary square
squareShape.destroy();
break;
case ShapeType.TRIANGLE:
const triangleShape = new Konva.RegularPolygon({
sides: 3,
radius: 50,
...transform,
fill: color,
strokeScaleEnabled: false,
draggable: true,
});
// Convert triangle to image
const triangleDataURL = triangleShape.toDataURL();
const triangleImageObj = new Image();
triangleImageObj.src = triangleDataURL;
shape = new Konva.Image({
image: triangleImageObj,
x: triangleShape.x(),
y: triangleShape.y(),
width: triangleShape.width(),
height: triangleShape.height(),
...transform,
fill: "transparent",
strokeScaleEnabled: false,
draggable: true,
});
triangleImageObj.onload = () => {
this.mediaLayerRef.draw();
loaded()
};
// Clean up the temporary triangle
triangleShape.destroy();
break;
default:
throw new Error("Invalid shape type");
}
super({
selectionManagerRef: selectionManagerRef,
mediaLayerRef: mediaLayerRef,
kNode: shape,
});
this.shapeType = shapeType;
this.size = size;
// Add shape to group
const centerPosition = NodeUtilities.positionNodeOnCanvasCenter({
canvasOffset: canvasPosition,
componentSize: canvasSize,
maxSize: canvasSize,
});
this.kNode.setPosition(centerPosition);
this.mediaLayerRef.add(this.kNode);
this.listenToBaseKNode();
this.mediaLayerRef.draw();
}
public getNodeData(canvasPosition: Position) {
const data: NodeData = {
type: NodeType.SHAPE,
transform: {
position: {
x: this.kNode.position().x - canvasPosition.x,
y: this.kNode.position().y - canvasPosition.y,
},
size: this.kNode.size(),
rotation: this.kNode.rotation(),
scale: {
x: this.kNode.scaleX(),
y: this.kNode.scaleY(),
},
zIndex: this.kNode.getZIndex(),
},
};
return data;
}
}
@@ -4,11 +4,12 @@ export enum NodeType {
IMAGE = "image",
TEXT = "text",
VIDEO = "video",
SHAPE = "shape",
}
export const transparent = Colors.transparent;
export const primaryOrange = Colors.primaryOrange;
export const highlightStrokeWidth = 10;
export const highlightStrokeWidth = 2;
export const minNodeSize = {
width: 200,
height: 200,
@@ -3,3 +3,4 @@ export { BaseNode } from "./BaseNode";
export { ImageNode } from "./ImageNode";
export { VideoNode } from "./VideoNode";
export { TextNode } from "./TextNode";
export { ShapeNode , ShapeType} from "./ShapeNode";
@@ -4,11 +4,12 @@ import { MediaNode, Position, Transformation } from "../types";
import { uiAccess, uiEvents } from "~/signals";
import { NetworkedNode } from "../Nodes/NetworkedNode";
import { LoadingBarStatus } from "~/components/ui";
import { ImageNode, TextNode, VideoNode } from "../Nodes";
import { ImageNode, ShapeNode, TextNode, VideoNode } from "../Nodes";
import {
calculateContextualsPosition,
getImageNodeButtonStates,
getMultiSelectButtonStates,
getShapeNodeButtonStates,
getTextNodeButtonStates,
getVideoNodeButtonStates,
} from "./ToolbarNodeUtilities";
@@ -79,7 +80,7 @@ export class SelectionManager {
public isDisabled() {
return this.disable;
}
public selectNode(node: MediaNode, doNotDraw?: boolean): boolean {
public selectNode(node: MediaNode, doNotDraw?: boolean, showContextMenu = false): boolean {
if (
this.disabled || // no-op when disabled
this.selectedNodes.has(node) || // if the node is already selected
@@ -99,7 +100,11 @@ export class SelectionManager {
this.selectedNodes.add(node);
this.updateNodeTransformer();
this.updateContextComponents();
this.showContextComponents();
console.log("SelectionManager > selectNode > showContextMenu", showContextMenu);
if (showContextMenu) {
this.showContextComponents();
}
if (!doNotDraw) {
this.mediaLayerRef.batchDraw();
@@ -188,8 +193,8 @@ export class SelectionManager {
// done, clear data and show menu
this.initialPositions.clear();
this.updateContextComponents();
this.showContextComponents();
}
private getKNodeTransformation(kNode: Konva.Node) {
return {
kNodeId: kNode._id,
@@ -273,17 +278,29 @@ export class SelectionManager {
if (this.selectedNodes.size === 1) {
if (node instanceof ImageNode) {
showOrUpdate({
knodeIds: [node.kNode.id()],
locked: node.isLocked(),
buttonStates: getImageNodeButtonStates({ locked: node.isLocked() }),
});
}
if (node instanceof TextNode) {
else if (node instanceof ShapeNode) {
console.debug("Selected node is of type ShapeNode", node.kNode.id())
showOrUpdate({
knodeIds: [node.kNode.id()],
locked: node.isLocked(),
buttonStates: getTextNodeButtonStates({ locked: node.isLocked() }),
buttonStates: getShapeNodeButtonStates({ locked: node.isLocked() }),
color: node.getNodeData({ x: 0, y: 0 })?.textNodeData?.color ?? "#000000",
});
}
if (node instanceof VideoNode) {
else if (node instanceof TextNode) {
showOrUpdate({
knodeIds: [node.kNode.id()],
locked: node.isLocked(),
buttonStates: getTextNodeButtonStates({ locked: node.isLocked() }),
color: node.getNodeData({ x: 0, y: 0 })?.textNodeData?.color ?? "#000000",
});
}
else if (node instanceof VideoNode) {
showOrUpdate({
locked: node.isLocked(),
buttonStates: getVideoNodeButtonStates({ locked: node.isLocked() }),
@@ -40,8 +40,9 @@ export class SelectorSquare {
stageRef: Konva.Stage;
}) {
this.kSquare = new Konva.Rect({
fill: " rgb(250, 128, 114, 0.3)",
stroke: "salmon",
fill: "rgba(45, 129, 255, 0.3)",
stroke: "#00a1ff",
strokeWidth: 1,
visible: false,
// disable events to not interrupt with events
listening: false,
@@ -51,7 +51,8 @@ export function getImageNodeButtonStates(
buttonName === ButtonNames.AI_STYLIZE ||
buttonName === ButtonNames.SEGMENTATION ||
buttonName === ButtonNames.DOWNLOAD ||
buttonName === ButtonNames.CHROMA
buttonName === ButtonNames.CHROMA ||
buttonName === ButtonNames.COLOR
) {
buttonStates[buttonName] = {
disabled: true,
@@ -130,7 +131,9 @@ export function getVideoNodeButtonStates(
return buttonStates;
}
// soon to come feature is disabled
if (buttonName === ButtonNames.AI_STYLIZE) {
if (buttonName === ButtonNames.AI_STYLIZE ||
buttonName === ButtonNames.COLOR
) {
buttonStates[buttonName] = {
disabled: true,
hidden: true,
@@ -149,7 +152,8 @@ export function getVideoNodeButtonStates(
{} as ContextualToolbarProps["buttonStates"],
);
}
export function getMultiSelectButtonStates(
export function getShapeNodeButtonStates(
props: { locked?: boolean | "unknown" } = {},
) {
const ButtonNames = ToolbarNodeButtonNames;
@@ -157,7 +161,7 @@ export function getMultiSelectButtonStates(
(buttonStates, buttonName) => {
// transform button depends on locked state
if (props.locked !== undefined && buttonName === ButtonNames.TRANSFORM) {
buttonStates[buttonName] = {
buttonStates.TRANSFORM = {
disabled: props.locked === "unknown" || props.locked === true,
hidden: false,
active: true,
@@ -191,3 +195,47 @@ export function getMultiSelectButtonStates(
{} as ContextualToolbarProps["buttonStates"],
);
}
export function getMultiSelectButtonStates(
props: { locked?: boolean | "unknown" } = {},
) {
const ButtonNames = ToolbarNodeButtonNames;
return Object.values(ButtonNames).reduce(
(buttonStates, buttonName) => {
// transform button depends on locked state
if (props.locked !== undefined && buttonName === ButtonNames.TRANSFORM) {
buttonStates[buttonName] = {
disabled: props.locked === "unknown" || props.locked === true,
hidden: false,
active: true,
};
return buttonStates;
}
// hidden buttons
if (
buttonName === ButtonNames.AI_STYLIZE ||
buttonName === ButtonNames.SEGMENTATION ||
buttonName === ButtonNames.DOWNLOAD ||
buttonName === ButtonNames.CHROMA ||
buttonName === ButtonNames.COLOR
) {
buttonStates[buttonName] = {
disabled: true,
hidden: true,
active: false,
};
return buttonStates;
}
// all other buttons
buttonStates[buttonName] = {
disabled: false,
hidden: false,
active: false,
};
return buttonStates;
},
{} as ContextualToolbarProps["buttonStates"],
);
}
@@ -6,7 +6,7 @@ import { Group } from "konva/lib/Group";
import { invoke } from "@tauri-apps/api/core";
import { FileUtilities } from "../FileUtilities/FileUtilities";
import { ImageNode, VideoNode, TextNode } from "../Nodes";
import { ImageNode, VideoNode, TextNode, ShapeNode, ShapeType } from "../Nodes";
import { MediaNode } from "../types";
import { RenderTask } from "./RenderTask";
@@ -16,14 +16,15 @@ import { OffScreenSceneCanvas } from "./OffScreenSceneCanvas";
export class RealTimeDrawEngine {
private videoNodes: VideoNode[];
private imageNodes: (ImageNode | TextNode)[];
private imageNodes: (ImageNode | TextNode | ShapeNode)[];
private offScreenCanvas: OffscreenCanvas;
private outputBitmap: ImageBitmap | undefined;
// private frames: ImageBitmap[];
// capturing composite within window
private bgLayerRef: Konva.Layer;
private mediaLayerRef: Konva.Layer;
private height: number;
@@ -41,18 +42,19 @@ export class RealTimeDrawEngine {
public videoLoadingCanvas: VideoNode | undefined;
public fps: number = 24;
public currentPrompt: string;
public currentStrength: number;
constructor({
width,
height,
bgLayerRef,
mediaLayerRef,
offScreenCanvas,
}: {
width: number;
height: number;
bgLayerRef: Konva.Layer;
mediaLayerRef: Konva.Layer;
previewLayerRef: Konva.Layer;
offScreenCanvas: OffscreenCanvas;
}) {
this.videoLoadingCanvas = undefined;
@@ -74,18 +76,17 @@ export class RealTimeDrawEngine {
this.offScreenCanvas.width = this.width;
this.offScreenCanvas.height = this.height;
this.bgLayerRef = bgLayerRef;
// this is the whole canvas
this.mediaLayerRef = mediaLayerRef;
// this.previewLayerRef = previewLayerRef;
// Set background layer to red and media layer to green for visibility
this.port = undefined;
this.fps = 24;
this.currentPrompt = "";
this.currentStrength = 100;
// This is captures a subset of the medialayer ref
this.captureCanvas = new Konva.Rect({
name: "CaptureCanvas",
@@ -99,20 +100,6 @@ export class RealTimeDrawEngine {
draggable: false,
});
// Add preview canvas with same dimensions but different style
// this.previewCanvas = new Konva.Rect({
// name: "PreviewCanvas",
// x: this.positionX,
// y: this.positionY,
// width: this.width,
// height: this.height,
// fill: "white",
// stroke: "blue",
// strokeWidth: 1,
// dash: [5, 5],
// draggable: false,
// });
this.previewCanvas = new Konva.Image({
name: "PreviewCanvas",
x: this.positionX,
@@ -120,21 +107,90 @@ export class RealTimeDrawEngine {
width: this.width,
height: this.height,
image: undefined,
stroke: "blue",
stroke: "black",
strokeWidth: 1,
dash: [5, 5],
draggable: false,
fill: "white",
});
this.bgLayerRef.add(this.captureCanvas);
this.bgLayerRef.add(this.previewCanvas);
this.mediaLayerRef.add(this.captureCanvas);
this.mediaLayerRef.add(this.previewCanvas);
// send back
this.captureCanvas.setZIndex(0);
this.previewCanvas.setZIndex(0);
this.previewCanvas.setZIndex(1);
// Add mouse events for preview canvas copying
//this.previewCopyListener();
//this.debug();
}
public previewCopyListener() {
this.previewCanvas.on('mousedown touchstart', () => {
if (!this.outputBitmap) {
console.log("No preview image to copy");
return;
}
// TODO create it as a node.
// const imageNode = new ImageNode({
// mediaLayerRef: this.mediaLayer,
// canvasPosition: this.renderEngine.captureCanvas.position(),
// canvasSize: this.renderEngine.captureCanvas.size(),
// imageFile: imageFile,
// selectionManagerRef: this.selectionManager,
// });
// Create draggable preview copy
const previewCopy = new Konva.Image({
x: this.previewCanvas.x(),
y: this.previewCanvas.y(),
width: this.width,
height: this.height,
image: this.outputBitmap,
draggable: true,
listening: true
});
this.mediaLayerRef.add(previewCopy);
previewCopy.moveToTop();
this.mediaLayerRef.draw();
// Start dragging immediately
previewCopy.startDrag();
// Handle drag events
previewCopy.on('dragmove', () => {
this.mediaLayerRef.draw();
});
previewCopy.on('dragend', () => {
const previewBox = previewCopy.getClientRect();
const captureBox = this.captureCanvas.getClientRect();
if (Konva.Util.haveIntersection(previewBox, captureBox)) {
// Snap to capture canvas position
previewCopy.position({
x: this.captureCanvas.x(),
y: this.captureCanvas.y()
});
this.mediaLayerRef.batchDraw();
} else {
// Remove if dropped outside capture area
previewCopy.destroy();
this.mediaLayerRef.batchDraw();
}
});
});
}
public findImageNodeById(id: string): (ImageNode | TextNode | ShapeNode | undefined) {
return this.imageNodes.find(node => {
if (node.kNode) {
return node.kNode.id() === id;
}
return false;
});
}
async updateCaptureCanvas(
width: number | undefined,
height: number | undefined,
@@ -158,7 +214,7 @@ export class RealTimeDrawEngine {
const oldPositionY = this.positionY;
// recompute the position
const padBetweenCaptureAndPreview = 10;
const padBetweenCaptureAndPreview = 2;
this.positionX =
window.innerWidth / 2 -
this.width / 2 -
@@ -266,24 +322,28 @@ export class RealTimeDrawEngine {
});
return canvas;
}
public isProcessing = false;
private handleNodeDragEnd = async () => {
// Clean up any existing state
if (this.isProcessing) {
console.log("isProcessing Returning")
return;
}
console.log("Node drag ended");
this.isProcessing = true;
await this.render();
};
public addNodes(node: MediaNode) {
if (node instanceof VideoNode) {
this.videoNodes.push(node);
node.kNode.on("dragend", this.handleNodeDragEnd);
} else if (node instanceof ImageNode || node instanceof TextNode) {
public async addNodes(node: MediaNode) {
if (node instanceof ImageNode || node instanceof TextNode || node instanceof ShapeNode) {
console.debug("Adding node:", node);
this.imageNodes.push(node);
console.log(this.imageNodes)
node.kNode.on("dragend", this.handleNodeDragEnd);
}
}
public removeNodes(node: MediaNode) {
@@ -317,6 +377,7 @@ export class RealTimeDrawEngine {
try {
const box = config.layerOfInterest.getClientRect();
const stage = config.layerOfInterest.getStage();
const x = config.x !== undefined ? config.x : Math.floor(box.x);
const y = config.y !== undefined ? config.y : Math.floor(box.y);
@@ -427,10 +488,37 @@ export class RealTimeDrawEngine {
});
}
public async render() {
// only pick nodes that intersect with the canvas on screen bounds to freeze.
this.mediaLayerRef.draw();
public async saveOutput() {
if (!this.outputBitmap) {
console.error("No output bitmap available to save");
return;
}
try {
const canvas = document.createElement("canvas");
canvas.width = this.outputBitmap.width;
canvas.height = this.outputBitmap.height;
const ctx = canvas.getContext("2d");
if (!ctx) throw new Error("Failed to get 2D context");
ctx.drawImage(this.outputBitmap, 0, 0);
const blob = await new Promise<Blob>((resolve) => {
canvas.toBlob((b) => resolve(b!), "image/png");
});
await FileUtilities.blobToFileJpeg(blob, "output");
} catch (error) {
console.error("Error saving output:", error);
}
}
public async render() {
// only pick nodes that intersect wi th the canvas on screen bounds to freeze.
this.mediaLayerRef.draw();
// Output all nodes in mediaLayerRef
const nodes = this.mediaLayerRef.getChildren();
console.log("All nodes in mediaLayer:", nodes);
console.log(
`context: x:${this.positionX} y:${this.positionY} ${this.width} x ${this.height}`,
);
@@ -447,17 +535,29 @@ export class RealTimeDrawEngine {
test: false,
})) as ImageBitmap;
// Test code
if (true) {
this.outputBitmap = bitmap;
this.previewCanvas.image(bitmap);
this.isProcessing = false;
return;
}
try {
const base64Bitmap = await this.imageBitmapToBase64(bitmap);
console.log("Sending to server");
const base64BitmapResponse = await invoke("infer_image", {
image: base64Bitmap,
prompt: this.currentPrompt,
strength: this.currentStrength,
});
console.log("RESPONSE IS");
console.log(base64BitmapResponse);
const decoded = await this.base64ToImageBitmap(
base64BitmapResponse as string,
);
this.outputBitmap = decoded;
this.previewCanvas.image(decoded);
} catch (error) {
console.error("Error during image processing:", error);
+3 -1
View File
@@ -2,6 +2,8 @@ export enum AppModes {
INIT = "INIT",
PREVIEW = "PREVIEW",
SELECT = "SELECT",
PAINT = "PAINT",
ERASER = "ERASER",
RENDERING = "RENDERING",
}
@@ -14,5 +16,5 @@ export const VideoResolutions = {
export const Colors = {
transparent: "rgba(0,0,0,0)",
translucentBlack: "rgba(0,0,0,0.3)",
primaryOrange: "rgba(230,100,98,1)",
primaryOrange: "rgba(45, 129, 255, 1)",
};
@@ -1,4 +1,4 @@
import { NodeType, BaseNode, ImageNode, TextNode, VideoNode } from "../Nodes";
import { NodeType, BaseNode, ImageNode, TextNode, VideoNode, ShapeNode } from "../Nodes";
import { TransformationData } from "./Transformation";
@@ -6,7 +6,7 @@ import { TextNodeData } from "./Text";
export type MediaNode =
// | NetworkedNode
BaseNode | VideoNode | ImageNode | TextNode;
BaseNode | VideoNode | ImageNode | TextNode | ShapeNode;
export type ImageNodeData = {
mediaFileUrl: string;
@@ -26,6 +26,10 @@ export type VideoNodeData = {
extractionUrl?: string;
};
export type ShapeNodeData = {
shape: "triangle" | "square" | "circle";
}
export type NodeData = {
type: NodeType;
transform: TransformationData;
@@ -50,6 +50,20 @@ export const ContextualToolbarNode = () => {
};
}, []);
const handleColorChange = (color: string) => {
// Don't do anything if no node is known to be bound
if (uiAccess.toolbarNode.signal.value.knodeIds.length < 1) {
return;
}
// Update the UI component
uiAccess.toolbarNode.setColor(color);
// Signal the engine with the node ID (just grab the first -and expected only- node)
const nodeID = uiAccess.toolbarNode.signal.value.knodeIds[0];
dispatchUiEvents.toolbarNode.setSelectedColor({ kNodeId: nodeID, color });
}
const boundedTopPosition = Math.min(position.y, winSize.height - 164);
return (
<Transition
@@ -71,6 +85,8 @@ export const ContextualToolbarNode = () => {
onLockClicked={(e) => {
dispatchUiEvents.toolbarNode.lock(e);
}}
color={uiAccess.toolbarNode.signal.value.color}
onColorChange={handleColorChange}
/>
</Transition>
);
@@ -23,6 +23,8 @@ import { useLayoutContext } from "./contextSignals/layout";
import { useRenderCounter } from "~/hooks/useRenderCounter";
import { useNavigate } from "react-router-dom";
import { SignaledMagicBox } from "./SignaledMagicBox";
import { SignaledPromptSlider } from "./SignaledPromptSlider";
import { SignaledPromptText } from "./SignaledPromptText";
export const KonvaRootComponent = ({
className,
@@ -54,18 +56,20 @@ export const KonvaRootComponent = ({
<KonvaCanvasContainer
ref={konvaContainerCallbackRef}
className={className}
// retreive the classNames from the parent for sizing/styling
// retreive the classNames from the parent for sizing/styling
/>
<SignaledMagicBox />
{/* <SignaledMagicBox /> */}
<SignaledCanvasDragDropFiles
openAddImage={appUiContext.openAddImage}
openAddVideo={appUiContext.openAddVideo}
/>
<SignaledPromptSlider />
<SignaledPromptText />
<SignaledToolbarMain
layoutSignal={layoutContext.signal}
appUiContext={appUiContext}
/>
<SignaledToolbarVideoExtraction />
{/* <SignaledToolbarVideoExtraction /> */}
<SignaledDialogs
appUiSignal={appUiContext.signal}
resetAll={appUiContext.resetAll}
@@ -73,7 +77,7 @@ export const KonvaRootComponent = ({
<ContextualLoadingBar />
<ContextualToolbarNode />
<ContextualButtonRetry />
{/* <ContextualButtonRetry /> */}
</>
);
};
@@ -0,0 +1,28 @@
import { useSignalEffect } from "@preact/signals-react";
import { Slider } from "~/components/ui";
import { dispatchUiEvents } from "~/signals";
import { promptStrength } from "~/signals/uiEvents/promptSettings";
export const SignaledPromptSlider = () => {
const sliderValue = promptStrength.value;
const onSliderValueChanged =
dispatchUiEvents.promptSettings.setPromptStrength;
// This makes component re-render whenever the signal value changes
useSignalEffect(() => {
promptStrength.value;
});
return (
<div className="fixed bottom-[138px] left-1/2 h-4 w-[400px] -translate-x-1/2 shadow-lg">
<Slider
min={0}
max={100}
value={sliderValue}
onChange={onSliderValueChanged}
step={1}
innerLabel={`Strength - ${sliderValue}%`}
/>
</div>
);
};
@@ -0,0 +1,39 @@
import { useSignalEffect } from "@preact/signals-react";
import { useRef } from "react";
import { ResizeType, Textarea } from "~/components/ui";
import { dispatchUiEvents } from "~/signals";
import { promptText } from "~/signals/uiEvents/promptSettings";
export const SignaledPromptText = ({}) => {
const textAreaRef = useRef<HTMLTextAreaElement>(null);
const TextAreaStates = {
placeholder: "Type your prompt here...",
label: "",
resize: "none" as ResizeType,
rows: 3,
};
const textContent = promptText.value;
const onTextChanged = (event: React.ChangeEvent<HTMLTextAreaElement>) => {
dispatchUiEvents.promptSettings.setPrompt(event.target.value);
};
// This makes component re-render whenever the signal value changes
useSignalEffect(() => {
promptText.value;
});
return (
<div className="fixed bottom-5 left-1/2 w-[400px] -translate-x-1/2 text-[16px] font-medium shadow-lg">
<Textarea
className="glass rounded-xl"
{...TextAreaStates}
ref={textAreaRef}
onChange={onTextChanged}
value={textContent}
spellCheck={false}
/>
</div>
);
};
@@ -12,6 +12,7 @@ import { LayoutSignalType } from "./contextSignals/layout";
import { AppUiContextInterface } from "./contextSignals/appUi";
import { twMerge } from "tailwind-merge";
import { paperWrapperStyles } from "~/components/styles";
import { dispatchUiEvents } from "~/signals";
export const SignaledToolbarMain = ({
// layoutSignal,
@@ -43,6 +44,19 @@ export const SignaledToolbarMain = ({
return () => appUiContext.openAddImage();
case ToolbarMainButtonNames.ADD_VIDEO:
return () => appUiContext.openAddVideo();
case ToolbarMainButtonNames.ADD_CIRCLE:
return () => dispatchUiEvents.addShapeToEngine({ shape: "circle" });
case ToolbarMainButtonNames.ADD_TRIANGLE:
return () => dispatchUiEvents.addShapeToEngine({ shape: "triangle" });
case ToolbarMainButtonNames.ADD_SQUARE:
return () => dispatchUiEvents.addShapeToEngine({ shape: "square" });
case ToolbarMainButtonNames.SELECT:
case ToolbarMainButtonNames.ERASER:
case ToolbarMainButtonNames.PAINT:
return (e: React.MouseEvent<HTMLButtonElement, MouseEvent>) => {
toolbarMain.setMode(buttonName);
dispatchers[buttonName](e);
};
default:
return dispatchers[buttonName];
}
@@ -69,7 +83,7 @@ export const SignaledToolbarMain = ({
dispatchers.loadingBarRetry(e);
};
return (
<div className="fixed bottom-4 left-1/2 -translate-x-1/2">
<div className="fixed left-4 top-1/2 -translate-y-1/2">
<Transition
as="div"
className={twMerge(
+28 -5
View File
@@ -4,10 +4,11 @@
@layer base {
:root {
@apply bg-ui-background font-sans text-[#222222];
@apply bg-ui-background font-sans text-white;
}
.pegboard {
background-image: radial-gradient(#3f3f466b 1px, transparent 0);
background-image: radial-gradient(#ffffff25 1px, transparent 0);
background-color: #1b1b1f;
background-size: 36px 36px;
@apply bg-center;
}
@@ -41,7 +42,24 @@
}
input {
@apply border-0 px-2 py-1 outline-none -outline-offset-2 focus:outline-primary-500 focus:ring-0;
@apply bg-ui-controls border-0 px-2 py-1 outline-none -outline-offset-2 focus:outline-primary-500 focus:ring-0;
}
textarea {
@apply bg-ui-controls placeholder:text-white/50;
/* Webkit scrollbar styles */
&::-webkit-scrollbar {
@apply w-1;
}
&::-webkit-scrollbar-track {
@apply bg-transparent;
}
&::-webkit-scrollbar-thumb {
@apply rounded-full bg-white/20 hover:bg-white/30;
}
}
}
@@ -54,10 +72,10 @@
.dotted-pattern {
background-image: radial-gradient(
circle at 1px 1px,
rgb(0 0 0 / 0.25) 1px,
rgb(255 255 255 / 0.25) 1px,
transparent 0
);
background-size: 32px 32px;
background-size: 40px 40px;
}
.gradient-text {
@@ -87,6 +105,11 @@
background-position: 100% 0;
animation: highlight-sweep 3s infinite linear;
}
.glass {
@apply relative border border-ui-border shadow-lg backdrop-blur-xl transition-colors duration-300;
@apply bg-ui-controls/50 hover:bg-ui-controls/90 !important;
}
}
@keyframes glow {
+140 -78
View File
@@ -1,107 +1,169 @@
import Navbar from "../../../components/landing/Navbar";
import Hero from "../../../components/landing/Hero";
import FeatureCard from "../../../components/landing/FeatureCard";
import Footer from "../../../components/landing/Footer";
import { Button } from "~/components/ui";
import { faApple } from "@fortawesome/free-brands-svg-icons";
import { faDesktop } from "@fortawesome/pro-solid-svg-icons";
import { useIsMobile } from "~/hooks/useIsMobile";
const Landing = () => {
const isMobile = useIsMobile();
const features = [
{
title: "Step 1: Compose Your Scene",
title: "Lorem ipsum dolor sit",
description:
"Begin your story on our intuitive canvas. Drop and arrange photos and videos to create layered scenes. Position, scale, and transform elements with precision to build your perfect composition. Our dynamic canvas makes it easy to bring your vision to life.",
image: "/images/feature_compositing.png",
gradient: "bg-gradient-to-br from-purple-500/20 to-blue-500/20",
"Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do eiusmod tempor incididunt.",
image: "/images/features/ai-creation.jpg",
tag: "Core",
},
{
title: "Step 2: Smart Segmentation",
title: "Ut enim ad minim",
description:
"Clean up your scenes with powerful AI segmentation. Remove green screens instantly, isolate subjects from any background, and perfect your layer masks with intelligent tools. Create professional-quality compositions with just a few clicks.",
image: "/images/feature_segmentation.png",
gradient: "bg-gradient-to-br from-emerald-500/20 to-cyan-500/20",
"Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea.",
image: "/images/features/collaboration.jpg",
tag: "Workflow",
},
{
title: "Step 3: AI Style Transfer",
title: "Duis aute irure dolor",
description:
"Transform your composition with AI style transfer magic. Choose from our curated collection of artistic styles. Apply consistent styling across your entire scene to create a cohesive visual masterpiece.",
image: "/images/feature_ai_stylize.png",
gradient: "bg-gradient-to-br from-rose-500/20 to-orange-500/20",
},
{
title: "Step 4: Render Your Video",
description:
"Render your styled composition into a stunning final movie. Export in high quality, ready to share with the world.",
image: "/images/feature_render.png",
gradient: "bg-gradient-to-br from-blue-500/20 to-indigo-500/20",
"Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla.",
image: "/images/features/style-models.jpg",
tag: "Tools",
},
];
const scrollToFeature = (index: number) => {
const element = document.getElementById(`feature-${index + 1}`);
if (element) {
element.scrollIntoView({ behavior: "smooth", block: "center" });
}
};
return (
<div className="min-h-screen bg-white">
<div className="relative min-h-screen bg-[#101014] text-white">
<Navbar />
<div>
<Hero onFeatureClick={scrollToFeature} />
<section className="relative overflow-hidden py-32">
{/* Decorative elements */}
<div className="dotted-pattern absolute inset-0 opacity-50" />
<div className="relative mx-auto max-w-7xl px-4 sm:px-6 lg:px-8">
{/* Section header with enhanced styling */}
<div className="mx-auto mb-24 max-w-3xl text-center">
<div className="mb-6 inline-flex items-center rounded-full bg-gradient-to-r from-purple-500/10 via-blue-500/10 to-purple-500/10 px-8 py-2.5">
<span className="whitespace-nowrap bg-gradient-to-r from-purple-600 via-blue-600 to-purple-600 bg-clip-text text-sm font-semibold text-transparent">
AI-POWERED WORKFLOW
<div className="dotted-pattern absolute inset-0 z-[0] opacity-50" />
{/* Hero Section */}
<div className="relative flex overflow-hidden pt-24 xl:min-h-[1000px] xl:items-center xl:pt-0">
{/* Content Container */}
<div className="relative z-10 mx-auto grid h-full w-full max-w-[1920px] grid-cols-12 items-center px-6 md:px-16 xl:px-32">
<div className="z-10 col-span-12 w-full xl:absolute xl:left-0 xl:top-1/2 xl:max-w-4xl xl:-translate-y-1/2 xl:pl-32 before:xl:absolute before:xl:-inset-0 before:xl:-z-10 before:xl:mr-40 before:xl:rounded-full before:xl:bg-black/90 before:xl:to-transparent before:xl:blur-[180px]">
<div className="max-w-2xl">
<div className="mb-6">
<span className="text-lg font-semibold uppercase tracking-widest text-gray-400">
MIRA
</span>
</div>
<h2 className="mb-6 text-4xl font-bold tracking-tight text-gray-900">
Create Stunning Videos in
<span className="relative">
<span className="gradient-text relative z-10">
{" "}
Four Simple Steps
</span>
<span className="bg-rose-200/50 absolute -bottom-2 left-0 z-0 h-3 w-full" />
</span>
</h2>
<p className="text-xl leading-relaxed text-gray-600">
From composition to final render, transform your media into
cinematic stories with our powerful AI-enhanced workflow.
<h1 className="mb-6 font-bold leading-tight lg:text-6xl">
Realtime AI editor.
<br />
On your local machine.
</h1>
<p className="mb-10 max-w-xl text-lg leading-relaxed">
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Sed do
eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut
enim ad minim veniam, quis nostrud exercitation.
</p>
</div>
{/* Features stack with enhanced visual separation */}
<div className="relative space-y-32">
{/* Vertical timeline line */}
<div className="to-rose-500/20 absolute left-1/2 top-0 h-full w-px bg-gradient-to-b from-purple-500/20 via-blue-500/20" />
{features.map((feature, index) => (
<div
key={index}
id={`feature-${index + 1}`}
className="relative"
<div className="flex flex-col gap-4 sm:flex-row">
<Button
className={`w-full transform rounded-lg px-8 py-4 font-semibold transition sm:w-auto ${
isMobile
? "cursor-not-allowed bg-[#2D81FF]"
: "bg-[#2D81FF] hover:scale-105 hover:bg-[#438AF6]"
}`}
disabled={isMobile}
icon={isMobile ? faDesktop : faApple}
>
{/* Step number bubble */}
<div className="absolute left-1/2 -translate-x-1/2 -translate-y-16">
<div className="flex h-12 w-12 items-center justify-center rounded-full border bg-white shadow-lg">
<span className="bg-gradient-to-br from-purple-600 to-purple-600 bg-clip-text text-xl font-bold text-transparent">
{index + 1}
</span>
</div>
</div>
<FeatureCard {...feature} isReversed={index % 2 === 1} />
</div>
))}
{isMobile ? "Download on desktop" : "Download for MacOS"}
</Button>
<button className="w-full transform rounded-lg border border-white/30 px-8 py-4 font-semibold transition hover:scale-105 hover:bg-white/10 sm:w-auto">
Learn more
</button>
</div>
</div>
</div>
</section>
<div className="col-span-12 mt-12 aspect-video w-full transform overflow-hidden rounded-xl border border-white/[16%] bg-transparent md:mt-16 xl:col-span-9 xl:col-start-4 xl:mt-0">
<img
src="/images/landing_hero.png"
alt="AI Video Creation"
className="h-full w-full object-cover object-top opacity-90"
/>
</div>
</div>
</div>
{/* Features Section */}
<div className="relative z-10 mx-auto mb-32 max-w-[1920px] px-4 py-20 sm:px-24 lg:px-32">
<div className="mb-8 flex items-center justify-between">
<h1 className="text-4xl font-bold">Features</h1>
{/* <button className="text-sm text-[#0096CE] hover:text-[#0084B5]">
View all
</button> */}
</div>
<div className="grid gap-6 md:grid-cols-3">
{features.map((item, index) => (
<div
key={index}
className="group cursor-pointer overflow-hidden rounded-xl bg-[#1C1C20] transition hover:bg-[#27272B]"
>
<div className="aspect-video overflow-hidden">
<img
src={item.image}
alt={item.title}
className="h-full w-full object-cover transition group-hover:scale-105"
/>
</div>
<div className="p-6">
{/* <div className="mb-4 text-sm text-[#0096CE]">{item.tag}</div> */}
<h3 className="mb-2 text-xl font-semibold">{item.title}</h3>
<p className="text-gray-400">{item.description}</p>
</div>
</div>
))}
</div>
{/* Installation Steps Section */}
<div className="mt-36">
<h1 className="mb-12 text-5xl font-bold">
How to install Mira AI Editor
</h1>
<div className="grid gap-6 md:grid-cols-3">
<div className="rounded-xl bg-[#1C1C20] p-8">
<div className="mb-4 w-fit rounded-full bg-white/15 px-3 py-1 text-sm font-medium text-white">
STEP 1
</div>
<h3 className="mb-4 text-2xl font-semibold">Download the app</h3>
<p className="mb-6 text-gray-400">
Download and install our application to start creating amazing
AI-powered content.
</p>
<Button className="rounded-lg bg-[#2D81FF] px-4 py-2 text-sm font-medium hover:bg-[#438AF6]">
Download App
</Button>
</div>
<div className="rounded-xl bg-[#1C1C20] p-8">
<div className="mb-4 w-fit rounded-full bg-white/15 px-3 py-1 text-sm font-medium text-white">
STEP 2
</div>
<h3 className="mb-4 text-2xl font-semibold">Create an account</h3>
<p className="mb-6 text-gray-400">
Sign up for an account to access all features and start your
journey.
</p>
<Button className="text-sm font-medium" variant="secondary">
Sign up
</Button>
</div>
<div className="rounded-xl bg-[#1C1C20] p-8">
<div className="mb-4 w-fit rounded-full bg-white/15 px-3 py-1 text-sm font-medium text-white">
STEP 3
</div>
<h3 className="mb-4 text-2xl font-semibold">Start creating</h3>
<p className="mb-6 text-gray-400">
Begin creating with our realtime AI editor!
</p>
</div>
</div>
</div>
</div>
<Footer />
@@ -8,6 +8,7 @@ import { ToolbarUserProfile } from "~/components/features";
//Components of the Konva App are all in the KonvaComponent
import { KonvaRootComponent } from "~/KonvaRootComponent";
import { ToolbarTopLeft } from "~/components/features/ToolbarTopLeft";
import { ToolbarTopRight } from "~/components/features/ToolbarTopRight";
export const Main = withProtectionRoute(() => {
// This is a hook that will log the number of times the component has rerendered
@@ -25,6 +26,7 @@ export const Main = withProtectionRoute(() => {
/>
<div className="absolute top-0 flex w-full justify-between px-2">
<ToolbarTopLeft />
<ToolbarTopRight />
{/* <ToolbarUserProfile /> */}
</div>
</div>
@@ -42,12 +42,6 @@ export const DialogAddImage = ({
closeCallback();
}
function handleEnter() {
if (currFile) {
onAddImage(currFile);
}
handleClose();
}
return (
<Dialog open={isOpen} onClose={closeCallback} className="relative z-50">
<div className={dialogBackgroundStyles}>
@@ -58,7 +52,7 @@ export const DialogAddImage = ({
"w-full max-w-2xl",
)}
>
<DialogTitle className="text-3xl font-bold">Upload Image</DialogTitle>
<DialogTitle className="text-2xl font-bold">Upload Image</DialogTitle>
<div className="flex flex-col rounded-lg border-2 border-dashed border-ui-border">
<FileUploader
title=""
@@ -66,6 +60,10 @@ export const DialogAddImage = ({
file={currFile}
setFile={(file: File | null) => {
setAssetFile(file);
if (file) {
onAddImage(file);
handleClose();
}
}}
/>
{currFile && (
@@ -84,9 +82,6 @@ export const DialogAddImage = ({
<Button onClick={handleClose} variant="secondary">
Cancel
</Button>
<Button onClick={handleEnter} disabled={currFile === null}>
Enter
</Button>
</div>
</DialogPanel>
</div>
@@ -73,7 +73,7 @@ export const DialogEditText = ({
<Dialog open={isOpen} onClose={closeCallback} className="relative z-50">
<div className={dialogBackgroundStyles}>
<DialogPanel className={twMerge(paperWrapperStyles, dialogPanelStyles)}>
<DialogTitle className="text-3xl font-bold">Edit Text</DialogTitle>
<DialogTitle className="text-2xl font-bold">Edit Text</DialogTitle>
<TextEditor
text={text}
formatData={textFormatData}
@@ -24,7 +24,7 @@ export const DragAndDropZone = ({ file, fileTypes }: Props) => {
// const fileName = file && getFileName(file).toUpperCase();
const wrapperClassName = twMerge(
"group cursor-pointer px-6 py-12 bg-gray-100 hover:bg-gray-200/60 transition-colors duration-150 ease-in-out",
"group cursor-pointer px-6 py-12 bg-ui-controls hover:bg-white/25 transition-colors duration-150 ease-in-out rounded-lg",
!file && "flex flex-col items-center justify-center gap-",
file && "flex items-center gap-2 py-6",
// "rounded-lg border-2 border-dashed border-ui-border",
@@ -44,12 +44,12 @@ export const ToolbarButton = ({
}
const mergedButtonClasses = twMerge(
"rounded-lg py-2 px-3 hover:bg-gray-200/50 transition-all duration-100",
children ? "w-fit flex items-center gap-2 text-nowrap" : "size-10",
active && "pointer-events-none text-primary ",
"rounded-lg hover:bg-white/15 transition-all duration-100 border-2 border-transparent text-white/80 px-2",
children ? "w-fit flex items-center gap-2.5 text-nowrap" : "size-10",
active && "bg-primary/30 border-2 border-primary hover:bg-primary/30",
disabled && "pointer-events-none text-secondary-300",
buttonProps.prominent &&
"highlight-button border border-primary-400/30 text-primary-500 hover:bg-primary-100/40 hover:border-primary-400/60",
"border border-primary-400/30 text-primary-500 hover:bg-primary-100/40 hover:border-primary-400/60",
customButtonClassNames,
);
@@ -57,6 +57,7 @@ export const ToolbarButton = ({
<button
className={mergedButtonClasses}
disabled={disabled}
autoFocus={false}
{...restButtonProps}
onClick={(e) => {
e.preventDefault();
@@ -73,7 +74,11 @@ export const ToolbarButton = ({
</button>
);
if (tooltip) {
return <Tooltip tip={tooltip}>{Button}</Tooltip>;
return (
<Tooltip position="right" tip={tooltip}>
{Button}
</Tooltip>
);
}
return Button;
};
@@ -11,8 +11,6 @@ export const ButtonPreviewAndRender = ({
}) => {
return (
<div className="flex">
<ButtonPreview {...buttonPreviewProps} />
<ButtonRender {...buttonRenderProps} />
</div>
);
};
@@ -1,25 +1,30 @@
import { Fragment, MouseEventHandler } from "react";
import { twMerge } from "tailwind-merge";
import { Popover, PopoverButton, PopoverPanel } from "@headlessui/react";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import {
faArrowRotateLeft,
faArrowRotateRight,
faCircle,
faEraser,
faFilePlus,
faFilm,
faFloppyDisk,
faImage,
faLocationArrow,
faMagicWandSparkles,
faPaintbrush,
faShapes,
faSquare,
faText,
faTriangle,
} from "@fortawesome/pro-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { Popover, PopoverButton, PopoverPanel } from "@headlessui/react";
import { Fragment, MouseEventHandler } from "react";
import { twMerge } from "tailwind-merge";
import { ToolbarButton } from "../ToolbarButton";
// style and constants
import { paperWrapperStyles, toolTipStyles } from "~/components/styles";
import { ToolbarMainButtonNames } from "./enum";
import { ButtonPreviewAndRender } from "./ButtonPreviewAndRender";
import { dispatchUiEvents, uiEvents } from "~/signals/uiEvents";
import { paintColor } from "~/signals/uiEvents/toolbarMain/paintMode";
import { ColorPicker } from "~/components/ui/TextEditor/ColorPicker";
export const ToolbarMain = ({
disabled = false,
@@ -38,13 +43,14 @@ export const ToolbarMain = ({
<>
<div
className={twMerge(
"flex w-fit items-center divide-x divide-ui-border",
"flex h-fit flex-col items-center gap-y-2",
paperWrapperStyles,
"glass",
disabled &&
"pointer-events-none cursor-default bg-ui-border shadow-md",
)}
>
<div className="flex items-center gap-2 pr-2">
<div className="flex flex-col items-center gap-2">
<ToolbarButton
icon={faLocationArrow}
iconProps={{ className: "fa-flip-horizontal" }}
@@ -56,44 +62,137 @@ export const ToolbarMain = ({
buttonProps={buttonProps.ADD_TEXT}
tooltip="Add Text"
/>
<Popover className="relative">
{({ close }) => (
<>
<PopoverButton as={Fragment}>
<button
data-tooltip="Add Shapes"
className={twMerge(
"size-10 rounded-lg border-2 border-transparent text-white/80 transition-all duration-100 hover:bg-white/15",
toolTipStyles.base,
toolTipStyles.right,
)}
>
<FontAwesomeIcon icon={faShapes} />
</button>
</PopoverButton>
<PopoverPanel
anchor="right"
className={twMerge(
"flex flex-row [--anchor-gap:12px]",
paperWrapperStyles,
)}
>
<ToolbarButton
icon={faSquare}
buttonProps={{
...buttonProps.ADD_SQUARE,
onClick: (e) => {
buttonProps.ADD_SQUARE.onClick?.(e);
close();
},
}}
/>
<ToolbarButton
icon={faCircle}
buttonProps={{
...buttonProps.ADD_CIRCLE,
onClick: (e) => {
buttonProps.ADD_CIRCLE.onClick?.(e);
close();
},
}}
/>
<ToolbarButton
icon={faTriangle}
buttonProps={{
...buttonProps.ADD_TRIANGLE,
onClick: (e) => {
buttonProps.ADD_TRIANGLE.onClick?.(e);
close();
},
}}
/>
</PopoverPanel>
</>
)}
</Popover>
<Popover className="relative">
<PopoverButton as={Fragment}>
<button
data-tooltip="Add..."
data-tooltip="Add Media"
className={twMerge(
"size-10 rounded-lg p-2 hover:bg-gray-200/50",
toolTipStyles,
"size-10 rounded-lg border-2 border-transparent text-white/80 transition-all duration-100 hover:bg-white/15",
toolTipStyles.base,
toolTipStyles.right,
)}
>
<FontAwesomeIcon icon={faFilePlus} />
</button>
</PopoverButton>
<PopoverPanel
anchor="bottom"
anchor="right"
className={twMerge(
// "absolute bottom-full left-1/2 z-10 mb-2 -translate-x-1/2",
"flex flex-col [--anchor-gap:16px]",
"flex [--anchor-gap:12px]",
paperWrapperStyles,
)}
>
<ToolbarButton icon={faImage} buttonProps={buttonProps.ADD_IMAGE}>
Add Image
<span className="text-[16px]">Add Image</span>
</ToolbarButton>
<ToolbarButton icon={faFilm} buttonProps={buttonProps.ADD_VIDEO}>
{/* <ToolbarButton icon={faFilm} buttonProps={buttonProps.ADD_VIDEO}>
Add Video
</ToolbarButton>
</ToolbarButton> */}
</PopoverPanel>
</Popover>
<ToolbarButton
icon={faMagicWandSparkles}
buttonProps={{
...buttonProps.AI_STYLIZE,
prominent: true,
}}
tooltip="AI Stylize"
/>
</div>
<div className="flex items-center gap-2 pl-2">
<hr className="w-full border-t border-white/15" />
<div className="flex flex-col items-center gap-2">
<div className="flex flex-col items-center gap-2">
{buttonProps.PAINT.active ? (
<div className="relative">
<ColorPicker
color={paintColor.value}
onChange={dispatchUiEvents.toolbarMain.setPaintColor}
faIcon={faPaintbrush}
borderStyle="border-2 bg-primary/30 border-2 border-primary hover:bg-primary/30 text-white"
showBar={false}
staticIconColor="white"
streamChanges
defaultOpen={true}
anchor="right"
anchorGap={12}
closeOnMouseLeave={true}
/>
<div
className="pointer-events-none absolute -bottom-1.5 -right-1.5 h-[18px] w-[18px] rounded-full border border-gray-400"
style={{ backgroundColor: paintColor.value }}
/>
</div>
) : (
<ToolbarButton
icon={faPaintbrush}
buttonProps={buttonProps.PAINT}
tooltip="Paint Brush"
/>
)}
<ToolbarButton
icon={faEraser}
buttonProps={buttonProps.ERASER}
tooltip="Eraser"
/>
</div>
</div>
<hr className="w-full border-t border-white/15" />
<div className="flex flex-col items-center gap-2">
<ToolbarButton
icon={faArrowRotateLeft}
buttonProps={buttonProps.UNDO}
@@ -104,15 +203,11 @@ export const ToolbarMain = ({
buttonProps={buttonProps.REDO}
tooltip="Redo"
/>
<ToolbarButton
{/* <ToolbarButton
icon={faFloppyDisk}
buttonProps={buttonProps.SAVE}
tooltip="Save"
/>
<ButtonPreviewAndRender
buttonPreviewProps={buttonProps.PREVIEW}
buttonRenderProps={buttonProps.DOWNLOAD}
/>
/> */}
</div>
</div>
</>
@@ -4,6 +4,11 @@ export enum ToolbarMainButtonNames {
ADD_TEXT = "ADD_TEXT",
ADD_IMAGE = "ADD_IMAGE",
ADD_VIDEO = "ADD_VIDEO",
ADD_CIRCLE = "ADD_CIRCLE",
ADD_TRIANGLE = "ADD_TRIANGLE",
ADD_SQUARE = "ADD_SQUARE",
PAINT = "PAINT",
ERASER = "ERASER",
AI_STYLIZE = "AI_STYLIZE",
UNDO = "UNDO",
REDO = "REDO",
@@ -4,14 +4,19 @@ import {
faLockKeyholeOpen,
faLockKeyhole,
faUnlockKeyhole,
faPalette,
} from "@fortawesome/pro-solid-svg-icons";
import {
ToolbarButton,
ToolbarButtonProps,
} from "~/components/features/ToolbarButton";
import { paperWrapperStyles } from "~/components/styles";
import { paperWrapperStyles, toolTipStyles } from "~/components/styles";
import { ToolbarNodeButtonNames } from "./enums";
import { ToolbarNodeButtonData } from "./data";
import { Popover, PopoverButton, PopoverPanel } from "@headlessui/react";
import { Fragment } from "react/jsx-runtime";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { ColorPicker } from "~/components/ui/TextEditor/ColorPicker";
export interface ToolbarNodeProps {
disabled?: boolean;
@@ -24,6 +29,8 @@ export interface ToolbarNodeProps {
buttonsProps?: {
[key in ToolbarNodeButtonNames]: ToolbarButtonProps;
};
color?: string;
onColorChange?: (color: string) => void;
}
export const ToolbarNode = ({
disabled,
@@ -31,6 +38,8 @@ export const ToolbarNode = ({
lockDisabled,
onLockClicked,
buttonsProps,
color = "#000000",
onColorChange = () => { },
}: ToolbarNodeProps) => {
const handleOnLockClicked: React.MouseEventHandler<HTMLButtonElement> = (
e,
@@ -45,7 +54,7 @@ export const ToolbarNode = ({
className={twMerge(
paperWrapperStyles,
disabled && "pointer-events-none cursor-default bg-ui-border shadow-md",
"flex gap-2 transition",
"flex flex-col gap-2 transition",
)}
>
<ToolbarButton
@@ -53,7 +62,7 @@ export const ToolbarNode = ({
className: twMerge(
locked && "text-primary hover:bg-primary hover:text-white",
lockDisabledOrUnknown &&
"text-secondary-300 hover:text-secondary-300",
"text-secondary-300 hover:text-secondary-300",
),
disabled: lockDisabledOrUnknown,
}}
@@ -73,14 +82,24 @@ export const ToolbarNode = ({
{ToolbarNodeButtonData.map((buttonDatum, idx) => {
const buttonProps = buttonsProps?.[buttonDatum.name];
return (
<ToolbarButton
icon={buttonDatum.icon}
tooltip={buttonDatum.tooltip}
key={idx}
buttonProps={buttonProps}
/>
);
if (!buttonProps || buttonProps.hidden) {
return;
}
if (buttonDatum.name !== ToolbarNodeButtonNames.COLOR) {
return (
<ToolbarButton
icon={buttonDatum.icon}
tooltip={buttonDatum.tooltip}
key={idx}
buttonProps={buttonProps}
/>
);
} else {
return (
<ColorPicker color={color} onChange={onColorChange} faIcon={faPalette} borderStyle="" key={idx} />
)
}
})}
</div>
);
@@ -8,6 +8,7 @@ import {
faTrashCan,
faVectorSquare,
faDownload,
faPalette,
} from "@fortawesome/pro-solid-svg-icons";
export const ToolbarNodeButtonData = [
@@ -16,6 +17,11 @@ export const ToolbarNodeButtonData = [
icon: faVectorSquare,
tooltip: "Move",
},
{
name: ButtonNames.COLOR,
icon: faPalette,
tooltip: "Color",
},
{
name: ButtonNames.CHROMA,
icon: faTransporter,
@@ -7,4 +7,5 @@ export enum ToolbarNodeButtonNames {
MOVE_LAYER_UP = "MOVE_LAYER_UP",
MOVE_LAYER_DOWN = "MOVE_LAYER_DOWN",
DOWNLOAD = "DOWNLOAD",
COLOR = "COLOR",
}
@@ -1,28 +1,27 @@
import { twMerge } from "tailwind-merge";
import { paperWrapperStyles } from "~/components/styles";
import { faPlus, faQuestion } from "@fortawesome/pro-solid-svg-icons";
import { ToolbarButton } from "~/components/features/ToolbarButton";
// import { paperWrapperStyles } from "~/components/styles";
// import { faPlus, faQuestion } from "@fortawesome/pro-solid-svg-icons";
// import { ToolbarButton } from "~/components/features/ToolbarButton";
export const ToolbarTopLeft = () => {
return (
<div
className={twMerge(
paperWrapperStyles,
"z-20 mt-2 flex h-fit w-fit items-center gap-1 py-2.5 pl-4",
)}
>
<img
src="/brand/Storyteller-Logo-Black.png"
src="/brand/mira-logo.png"
alt="logo"
className="mr-4 h-8"
className="h-[30px] select-none pb-1"
/>
<ToolbarButton icon={faPlus}>
{/* <ToolbarButton icon={faPlus}>
<span className="font-semibold">New Board</span>
</ToolbarButton>
<ToolbarButton icon={faQuestion}>
<span className="font-semibold">Help</span>
</ToolbarButton>
</ToolbarButton> */}
</div>
);
};
@@ -0,0 +1,31 @@
import { faSave } from "@fortawesome/pro-solid-svg-icons";
import { twMerge } from "tailwind-merge";
import { Button } from "~/components/ui/Button";
import { toolbarMain } from "~/signals/uiAccess/toolbarMain";
import { dispatchers } from "~/signals/uiEvents/toolbarMain";
import { ToolbarMainButtonNames } from "../ToolbarMain/enum";
// import { paperWrapperStyles } from "~/components/styles";
// import { faPlus, faQuestion } from "@fortawesome/pro-solid-svg-icons";
// import { ToolbarButton } from "~/components/features/ToolbarButton";
export const ToolbarTopRight = () => {
const saveButtonState =
toolbarMain.signal.value.buttonStates[ToolbarMainButtonNames.SAVE];
return (
<div
className={twMerge(
"z-20 mt-2 flex h-fit w-fit items-center gap-1 p-4 py-2.5",
)}
>
<Button
icon={faSave}
onClick={dispatchers[ToolbarMainButtonNames.SAVE]}
disabled={saveButtonState?.disabled}
>
Save Image
</Button>
</div>
);
};
@@ -9,18 +9,18 @@ const Footer = () => {
const navigate = useNavigate();
return (
<footer className="border-t border-gray-100 bg-gray-50">
<footer className="relative z-10 bg-[#18181C]">
<div className="mx-auto max-w-7xl px-4 py-16 sm:px-6 lg:px-8">
<div className="flex flex-col items-center space-y-10 text-center">
<img
src="/brand/Storyteller-Logo-Black.png"
alt="Storyteller Board"
className="h-9"
src="/brand/mira-logo.png"
alt="Mira Realtime AI Editor"
className="h-[34px] pb-1"
/>
<Button
onClick={() => navigate("/signup")}
className="group rounded-xl px-8 py-4 text-lg"
className="text-md group rounded-xl bg-[#2D81FF] px-8 py-4 hover:bg-[#438AF6]"
>
<span className="relative flex items-center">
Start Creating for Free
@@ -3,26 +3,31 @@ import { Button } from "../ui";
const Navbar = () => {
const navigate = useNavigate();
return (
<nav className="fixed top-0 z-50 w-full border-b border-gray-100 bg-white/80 backdrop-blur-md">
<div className="mx-auto max-w-7xl px-4 sm:px-6 lg:px-8">
<div className="flex h-20 items-center justify-between">
<nav className="fixed top-0 z-50 w-full bg-[rgba(16,16,20,0.7)] backdrop-blur-3xl">
<div className="mx-auto max-w-[1920px] px-4 sm:px-24 lg:px-32">
<div className="flex h-[72px] items-center justify-between">
<a href="/" className="flex items-center">
<img
src="/brand/Storyteller-Logo-Black.png"
alt="Storyteller Board"
className="h-9"
src="/brand/mira-logo.png"
alt="Mira Realtime AI Editor"
className="h-[34px] pb-1"
/>
</a>
<div className="flex items-center space-x-5">
<a
href="/login"
className="font-medium !text-gray-600 hover:!text-gray-900"
<div className="flex items-center space-x-3">
<Button
variant="secondary"
className=""
onClick={() => navigate("/signup")}
>
Login
</a>
<Button onClick={() => navigate("/signup")}>Sign Up</Button>
Sign up
</Button>
<Button
className="hidden bg-[#2D81FF] hover:bg-[#438AF6] md:block"
onClick={() => []}
>
Download
</Button>
</div>
</div>
</div>
+23 -14
View File
@@ -1,26 +1,35 @@
import { twMerge } from "tailwind-merge";
export const paperWrapperStyles =
"rounded-xl border border-ui-border bg-ui-panel p-2 shadow-lg";
"rounded-xl border border-ui-border p-1.5 shadow-lg bg-ui-panel";
export const transitionTimingStyles = "transition duration-150 ease-in-out";
export const dialogBackgroundStyles =
"fixed inset-0 flex w-screen items-center justify-center bg-black/50";
export const toolTipStyles = twMerge(
"relative",
"before:w-0 before:h-0 before:absolute",
"before:left-1/2 before:bottom-full before:mb-[1px] before:-translate-x-1/2 before:z-50",
"before:border-l-8 before:border-l-transparent",
"before:border-r-8 before:border-r-transparent",
"before:border-t-8 before:border-t-white",
"after:absolute after:left-1/2 after:bottom-full after:-translate-x-1/2 after:z-40",
"after:content-[attr(data-tooltip)] after:text-black after:text-nowrap",
"after:rounded-lg after:border after:border-ui-border after:bg-ui-panel after:px-2.5 after:py-1.5 after:mb-2 after:shadow-xl",
"after:hidden before:hidden hover:after:block hover:before:block",
);
export const toolTipStyles = {
base: twMerge(
"relative",
"after:absolute after:z-40",
"after:content-[attr(data-tooltip)] after:text-black after:text-nowrap after:text-sm after:font-medium after:text-white/80 after:glass",
"after:rounded-lg after:border after:border-ui-border after:bg-ui-panel after:px-2.5 after:py-1 after:shadow-xl",
"after:hidden hover:after:block",
),
top: twMerge(
"after:left-1/2 after:bottom-full after:-translate-x-1/2 after:mb-3.5",
),
bottom: twMerge(
"after:left-1/2 after:top-full after:-translate-x-1/2 after:mt-3.5",
),
left: twMerge(
"after:top-1/2 after:right-full after:-translate-y-1/2 after:mr-3.5",
),
right: twMerge(
"after:top-1/2 after:left-full after:-translate-y-1/2 after:ml-3.5",
),
};
export const dialogPanelStyles = twMerge(
"flex max-w-5xl flex-col justify-between gap-4 p-6",
"flex max-w-5xl flex-col justify-between gap-4 p-6 bg-[#1C1C20]",
);
@@ -31,14 +31,14 @@ export const Button = ({
return "bg-tertiary hover:bg-tertiary-400 text-white focus-visible:outline-tertiary-600";
}
case "secondary": {
return "bg-gray-600 hover:bg-gray-500 text-white focus-visible:outline-secondary";
return "bg-ui-controls hover:bg-ui-controls/80 text-white focus-visible:outline-secondary";
}
case "action": {
return " bg-action hover:bg-action-500 text-white focus-visible:outline-action";
}
case "primary":
default: {
return "bg-primary hover:bg-primary-400 text-white focus-visible:outline-primary-600";
return "bg-primary hover:bg-primary-600 text-white focus-visible:outline-primary-600";
}
}
}
@@ -29,25 +29,25 @@ export const Combobox = ({ options, value, onChange }: ComboBoxInterface) => {
onChange={onChange}
onClose={() => setQuery("")}
>
<div className="relative flex h-10 rounded-md border">
<div className="relative flex h-10 rounded-md border border-ui-border">
<ComboboxInput
className="rounded-md"
className="rounded-lg px-3"
displayValue={(font: string) => font}
onChange={(event) => setQuery(event.target.value)}
/>
<ComboboxButton className="absolute right-0 top-0 flex h-10 items-center justify-center hover:text-primary">
<FontAwesomeIcon icon={faChevronDown} className="size-4 p-2" />
<ComboboxButton className="absolute right-0 top-0 flex h-10 items-center justify-center px-3 transition-colors duration-300 hover:text-primary">
<FontAwesomeIcon icon={faChevronDown} className="size-4" />
</ComboboxButton>
</div>
<ComboboxOptions
anchor="bottom"
className="mt-1 w-[var(--input-width)] rounded-lg border empty:invisible"
className="mt-1 w-[var(--input-width)] rounded-lg border border-ui-border empty:invisible"
>
{filteredOptions.map((option, idx) => (
<ComboboxOption
key={idx}
value={option}
className="cursor-pointer bg-white p-2 data-[focus]:bg-primary-200 data-[selected]:font-medium"
className="bg-ui-controls cursor-pointer px-3 py-2 data-[focus]:bg-gray-600 data-[selected]:font-medium"
>
{option}
</ComboboxOption>
+29 -12
View File
@@ -12,6 +12,7 @@ interface SliderProps {
showTooltip?: boolean;
tooltipContent?: ReactNode;
suffix?: string;
innerLabel?: string;
}
export const Slider = ({
@@ -24,6 +25,7 @@ export const Slider = ({
showTooltip,
tooltipContent,
suffix,
innerLabel,
}: SliderProps) => {
const [localValue, setLocalValue] = useState(value);
const sliderRef = useRef<HTMLDivElement>(null);
@@ -88,25 +90,40 @@ export const Slider = ({
<div
ref={sliderRef}
className={twMerge(
"relative h-2.5 w-full cursor-pointer rounded-full bg-gray-500",
"glass group relative h-7 w-full cursor-pointer overflow-hidden rounded-lg border border-ui-border",
isDragging && "!bg-ui-controls/90",
className,
)}
onMouseDown={handleMouseDown}
>
<div
className="absolute h-2.5 rounded-full bg-primary"
style={{ width: `${percentage}%` }}
></div>
<div
className={twMerge(
"absolute -top-1 h-[18px] w-[18px] -translate-x-1/2 transform rounded-full bg-gray-200 transition-colors",
isDragging ? "bg-white" : "bg-gray-300 hover:bg-white",
"absolute h-7 bg-primary/30 transition-colors duration-300 group-hover:bg-primary/50",
isDragging && "!bg-primary/50",
)}
style={{ left: `${percentage}%` }}
onMouseDown={handleMouseDown}
onMouseEnter={handleMouseEnter}
onMouseLeave={handleMouseLeave}
/>
style={{ width: `${percentage}%` }}
>
{innerLabel && (
<span
className={twMerge(
"absolute top-1/2 ml-2.5 -translate-y-1/2 text-nowrap text-sm font-medium text-white/60 transition-colors duration-300 group-hover:text-white",
isDragging && "!text-white",
)}
>
{innerLabel}
</span>
)}
<div
className={twMerge(
"absolute right-0 top-1/2 mr-1.5 h-3.5 w-0.5 -translate-y-1/2",
isDragging ? "bg-white" : "bg-white/50",
)}
onMouseDown={handleMouseDown}
onMouseEnter={handleMouseEnter}
onMouseLeave={handleMouseLeave}
/>
</div>
{showTooltip && (
<Transition
as={"div"}
@@ -7,10 +7,16 @@ export type ResizeType =
| "both"
| "horizontal"
| "vertical"
| "block"
| "inline"
| undefined;
const resizeStyles = {
none: "resize-none",
both: "resize",
horizontal: "resize-x",
vertical: "resize-y",
undefined: "",
};
export interface TextareaInterface
extends TextareaHTMLAttributes<HTMLTextAreaElement> {
label?: string;
@@ -21,6 +27,8 @@ export const Textarea = forwardRef<HTMLTextAreaElement, TextareaInterface>(
{ className, label, resize = "vertical", id, ...rest }: TextareaInterface,
ref,
) => {
const resizeStyle = resizeStyles[resize];
return (
<div className="flex flex-col">
{label && <label htmlFor={id ? id : kebabCase(label)}>{label}</label>}
@@ -29,17 +37,16 @@ export const Textarea = forwardRef<HTMLTextAreaElement, TextareaInterface>(
ref={ref}
id={id ? id : label ? kebabCase(label) : undefined}
className={twMerge(
"border-ui-panel-border bg-ui-controls rounded-lg border px-3 py-2",
"rounded-lg border border-ui-border px-4 py-2.5 text-white outline-none focus:outline-none",
className,
resizeStyle,
)}
style={{
outline: "2px solid transparent",
transition: "outline-color 0.15s ease-in-out",
resize: resize,
}}
onFocus={(e) => {
// disableHotkeyInput(DomLevels.INPUT);
e.currentTarget.style.outlineColor = "#e66462";
e.currentTarget.style.outlineColor = "#2d81ff";
}}
onBlur={(e) => {
// enableHotkeyInput(DomLevels.INPUT);
@@ -1,6 +1,6 @@
import {
faAlignCenter,
faAlignJustify,
// faAlignJustify,
faAlignLeft,
faAlignRight,
} from "@fortawesome/pro-solid-svg-icons";
@@ -1,6 +1,10 @@
import { ChangeEvent, useState } from "react";
import { ChangeEvent, useState, useEffect } from "react";
import { twMerge } from "tailwind-merge";
import { faFont, faHashtag } from "@fortawesome/pro-solid-svg-icons";
import {
faFont,
faHashtag,
IconDefinition,
} from "@fortawesome/pro-solid-svg-icons";
import { FontAwesomeIcon } from "@fortawesome/react-fontawesome";
import { Popover, PopoverButton, PopoverPanel } from "@headlessui/react";
import { HexColorPicker } from "react-colorful";
@@ -12,10 +16,31 @@ import { paperWrapperStyles } from "~/components/styles";
export const ColorPicker = ({
color: prevColor,
onChange,
faIcon = faFont,
borderStyle,
showBar = true,
anchor = "bottom",
anchorGap = 0,
fillBg = false,
staticIconColor,
streamChanges = false,
defaultOpen = false,
closeOnMouseLeave = false,
}: {
color: string;
onChange: (newColor: string) => void;
faIcon?: IconDefinition;
borderStyle?: string;
showBar?: boolean;
anchor?: "top" | "right" | "bottom" | "left";
anchorGap?: number;
fillBg?: boolean;
staticIconColor?: string;
streamChanges?: boolean;
defaultOpen?: boolean;
closeOnMouseLeave?: boolean;
}) => {
const [isOpen, setIsOpen] = useState(defaultOpen);
const [{ currColor, textInput }, setStates] = useState<{
currColor: string;
textInput: string;
@@ -24,13 +49,18 @@ export const ColorPicker = ({
textInput: prevColor.substring(1),
});
const handleSelectColor = (newColor: string) => {
setStates({ currColor: newColor, textInput: newColor.substring(1) });
};
const isHexColorCode = (color: string) => {
const hexColorRegex = /^(?:[0-9a-fA-F]{3}){1,2}$/;
return hexColorRegex.test(color);
useEffect(() => {
setIsOpen(defaultOpen);
}, [defaultOpen]);
const handleMouseLeave = (close: () => void) => {
if (closeOnMouseLeave) {
onChange(currColor);
setIsOpen(false);
close();
}
};
const handleTextInput = (e: ChangeEvent<HTMLInputElement>) => {
if (isHexColorCode(e.target.value)) {
setStates({
@@ -44,55 +74,101 @@ export const ColorPicker = ({
}));
}
};
const isHexColorCode = (color: string) => {
const hexColorRegex = /^(?:[0-9a-fA-F]{3}){1,2}$/;
return hexColorRegex.test(color);
};
return (
<Popover className="relative">
<PopoverButton className="flex size-10 flex-col items-center gap-1 rounded-md border p-2">
<FontAwesomeIcon icon={faFont} />
<span className="h-1 w-full" style={{ backgroundColor: prevColor }} />
</PopoverButton>
<PopoverPanel
anchor="bottom"
className={twMerge(
paperWrapperStyles,
"flex flex-col items-center gap-2",
)}
>
{({ close }) => {
return (
<>
<HexColorPicker color={currColor} onChange={handleSelectColor} />
<Input
style={{ width: "198px" }}
icon={faHashtag}
value={textInput}
onChange={handleTextInput}
{({ open, close }) => (
<>
<PopoverButton
className={twMerge(
"flex size-10 items-center gap-1 rounded-lg bg-ui-controls p-2",
borderStyle,
showBar ? "flex-col" : "justify-center",
)}
style={fillBg ? { backgroundColor: prevColor } : {}}
onMouseEnter={() => defaultOpen && setIsOpen(true)}
>
<FontAwesomeIcon
icon={faIcon}
color={staticIconColor ?? (fillBg ? undefined : prevColor)}
/>
{showBar && (
<span
className="h-1 w-full"
style={{ backgroundColor: prevColor }}
/>
<div className="flex w-full justify-center gap-2">
<Button
variant="secondary"
onClick={() => {
)}
</PopoverButton>
{(open || isOpen) && (
<div onMouseLeave={() => handleMouseLeave(close)}>
<PopoverPanel
anchor={anchor}
className={twMerge(
paperWrapperStyles,
"flex flex-col items-center gap-2 overflow-hidden",
)}
style={
{
"--anchor-gap": `${anchorGap}px`,
} as React.CSSProperties
}
static
>
<HexColorPicker
className="overflow-hidden"
color={currColor}
onChange={(color) => {
setStates({
currColor: prevColor,
textInput: prevColor.substring(1),
currColor: color,
textInput: color.substring(1),
});
close();
if (streamChanges) {
onChange(color);
}
}}
>
Cancel
</Button>
<Button
onClick={() => {
onChange(currColor);
close();
}}
>
OK
</Button>
</div>
</>
);
}}
</PopoverPanel>
/>
<Input
style={{ width: "198px" }}
icon={faHashtag}
value={textInput}
onChange={handleTextInput}
/>
{!closeOnMouseLeave && (
<div className="flex w-full justify-center gap-2">
<Button
variant="secondary"
onClick={() => {
setStates({
currColor: prevColor,
textInput: prevColor.substring(1),
});
setIsOpen(false);
close();
}}
>
Cancel
</Button>
<Button
onClick={() => {
onChange(currColor);
setIsOpen(false);
close();
}}
>
OK
</Button>
</div>
)}
</PopoverPanel>
</div>
)}
</>
)}
</Popover>
);
};
@@ -151,7 +151,11 @@ export const TextEditor = ({
onChangeTextDecoration={onChangeTextDecoration}
/>
</div>
<Textarea {...unionedTextAreaProps} ref={textAreaRef} />
<Textarea
className="bg-white"
{...unionedTextAreaProps}
ref={textAreaRef}
/>
</div>
);
};
@@ -1,4 +1,4 @@
export const buttonGroupButton =
"size-10 rounded-none bg-white text-black hover:bg-gray-200 data-[selected=true]:text-primary-400 disabled:hover:bg-transparent disabled:text-gray-400";
"size-10 rounded-none bg-ui-controls text-white/80 hover:bg-white/30 data-[selected=true]:text-white data-[selected=true]:bg-primary/30 disabled:hover:bg-transparent disabled:text-gray-400 border border-ui-border";
export const buttonGroupGrouper =
"flex divide-x overflow-hidden rounded-md border";
"flex divide-x overflow-hidden rounded-md border border-ui-border";
@@ -7,15 +7,18 @@ export const Tooltip = ({
tip,
children,
forceShow,
position = "top",
}: {
tip: string;
children: ReactElement<any, string | JSXElementConstructor<any>>;
forceShow?: boolean;
position?: "top" | "bottom" | "left" | "right";
}) => {
const clonedChildren = cloneElement(children, {
"data-tooltip": tip,
className: twMerge(
toolTipStyles,
toolTipStyles.base,
toolTipStyles[position],
forceShow && "after:block before:block",
children.props.className,
),
@@ -0,0 +1,17 @@
import { useEffect, useState } from "react";
export const useIsMobile = () => {
const [isMobile, setIsMobile] = useState(false);
useEffect(() => {
const checkMobile = () => {
return /Android|webOS|iPhone|iPad|iPod|BlackBerry|IEMobile|Opera Mini/i.test(
navigator.userAgent,
);
};
setIsMobile(checkMobile());
}, []);
return isMobile;
};
@@ -1,5 +1,6 @@
import { signal } from "@preact/signals-react";
import { ToolbarMainButtonNames } from "~/components/features/ToolbarMain/enum";
import { isToolbarMode } from "~/signals/uiEvents/toolbarMain/toolbarMain";
interface ToolbarMainSignalInterface {
disabled: boolean;
@@ -42,6 +43,17 @@ export const toolbarMain = {
},
};
},
setMode(mode: ToolbarMainButtonNames) {
if (!isToolbarMode(mode)) {
return;
}
this.changeButtonState(ToolbarMainButtonNames.SELECT, { active: false });
this.changeButtonState(ToolbarMainButtonNames.ERASER, { active: false });
this.changeButtonState(ToolbarMainButtonNames.PAINT, { active: false });
this.changeButtonState(mode, { active: true });
}
};
function initButtonStates() {
@@ -10,12 +10,13 @@ type ButtonStates = {
};
};
export interface ContextualToolbarProps extends ContextualUi {
knodeIds: number[];
knodeIds: string[];
disabled: boolean;
downloadUrl?: string;
locked: boolean | "unknown";
lockDisabled: boolean;
buttonStates: ButtonStates;
color: string;
}
interface PartialContextualToolbarProps
extends Partial<Omit<ContextualToolbarProps, "buttonStates">> {
@@ -32,6 +33,7 @@ const toolbarNodeSignal = signal<ContextualToolbarProps>({
locked: false,
lockDisabled: false,
buttonStates: initButtonStates(),
color: "#000000",
});
export const toolbarNode = {
@@ -77,6 +79,11 @@ export const toolbarNode = {
hidden: false,
active: buttonStates[ButtonNames.TRANSFORM].active,
},
[ButtonNames.COLOR]: {
disabled: locked,
hidden: false,
active: buttonStates[ButtonNames.COLOR].active,
},
},
locked: locked,
};
@@ -135,6 +142,12 @@ export const toolbarNode = {
},
};
},
setColor(color: string) {
toolbarNodeSignal.value = {
...toolbarNodeSignal.value,
color,
}
},
batchButtonStates({ buttonStates }: { buttonStates: Partial<ButtonStates> }) {
const prevButtonStates = toolbarNodeSignal.value.buttonStates;
toolbarNodeSignal.value = {
@@ -1,11 +1,12 @@
import { signal, effect } from "@preact/signals-react";
import { TextNodeData } from "~/KonvaApp/types";
import { ShapeNodeData, TextNodeData } from "~/KonvaApp/types";
// ADDING IMAGES
const stagedImage = signal<File | null>(null);
const addImageToEngine = (image: File) => {
stagedImage.value = image;
};
const onGetStagedImage = (callback: (file: File) => void) => {
effect(() => {
if (stagedImage.value) {
@@ -48,15 +49,30 @@ const onAddTextToEngine = (callback: (data: TextNodeData) => void) => {
});
};
// Add shapes
const stagedShape = signal<ShapeNodeData | null>(null);
const addShapeToEngine = (data: ShapeNodeData) => {
stagedShape.value = data;
}
const onAddShapeToEngine = (callback: (data: ShapeNodeData) => void) => {
effect(() => {
if (stagedShape.value) {
callback(stagedShape.value);
}
});
};
//EXPORTS
export const dispatchers = {
addImageToEngine,
addVideoToEngine,
addTextToEngine,
addShapeToEngine,
};
export const events = {
onGetStagedImage,
onGetStagedVideo,
onAddTextToEngine,
onAddShapeToEngine,
};
@@ -24,6 +24,10 @@ import {
dispatchers as toolbarVideoExtractionDispatchers,
eventsHandlers as toolbarVideoExtractionEvents,
} from "./toolbarVideoExtraction";
import {
dispatchers as promptDispatchers,
events as promptEvents,
} from "./promptSettings";
export const uiEvents = {
...addMediaEvents,
@@ -34,6 +38,7 @@ export const uiEvents = {
toolbarMain: toolbarMainEvents,
toolbarNode: toolbarNodeEvents,
toolbarVideoExtraction: toolbarVideoExtractionEvents,
promptEvents: promptEvents,
};
export const dispatchUiEvents = {
...addMediaDispatchers,
@@ -44,4 +49,5 @@ export const dispatchUiEvents = {
toolbarMain: toolbarMainDispatchers,
toolbarNode: toolbarNodeDispatchers,
toolbarVideoExtraction: toolbarVideoExtractionDispatchers,
promptSettings: promptDispatchers,
};
@@ -0,0 +1,44 @@
import { effect, signal } from "@preact/signals-react";
// Prompt settings
// - Prompt text
// - Prompt strength
export const DEFAULT_PROMPT_TEXT: string = "";
export const promptText = signal<string>(DEFAULT_PROMPT_TEXT);
const setPrompt = (data: string) => {
promptText.value = data;
};
const onPromptTextChanged = (callback: (data: string) => void) => {
effect(() => {
if (promptText.value) {
callback(promptText.value);
}
});
};
export const DEFAULT_PROMPT_STRENGTH = 100;
export const promptStrength = signal<number>(DEFAULT_PROMPT_STRENGTH);
const setPromptStrength = (data: number) => {
// Round to integer, then clamp between 0 and 100
promptStrength.value = Math.max(0, Math.min(100, Math.round(data)));
};
const onPromptStrengthChanged = (callback: (data: number) => void) => {
effect(() => {
if (promptStrength.value) {
callback(promptStrength.value);
}
});
};
//EXPORTS
export const dispatchers = {
setPrompt,
setPromptStrength,
};
export const events = {
onPromptTextChanged,
onPromptStrengthChanged,
};
@@ -3,11 +3,14 @@ import {
loadingBarRetryDispatch,
loadingBarRetryEventHandler,
} from "./loadingBar";
import { onPaintColorChanged, setPaintColor } from "./paintMode";
export const dispatchers = {
...buttonDispatchers,
loadingBarRetry: loadingBarRetryDispatch,
setPaintColor,
};
export const eventsHandlers = {
...buttonEventsHandlers,
loadingBarRetry: loadingBarRetryEventHandler,
onPaintColorChanged
};
@@ -0,0 +1,19 @@
// Prompt settings
// - Prompt text
import { effect, signal } from "@preact/signals-react";
// - Prompt strength
export const DEFAULT_PAINT_COLOR: string = "#ff0000";
export const paintColor = signal<string>(DEFAULT_PAINT_COLOR);
export const setPaintColor = (data: string) => {
paintColor.value = data;
};
export const onPaintColorChanged = (callback: (data: string) => void) => {
effect(() => {
if (paintColor.value) {
callback(paintColor.value);
}
});
};
@@ -2,6 +2,7 @@ import { MouseEventHandler } from "react";
import { signal, effect, Signal } from "@preact/signals-react";
import { ToolbarMainButtonNames } from "~/components/features/ToolbarMain/enum";
import { AppModes } from "~/KonvaApp/constants";
const events = Object.values(ToolbarMainButtonNames).reduce(
(acc, buttonName) => {
@@ -25,22 +26,35 @@ const effectsCleanups = Object.values(ToolbarMainButtonNames).reduce(
[key in ToolbarMainButtonNames]: (() => void) | undefined;
},
);
export const isToolbarMode = (buttonName: ToolbarMainButtonNames) => {
return appModeMap[buttonName] !== undefined;
}
export const appModeMap: Record<string, AppModes> = {
[ToolbarMainButtonNames.SELECT]: AppModes.SELECT,
[ToolbarMainButtonNames.ERASER]: AppModes.ERASER,
[ToolbarMainButtonNames.PAINT]: AppModes.PAINT,
}
const defaultHandler = (buttonName: ToolbarMainButtonNames, callback: MouseEventHandler<HTMLButtonElement>) => {
if (effectsCleanups[buttonName]) {
effectsCleanups[buttonName]();
}
effectsCleanups[buttonName] = effect(() => {
if (events[buttonName].value) {
callback(events[buttonName].value);
events[buttonName].value = undefined;
}
});
}
export const buttonEventsHandlers = Object.values(
ToolbarMainButtonNames,
).reduce(
(acc, buttonName) => {
acc[buttonName] = {
onClick: (callback: MouseEventHandler<HTMLButtonElement>) => {
if (effectsCleanups[buttonName]) {
effectsCleanups[buttonName]();
}
effectsCleanups[buttonName] = effect(() => {
if (events[buttonName].value) {
callback(events[buttonName].value);
events[buttonName].value = undefined;
}
});
},
onClick: (callback: MouseEventHandler<HTMLButtonElement>) => { defaultHandler(buttonName, callback) },
};
return acc;
},
@@ -44,6 +44,11 @@ const buttonEventsRecords = Object.values(ButtonNames).reduce(
const buttonEventsHandlers = Object.values(ButtonNames).reduce(
(acc, buttonName) => {
// We define custom events/handlers for color changes
if (buttonName === ButtonNames.COLOR) {
return acc;
}
acc[buttonName] = {
onClick: (callback: MouseEventHandler<HTMLButtonElement>) => {
const { eventSignal, effectCleanup, lastEventTimestamp } =
@@ -111,9 +116,27 @@ const retryEventHandler = (callback: MouseEventHandler<HTMLButtonElement>) => {
const retryDispatcher = (e: React.MouseEvent<HTMLButtonElement>) => {
retryEvent.value = e;
};
export type NodeColor = {
kNodeId: string;
color: string;
}
const selectedColorEvent = signal<NodeColor | null>();
const setSelectedColor = (nodeColor: NodeColor) => {
selectedColorEvent.value = nodeColor;
}
const onSelectedColorChanged = (callback: (color: NodeColor) => void) => {
effect(() => {
if (selectedColorEvent.value) {
callback(selectedColorEvent.value);
}
});
};
export const dispatchers = {
lock: lockDispatcher,
retry: retryDispatcher,
setSelectedColor,
...buttonDispatchers,
};
export const eventsHandlers = {
@@ -123,5 +146,8 @@ export const eventsHandlers = {
retry: {
onClick: retryEventHandler,
},
color: {
onConfirmChanged: onSelectedColorChanged,
},
...buttonEventsHandlers,
};
+9 -4
View File
@@ -44,16 +44,21 @@ export default {
info: colors.sky[500],
// brand colors
primary: storytellerColors.sunglo,
primary: storytellerColors.azureRadiance,
secondary: storytellerColors.gunpowder,
tertiary: storytellerColors.aquamarineBlue,
ui: {
background: colors.zinc[100],
panel: colors.white,
border: colors.zinc[200],
background: "#101014",
panel: "#3E3E41",
border: "#FFFFFF0D",
controls: "#3E3E41",
},
},
backgroundImage: {
"radial-gradient":
"radial-gradient(circle at left, var(--tw-gradient-stops))",
},
},
plugins: [],
};
+14 -14
View File
@@ -1,19 +1,19 @@
export const storytellerColors = {
sunglo: {
//sunglo
DEFAULT: "#e66462", //=500
50: "#fdf3f3",
100: "#fce4e4",
200: "#facfce",
300: "#f6acab",
400: "#ef7c7a",
500: "#e66462",
600: "#cf3533",
700: "#ae2927",
800: "#902624",
900: "#782524",
950: "#410f0e",
azureRadiance: {
DEFAULT: "#2d81ff", //=500
50: "#eef6ff",
100: "#d9ebff",
200: "#bcdcff",
300: "#8ec7ff",
400: "#59a7ff",
500: "#2d81ff",
600: "#1b63f5",
700: "#144ee1",
800: "#173fb6",
900: "#19398f",
950: "#142457",
},
gunpowder: {
DEFAULT: "#39394c", //=950
50: "#f5f6f9",
@@ -79,9 +79,6 @@ export const ToolbarMain = ({
<ToolbarButton icon={faImage} buttonProps={buttonProps.ADD_IMAGE}>
Add Image
</ToolbarButton>
<ToolbarButton icon={faFilm} buttonProps={buttonProps.ADD_VIDEO}>
Add Video
</ToolbarButton>
</PopoverPanel>
</Popover>
<ToolbarButton
+12
View File
@@ -0,0 +1,12 @@
#!/usr/bin/env bash
# Tauri doesn't let you configure the frontend project directory statically, though they do provide an
# environment variable to pass it to the CLI. Without doing this, the tauri cli randomly walks the
# filesystem and finds the wrong frontend code.
# TODO(bt,2025-02-13): This is not the correct way to get the root dir
root_dir=$(pwd)
export TAURI_FRONTEND_PATH="${root_dir}/frontend/realtime"
cargo tauri build
+1 -1
View File
@@ -8,4 +8,4 @@
root_dir=$(pwd)
export TAURI_FRONTEND_PATH="${root_dir}/frontend/realtime"
cargo tauri dev
cargo tauri dev --no-watch