Compare commits

...
Author SHA1 Message Date
bflatastic 41c24e0655 focal length value to show on top when dragging slider 2025-04-11 02:02:09 +07:00
bflatastic 126f1b46d6 working focal length changing 2025-04-11 01:48:07 +07:00
bflatastic adfebced8e fixed - can drop object over letterbox now 2025-04-11 00:58:43 +07:00
bflatastic e85c861d42 removed 16:9 aspect choice from both 2d and 3d 2025-04-11 00:42:26 +07:00
bflatastic beeeae082a added shortcut "b" key to open/browse 3d assets modal 2025-04-11 00:37:09 +07:00
bflatastic 235b3a4ce5 restore day skybox and default to empty 2025-04-11 00:27:04 +07:00
bflatastic d554abe5d4 reducing wasd onboarding helper close delay 2025-04-11 00:10:48 +07:00
bflatastic f93441084f fixed - better grid raycast object drop on scene 2025-04-11 00:05:46 +07:00
bflatastic 06041a01e6 grid toggle button and better grid look 2025-04-10 23:31:05 +07:00
Kasisnu f22d267386 image-gen: job progress reporting should work 2025-04-10 21:00:07 +05:30
bflatastic 4d3e326374 controls 3d border styling fix 2025-04-10 21:00:21 +07:00
bflatastic 87f8540424 added more tabs to assets modal and save fix 2025-04-10 20:18:01 +07:00
bflatastic cff41464a6 help menu update 2025-04-10 18:10:25 +07:00
bflatastic 425b21b586 onboarding helper 2025-04-10 18:04:18 +07:00
bflatastic f3ef51f9ea top mode buttons signal linked to keybinds, default, asset modal defaulted to not reopen 2025-04-10 17:13:54 +07:00
bflatastic 51797b256e moved 3d asset button to top for visibility 2025-04-10 15:28:18 +07:00
Tanish Manku 2b453660da 3D Editor: Persist gizmo is FK mode (#493) 2025-04-10 01:18:57 -07:00
BFlat e55bc020df asset modal (#490)
- replaces sidebar with asset modal button on left side
- editor selection outline color change
2025-04-10 14:32:11 +07:00
Brandon Thomas 1ae844259f fix build, remove cuda from hakari 2025-04-09 22:07:12 -04:00
Brandon Thomas 05e64de573 share sora credential sync code between web and job 2025-04-09 21:32:22 -04:00
Brandon Thomas 61043c6ea3 recalculate hackari workspace acceleration package 2025-04-09 20:41:52 -04:00
Brandon Thomas 8845b177be rename workspace hack 2025-04-09 20:39:36 -04:00
Brandon Thomas c4e591ef0c refactor packages 2025-04-09 18:57:53 -04:00
Brandon Thomas ba071987d8 move ml crates into lib/ 2025-04-09 17:44:38 -04:00
Brandon Thomas 62fb799775 fix sora downloads 2025-04-09 17:38:35 -04:00
Brandon Thomas 241a9d3031 log details 2025-04-09 15:38:17 -04:00
ArEnScandMichael Chung 06a0a8070d polling jobs implemented (#492)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-09 15:15:13 -04:00
Brandon Thomas e8beb2ff8c fix missing enum variant 2025-04-09 15:02:47 -04:00
Brandon ThomasandKasisnu 76ccbe3291 Kasisnu's OAI inference job, task completion (#491)
Co-authored-by: Kasisnu <5044120+kasisnu@users.noreply.github.com>
2025-04-09 14:47:38 -04:00
Brandon Thomas 3817f208d3 ignore node modules directories in git 2025-04-09 01:23:30 -04:00
ArEnScandMichael Chung a22710e1a5 jobs backend doesn't work work so the last piece is just the UI from … (#489)
…the jobs response?

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-08 20:10:18 -04:00
ArEnScandMichael Chung e27ebc7961 properly wired up the image generation api (#488)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-08 18:56:57 -04:00
Brandon Thomas cafd37c313 lighten the GPU test heuristics 2025-04-08 18:42:45 -04:00
Brandon Thomas 6be886ce4d launch 3d engine locally in tauri (failing due to gpu acceleration) 2025-04-08 18:22:14 -04:00
Brandon Thomas 2652897109 try to fix netlify builds of 2d 2025-04-08 15:19:14 -04:00
Brandon Thomas 0cb594a1eb script to launch 2D + tauri on desktop 2025-04-08 15:03:32 -04:00
BFlat 2d635f3f8e added activity with dummy polling and image lightbox modal + other cleanup (#487) 2025-04-09 01:45:43 +07:00
Brandon Thomas ea783c14de forgotten file 2025-04-08 13:59:49 -04:00
BFlat 01093918da library modal (#486)
- for selecting existing images from library (+ in prompt box)
- for viewing media in library (my library button on top right)
2025-04-08 22:42:56 +07:00
Brandon Thomas 2ea9464be2 rewire prompting 2025-04-08 02:21:17 -04:00
Brandon Thomas b41b4f8e17 read/write same sora cookie, avoid cloudflare heuristics 2025-04-07 21:17:26 -04:00
ArEnScandMichael Chung 09cf4529c0 Feature/reshape UI (#482)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-07 18:40:17 -04:00
Brandon Thomas bf035f9048 login flow 2025-04-07 17:50:48 -04:00
Brandon Thomas 448c760339 fix login 2025-04-07 16:35:14 -04:00
bflatastic 7a55592bfd outliner glass bg 2025-04-07 22:25:12 +07:00
bflatastic a54977d392 in progress generation ui update 2025-04-07 21:16:50 +07:00
bflatastic f4268038c5 enqueue loading fix 2025-04-07 15:33:59 +07:00
bflatastic f81a215133 tooltip updates + close on click / popover open 2025-04-07 15:16:41 +07:00
Brandon Thomas b2016dcfb9 open window, bidirectional communication 2025-04-07 01:54:19 -04:00
Brandon Thomas 2ca80ed9dc we can inspect 3rd party website cookies (yikes, but also wow) 2025-04-07 01:21:19 -04:00
Brandon Thomas 320af54e14 sora auth 2025-04-06 22:48:04 -04:00
Brandon Thomas 391ca1db57 call 4o from desktop 2d app 2025-04-06 17:11:02 -04:00
Brandon Thomas 73de2781a9 new tauri app runs 2025-04-05 21:20:59 -04:00
Brandon Thomas a65bdcd882 Version of tauri app for 4o 2025-04-05 20:26:04 -04:00
Brandon Thomas 3c5fd9ce65 restore some sidebar items 2025-04-04 14:03:28 -04:00
bflatastic d82be338fc image uploading spinner 2025-04-04 23:05:31 +07:00
Kasisnu 23f762da8f image-gen: add a todo about the failing status code 2025-04-04 20:50:53 +05:30
BFlat 8b21489453 fix action upload (#480) 2025-04-04 21:56:31 +07:00
BFlat f9a9168ad5 camera settings modal ui (#479) 2025-04-04 20:56:39 +07:00
BFlat 8cbdd22b46 activity popover for generations (#478) 2025-04-04 18:32:43 +07:00
BFlat f456795fb1 reusable toggle button for system prompt (#477) 2025-04-04 16:35:17 +07:00
Tanish Manku 63d41f7d2c Some bug fixes and improvements for FK (#476) 2025-04-04 00:41:39 -07:00
Tanish Manku 14474e68c2 Add basic Forward Kinematics support (#475)
There's still some glitches around selection and deletion.
2025-04-04 00:34:09 -07:00
Michael Chung 2f44da8ac7 Changed api calls to use file upload and uuid idempotency token and stuff works. 2025-04-03 23:10:23 -04:00
Brandon Thomas 3d67f86a19 set sora creds externally 2025-04-03 22:26:38 -04:00
Brandon Thomas 2913d91d0d fix media file downloading 2025-04-03 18:47:57 -04:00
ArEnScandMichael Chung ceb77505e1 Feature/gen (#474)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-03 18:01:29 -04:00
ArEnScandMichael Chung 0cad3fe7da Wire screenshot upload and generate to the engine. (#470)
Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-03 17:38:34 -04:00
Tanish Manku 1ced74040f Fix: Letterboxing blocking pointer events 2025-04-03 14:09:35 -07:00
Brandon ThomasandKasisnu c9dd545ae1 Use OpenAI Sora GPT 4o image gen (#472)
Co-authored-by: Kasisnu <5044120+kasisnu@users.noreply.github.com>
2025-04-03 17:07:20 -04:00
bflatastic 8ff91690c1 bring back objects, skybox, creatures tabs 2025-04-03 23:24:14 +07:00
BFlat 97bbe0fe07 add image ui (#471) 2025-04-03 23:01:02 +07:00
BFlat 9361634d83 final fix ratio default to 3:2 (#469) 2025-04-03 12:41:40 +07:00
bflatastic 7cf0363555 fixed ratio in render 3:2 2025-04-03 12:32:45 +07:00
bflatastic c2ee3b0ce1 render default to 3:2 ratio 2025-04-03 12:30:57 +07:00
BFlat ca6fc13171 aspect ratio update (#468)
- added 3:2, 2:3 ratios (defaulted to 3:2 [1200 x 800 px])
- added letterbox accordingly
- aspect ratio buttons to show icons for each ratio
- popover to close when item selected
2025-04-03 12:23:54 +07:00
Brandon Thomas cb1ea1f828 fix CDN in local development 2025-04-02 18:48:44 -04:00
bflatastic 010e3be5dc make letterbox always visible 2025-04-02 22:59:59 +07:00
bflatastic d8dab1a66e fix trimming 2 2025-04-02 22:42:28 +07:00
bflatastic 97b236d04d paste trim fix 2025-04-02 22:40:56 +07:00
bflatastic 15fe59cc19 added enqueueing state to generate button, limited textarea height and updates 2025-04-02 22:36:53 +07:00
BFlat 89bec31540 removed video controls and add prompt box (#467) 2025-04-02 22:01:19 +07:00
ArEnScandMichael Chung 7568a0ef86 Fixes for the new 3d editor (#465)
Removed the login page and timeline and confusing UI components.
Extended scene to the bottom of the screen.

Co-authored-by: Michael Chung <michaelchung@michaels-mbp-2.home>
2025-04-01 19:11:42 -04:00
Brandon Thomas 6180222e15 netlify.toml for editor2d 2025-04-01 00:48:53 -04:00
Brandon Thomas 6d45ce29e2 Adding copy of realtime as editor2d for web deployment 2025-04-01 00:44:28 -04:00
Brandon Thomas 1da56d6d22 netlify build outputs dir 2025-04-01 00:06:22 -04:00
Brandon Thomas 91a1c00442 reserved names 2025-04-01 00:00:52 -04:00
Brandon Thomas 3db942f1f6 artcraft cors 2025-03-31 23:58:48 -04:00
Brandon Thomas 2e56291649 unblock rust docker builds 2025-03-31 21:41:54 -04:00
Brandon Thomas b31badc6d1 Editor3D - this directory is a fork of "studio gen 2" by Tanish.
We'll be using this for our image generation roadmap:

https://docs.google.com/document/d/1O9YI_iLwXkl2wAJJ23dFp3eHR-JzhlRHCBPD8psG-1E/edit

The original Studio Gen 2 is in the `frontend/studio-gen-2` directory.

The original Studio Gen 1 is in the `frontend/studio-gen-1` directory
2025-03-31 13:28:59 -04:00
Brandon Thomas ed399fcb21 rename storyteller animator to "studio-gen-2" 2025-03-31 13:25:51 -04:00
Brandon Thomas d858c9394b work in progress background removal adjustment 2025-03-28 01:33:52 -04:00
Brandon Thomas a19a6599ec rename model 2025-03-27 23:02:51 -04:00
Brandon Thomas 57c15f7c9f rename realtime to sketch 2025-03-27 22:41:40 -04:00
Brandon Thomas aa25ae8c87 load flux models from R2 2025-03-27 21:04:39 -04:00
Brandon Thomas c27055761e fix 2025-03-26 20:35:50 -04:00
Brandon Thomas abf3024892 add to downloaders 2025-03-26 20:35:11 -04:00
Brandon Thomas 2182d4dde1 lots of new models in R2 2025-03-26 20:32:05 -04:00
Brandon Thomas b632b927a3 save images to file 2025-03-26 13:12:54 -04:00
Brandon Thomas 4e2be12064 flux cherry picked in 2025-03-26 02:37:04 -04:00
Brandon Thomas 29b33cd327 save image handler 2025-03-25 22:43:25 -04:00
Brandon Thomas 032d0e1eae remove lock button 2025-03-25 20:05:49 -04:00
Brandon Thomas a3ddffeeee get rid of "transform" / "move" context menu 2025-03-25 19:54:51 -04:00
Brandon Thomas 28a5ff2f9f remove background from preview images 2025-03-25 19:43:48 -04:00
Brandon Thomas 1028e04e02 turn lcm println!() to debug!() level 2025-03-25 17:45:29 -04:00
Brandon Thomas 49c010db59 temporarily remove references to flux (temp fix) 2025-03-25 17:34:33 -04:00
Kasisnu 48ff60d3b8 super basic flux (#463) 2025-03-25 17:05:50 -04:00
Brandon Thomas 9c5b993c4f read seed from config too 2025-03-25 12:34:33 -04:00
Brandon Thomas a4aecc4d38 send seed as a parameter 2025-03-25 12:21:47 -04:00
Brandon Thomas 4d73230e84 add logging 2025-03-25 10:44:11 -04:00
Brandon Thomas a5a09f3e64 fixup inpaint stub 2025-03-25 01:44:20 -04:00
49894bd9b3 Generate Mode (#461)
Co-authored-by: Tanish Manku <-h>
Co-authored-by: bflatastic <bombayapirat@gmail.com>
2025-03-25 01:38:56 -04:00
Brandon Thomas 81c256e4f8 cleanup 2025-03-24 23:08:07 -04:00
Brandon Thomas 8d51d5b8ab can't trigger default features; switch to platform switches 2025-03-24 23:00:29 -04:00
Brandon Thomas 456bfd7bb9 trigger with --feature ml_models (need to make default) 2025-03-24 22:31:58 -04:00
Brandon Thomas cd3c864eff runs on mac 2025-03-24 21:46:25 -04:00
Brandon Thomas 990b7ff3e5 package split for mac (part one) (#459) 2025-03-24 20:18:57 -04:00
Brandon Thomas e3245dc593 fix on windows 2025-03-22 00:58:04 -04:00
Brandon Thomas 08590d0c61 log environment details 2025-03-21 01:11:34 -04:00
Brandon Thomas 32cb39a643 text_to_image endpoint 2025-03-20 23:28:59 -04:00
Brandon Thomas 5fdb4d1024 disable msi builds due to 2.0 gb limit for 32bit 2025-03-20 11:59:43 -04:00
Brandon Thomas d11945b685 download DIS model 2025-03-20 11:07:34 -04:00
Brandon Thomas f3babbcb19 if statements -> switch statements 2025-03-20 10:53:34 -04:00
Brandon Thomas f37011ffc6 remove background: frontend UX (#458) 2025-03-20 01:41:43 -04:00
Brandon Thomas de683b750a remove background from images 2025-03-18 22:10:18 -04:00
Brandon Thomas 4603c65d2a fix strength slider range 2025-03-18 18:21:39 -04:00
Brandon Thomas c5b0d837a9 fix bug with float strengths from frontend 2025-03-18 01:48:54 -04:00
Brandon Thomas 652785224a easier windows builds. put dlls in the windows_dlls folder 2025-03-14 04:22:18 -04:00
Brandon Thomas 8d3ed947d1 change default prompt, log end of endpoint logic 2025-03-14 03:20:46 -04:00
Brandon Thomas 243583f0b7 kill more dead code 2025-03-13 23:39:04 -04:00
Brandon Thomas 150223c48c refactor weights (#455) 2025-03-13 23:29:20 -04:00
bflatastic ddf6a7f54e top center ui bug fix and padding 2025-03-13 22:28:26 +07:00
Brandon Thomas 17c6e549b5 ignore windows dlls 2025-03-13 00:38:10 -04:00
Brandon Thomas 031d9d8f20 Download models from Cloudflare instead of HF (#454) 2025-03-13 00:32:43 -04:00
Brandon Thomas 43df6c9181 app default size 2025-03-12 21:58:27 -04:00
Tanish Manku a8e029497f Fix context menu showing up after transforming (#453) 2025-03-12 15:18:00 -07:00
Brandon Thomas 4936662ddd correct default path on windows 2025-03-12 18:08:23 -04:00
BFlat 883e32220c tabs and app mode signals (#450)
Usage example - Conditional Rendering Based on App Mode:
```typescript
import { appMode } from "~/signals";

function ConditionalComponent() {
  return (
    <div className="app-container">
      {/* Show/hide components based on current mode */}
      {appMode.value === 'image' && (
        <div className="image-tools">
          <h3>Image Mode Tools</h3>
        </div>
      )}
      
      {appMode.value === 'edit' && (
        <div className="edit-tools">
          <h3>Edit Mode Tools</h3>
        </div>
      )}
  );
}
```
2025-03-12 13:26:10 -04:00
Brandon Thomas 727eead5a7 fix windows search for missing stdc++.lib during build 2025-03-12 01:27:27 -04:00
Kasisnu b4cc5b84cf f16 2025-03-11 22:30:02 +05:30
Brandon Thomas d7e77775ca Add new premium plan identifiers 2025-03-11 02:15:46 -04:00
Brandon Thomas b75472c434 fix windows builds by disabling flash attn/cutlass, openssl, etc. 2025-03-11 00:49:30 -04:00
Brandon Thomas a4f876daa9 Multithreaded downloader + notifications (#447) 2025-03-10 21:51:07 -04:00
Tanish Manku 4b309a5dd8 Render shapes as vectors and revamp rendering logic (#442) 2025-03-07 14:41:11 -08:00
Tanish Manku 08384955e4 Remove 'React-Eyedrop' and unused imports (#443)
As simple as it sounds
2025-03-06 23:05:23 -08:00
Brandon Thomas 31037b83a3 downloader thread 2025-03-06 22:48:48 -05:00
Brandon Thomas 53d93cb1ac change name of app 2025-03-06 21:45:01 -05:00
Brandon Thomas a0de8fc603 load all models from model registry 2025-03-06 21:39:19 -05:00
Brandon Thomas ef3527dd85 weights directory, assets directory 2025-03-06 21:06:58 -05:00
Brandon Thomas 1a887d7414 don't download anything from huggingface 2025-03-06 19:27:22 -05:00
Brandon Thomas 48abc0d731 no longer download clip encoders from huggingface 2025-03-06 15:22:10 -05:00
CrossPr0duct 0233105393 Merge branch 'main' of github.com:storytold/storyteller-rust 2025-03-06 15:12:28 -05:00
CrossPr0duct 8f94f81db4 Background Hack Implemented without engine refactors. 2025-03-06 15:12:04 -05:00
Kasisnu 3f25aefc3f starting point (#440) 2025-03-06 10:41:08 -05:00
BFlat c2929cfdb6 color picker ux and make paint cursor visible on both dark/light bg (#441)
ux updates to color picker and paint cursor
2025-03-06 20:52:25 +07:00
Brandon Thomas a859bd427b download clip models (not using them yet) 2025-03-06 08:42:44 -05:00
Brandon Thomas 706becfd7e download unet 2025-03-05 19:32:41 -05:00
Brandon Thomas ada1f46e62 model downloading (wip) 2025-03-05 18:24:05 -05:00
bflatastic 3556dcbfd3 removed annoying focus on buttons 2 2025-03-05 23:40:33 +07:00
bflatastic 4ab6ca1caa disabled annoying focus ring 2025-03-05 23:21:11 +07:00
BFlat 74f5b14d18 added loading indicator text (#439)
added signal for loading indicator text change
2025-03-05 10:23:48 +07:00
CrossPr0duct ac6dcfbc05 Selector Square Bug fixed.
Added Loading to RealTimeDrawEngine.
2025-03-04 20:20:33 -05:00
Brandon Thomas 68b0acf8ed always send event for now 2025-03-04 19:43:30 -05:00
Brandon Thomas f783910c0a vendor in cutlass 2025-03-04 19:28:28 -05:00
Brandon Thomas f3ee145f65 vendor in candle-examples 2025-03-04 19:18:18 -05:00
Brandon Thomas f8a2f4f858 marking function async makes tauri not lock up main thread 2025-03-04 18:28:28 -05:00
Brandon Thomas 3d7e558884 also send model type with event 2025-03-04 18:23:21 -05:00
CrossPr0duct 51d941a35d Merge rust 2025-03-04 18:01:24 -05:00
CrossPr0duct e56ce47205 Remove eraser button 2025-03-04 18:00:55 -05:00
Brandon Thomas 26c7b29c17 cleanup unet caching, report download 2025-03-04 17:51:14 -05:00
Brandon Thomas 89d2898e5f send notification events 2025-03-04 17:10:05 -05:00
Brandon Thomas 8124cbbbe1 destructure 2025-03-04 16:32:45 -05:00
Brandon Thomas f6b3ceb537 fix bad abstraction for vae caching 2025-03-04 16:24:28 -05:00
CrossPr0duct 606d1d2345 Added the progress bar. 2025-03-04 15:53:32 -05:00
Brandon Thomas e088f8e4fd send test event to javascript from tauri 2025-03-04 15:13:55 -05:00
Brandon Thomas c29d6377a4 log strength 2025-03-03 20:28:52 -05:00
Brandon Thomas 494e1e6b2b release build profile, stop writing debug file at app startup 2025-03-03 15:35:05 -05:00
BFlat 4d31f702b4 toaster for notifications (#437)
added toaster for notifications and alerts using sonner library

import:
```
import { toast } from 'sonner';
```

basic usage just call:
```
toast('This is an normal toast');

toast.error('This is an error toast');

toast.success('This is an success toast');

toast('This is an action toast', {
          action: {
            label: 'Action',
            onClick: () => console.log('Action!'),
          },
        });

toast('My cancel toast', {
  cancel: <Button onClick={() => console.log('Cancel!')}>Cancel</Button>,
});
```

more usage check here: https://sonner.emilkowal.ski/toast
2025-03-04 00:27:15 +07:00
Brandon Thomas 202b7876b4 temp test for windows 2025-03-02 20:31:34 -05:00
CrossPr0duct 360c3a268b Fixing the flow feels more natural,
swaps to shape selection or image selection when that is added,
except for render copy.
2025-03-01 21:25:27 -05:00
CrossPr0duct e2d9600458 Ensure the drag layer sits on top. 2025-03-01 21:06:21 -05:00
CrossPr0duct 31194c6622 Fixed brush issues and fixed issues with the paint server embedding lengths. 2025-03-01 19:10:16 -05:00
CrossPr0duct 5045e44af7 Painted img2img requires rescaling the image. 2025-03-01 18:09:19 -05:00
CrossPr0duct d387a04d12 Server Changes modified So it doesn't crash.
Type Fixes.
2025-03-01 17:46:29 -05:00
CrossPr0duct 0276349b5d Add Background color wire. 2025-03-01 17:03:14 -05:00
CrossPr0duct 3c037b085b Update the server lora strength.
Add Rendering for strength and text events.
2025-03-01 15:41:39 -05:00
ArEnScandCrossPr0duct 7965afd96c Feature/live drawing (#436)
Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-03-01 15:27:06 -05:00
BFlat 17dd093a23 model selection modal with signals, dummy data and dummy download progress (#435) 2025-03-01 11:39:42 -05:00
BFlat 989d36dc6c loading indicator signals (#434)
added signals for the loading indicator
2025-03-01 18:07:40 +07:00
BFlat d23199c046 added loading bar (dev only) (#433) 2025-02-27 23:48:58 +07:00
Kasisnu e67cacc23f basic lcm implementation 2025-02-27 22:00:58 +05:30
CrossPr0duct 5d9358103c Brush Size added in. 2025-02-26 20:29:22 -05:00
CrossPr0duct c36bfeb47c Merge branch 'main' of github.com:storytold/storyteller-rust 2025-02-26 19:25:46 -05:00
CrossPr0duct ad289122e0 Add Copy Canvas. 2025-02-26 19:25:15 -05:00
Tanish Manku 35a8920db8 Show context menu at right click position (#432) 2025-02-26 15:39:47 -08:00
Tanish Manku 701d84602f Add an eyedrop to all color pickers (#431) 2025-02-26 15:39:34 -08:00
CrossPr0duct 1d2a52556d Remove loading bar in context menu 2025-02-26 14:16:08 -05:00
bflatastic af44de74bc text prompt area to force blur on outside click 2025-02-26 23:17:15 +07:00
bflatastic 920cb9e751 slider center fix 2025-02-26 22:12:02 +07:00
BFlat c9c0a1bdf7 dialog transitions, model buttons, ratio button (#430)
**changes:**
- refined dialogs to use a new BaseDialog component which has sleek
transitions on open/close
- added model buttons on top center (only visible if in dev mode
currently) - these open up a modal and has no added signals yet.
- added ratio button on top right (uses new reusable ToggleButton
component) (only visible if in dev mode currently) - not signaled yet.
- added tooltip to background color button
2025-02-26 17:24:59 +07:00
Tanish Manku c65b00adc8 Add Background button and menu to toolbar (#427) 2025-02-26 00:47:51 -08:00
Tanish Manku e199086cff Context Menu Redesign (#425)
- Fix right click
- New design
2025-02-26 00:42:33 -08:00
Brandon Thomas 92d1195abf Vendor candle (#429) 2025-02-26 02:30:42 -05:00
BFlat d663eed493 render bug fix (#428) 2025-02-26 14:27:33 +07:00
CrossPr0duct 7c70634746 Merge branch 'main' of github.com:storytold/storyteller-rust 2025-02-25 23:12:57 -05:00
CrossPr0duct f6039c9967 Preview Copy still WIP UX 2025-02-25 23:12:33 -05:00
CrossPr0duct 8fea9c1a5e Working on previews 2025-02-25 23:09:48 -05:00
Tanish Manku ecee0d2ea8 Make color picker and paint menu use hex alpha picker (#426) 2025-02-25 18:12:41 -08:00
CrossPr0duct 0d41a52d5e Hide server stuff 2025-02-25 20:33:30 -05:00
CrossPr0duct 8ce8d111eb Merge branch 'main' of github.com:storytold/storyteller-rust 2025-02-25 18:40:12 -05:00
CrossPr0duct b1c20c7d42 Adding a test server. 2025-02-25 18:39:48 -05:00
Tanish Manku 000638ab9f Add brush size to Paint and Erase (#424)
- Both modes' menus have been separated for better maintenance
- Slider is generic, can be reused
- New signals added for engine to hook into
- Sliders tested working on the UI side
2025-02-25 02:45:52 -08:00
CrossPr0duct ef8b308cf0 Fixed issue with the translate and undo. 2025-02-24 23:37:20 -05:00
CrossPr0duct 5ea12fd682 Fix shape recurson due to app mode. 2025-02-24 22:41:16 -05:00
ArEnScandCrossPr0duct 1d5c4df906 Feature/paint (#423)
Paint Feature.
Fixes any issues involving rendering to the other frame for undo redo
stack.
Adds the painting feature as well as colors for it.
Fixes issues with transform updates.

---------

Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-02-24 21:11:33 -05:00
ae6434e11e Add mode switching: SELECT, PAINT, ERASE (#420)
Co-authored-by: BFlat <bombayapirat@gmail.com>
Co-authored-by: CrossPr0duct <alicextensor@gmail.com>
2025-02-23 21:30:00 -05:00
Michael Chung 6077232cd3 Hide Preview Drag Layer as it is not done. 2025-02-21 17:21:43 -05:00
Michael Chung f013373180 Copy Shift Still needs some UX work but it does work.
Need to store it as a image node.
2025-02-21 00:39:59 -05:00
Michael Chung f0c1fcf516 Got the color changes to work. 2025-02-20 22:34:17 -05:00
Brandon Thomas 9f392de9c5 bump packages to enable windows builds 2025-02-20 18:10:07 -05:00
BFlat f5fb340de9 landing hero update (#419) 2025-02-21 01:49:28 +07:00
Tanish Manku 0392bda03c Forward color picker events to engine (#418) 2025-02-20 11:38:29 -05:00
Michael Chung 1bca478621 Move color picker into the correct toolbar 2025-02-19 23:56:45 -05:00
Michael Chung f9c966003a Fixed issue with shapes not having the context menu component. 2025-02-19 23:18:28 -05:00
Michael Chung 2d8245cd55 Added Shapes.
Removed Extra buttons.
2025-02-19 22:29:31 -05:00
Tanish Manku 9d1d605f5c Add shapes to global toolbar and color picker to contextual (#417)
Shapes scaffolding is fully set up, color picker isn't tied up to the
nodes so it doesn't actually change the color yet
2025-02-19 21:55:06 -05:00
Michael Chung 1e38fbad5c Turns out you need to convert these rasters into images. 2025-02-19 20:33:01 -05:00
Michael Chung 6e03df968d Remove duplicate adds?
Then fixed layering issue by removing useless background layer.
Need to fix shape rendering.
2025-02-19 19:34:06 -05:00
Michael Chung 13e0e4bb02 Merge branch 'main' of github.com:storytold/storyteller-rust 2025-02-19 18:00:26 -05:00
Michael Chung 442c3579e1 Fixed Modal to have the image enter the scene right away.
Added Shape Fixes resizes properly and drags.
Removing Progress Fill.
2025-02-19 17:59:14 -05:00
bflatastic b8a305e271 mobile download button disable landing 2025-02-19 22:58:22 +07:00
BFlat a70061bfd3 realtime product landing (#415) 2025-02-19 20:44:22 +07:00
Brandon Thomas e7a392b8e2 add cfg (#416) 2025-02-19 00:21:24 -05:00
Michael Chung 9147da22f2 ShapeNodes work need to fix transformations. 2025-02-19 00:09:47 -05:00
Brandon Thomas 29c2df3ee6 strength, negative prompt 2025-02-18 21:42:22 -05:00
Brandon Thomas 1db4278712 cleanup 2025-02-18 02:19:38 -05:00
Brandon Thomas 5ee910ad07 pass prompt from frontend 2025-02-18 01:57:37 -05:00
Brandon ThomasandKasisnu ff9d2c6b34 realtime + speedup (#414)
Co-authored-by: Kasisnu <5044120+kasisnu@users.noreply.github.com>
2025-02-18 01:29:56 -05:00
Michael Chung 462dcd7645 Removed buttons that didn't need to be there.
Wired up prompt and strength sliders and save image.
Removed some more code.
2025-02-14 17:35:48 -05:00
Michael Chung d1617f88a4 Fixed UI Positioning.
Tied events to the system.
Should accept realtime inputs on changes.
2025-02-14 17:10:14 -05:00
ArEnScandTanish c2e57d0815 Prompt Box and Strength (#412)
Co-authored-by: Tanish <tanish2k09.dev@gmail.com>
2025-02-14 15:44:15 -05:00
10386 changed files with 1922935 additions and 2590 deletions
+2 -2
View File
@@ -1,4 +1,4 @@
hakari-package = "my-workspace-hack"
hakari-package = "workspace-build-acceleration"
resolver = "1"
@@ -11,4 +11,4 @@ platforms = [
"x86_64-pc-windows-msvc",
]
exact-versions = true
exact-versions = true
+7
View File
@@ -32,6 +32,9 @@ target/
crates/lib/database_queries/target/
crates/lib/sqlite_queries/target/
# Javascript build directories
node_modules/
# Vim noise
*.swo
*.swp
@@ -66,3 +69,7 @@ scraps/
# These should be built by the platform
windows/sqlite/sqlite3.exp
windows/sqlite/sqlite3.lib
# ArtCraft Windows DLLs
crates/desktop/src-tauri/*.dll
windows_dlls/
+3
View File
@@ -2,6 +2,7 @@
<dictionary name="bt">
<words>
<w>arpabet</w>
<w>artcraft</w>
<w>backfill</w>
<w>backfilling</w>
<w>backfills</w>
@@ -31,6 +32,7 @@
<w>hifigan</w>
<w>hparams</w>
<w>huggingface</w>
<w>kasisnu</w>
<w>lipsync</w>
<w>mocap</w>
<w>mochi</w>
@@ -47,6 +49,7 @@
<w>techcrunch</w>
<w>transcoded</w>
<w>transcodings</w>
<w>unet</w>
<w>unpermitted</w>
<w>upserting</w>
<w>upserts</w>
Generated
+4095 -735
View File
File diff suppressed because it is too large Load Diff
+57 -3
View File
@@ -1,6 +1,8 @@
[workspace]
resolver = "2"
# This block defines the workspace members.
# When we add a library or binary crate, it needs an entry here.
members = [
# Types: common strongly-typed tokens, ids, payloads, etc.
"crates/_types/composite_identifiers",
@@ -22,7 +24,8 @@ members = [
"crates/cli/production/migration-tool",
# Desktop apps
"crates/desktop/src-tauri",
"crates/desktop/tauri-4o",
"crates/desktop/tauri-realtime",
# Shared libraries
"crates/lib/actix_cors_configs",
@@ -30,6 +33,7 @@ members = [
"crates/lib/api_clients/cloud_storage",
"crates/lib/api_clients/google_drive_common",
"crates/lib/api_clients/google_sign_in",
"crates/lib/api_clients/openai_sora_client",
"crates/lib/api_clients/thumbnail_generator",
"crates/lib/bootstrap",
"crates/lib/bucket_paths",
@@ -56,16 +60,20 @@ members = [
"crates/lib/files/mimetypes",
"crates/lib/files/videos",
"crates/lib/idempotency",
"crates/lib/image_algorithms",
"crates/lib/jobs_common",
"crates/lib/logging",
"crates/lib/machine_learning/ml_models",
"crates/lib/machine_learning/ml_weights_registry",
"crates/lib/machine_learning/prompt_engineering",
"crates/lib/markdown",
"crates/lib/migration",
"crates/lib/mysql_queries",
"crates/lib/password",
"crates/lib/prompt_engineering",
"crates/lib/redis_common",
"crates/lib/redis_schema",
"crates/lib/server_environment",
"crates/lib/shared_service_components",
"crates/lib/sqlx_mysql_helpers",
"crates/lib/storyteller_root",
"crates/lib/subprocess_common",
@@ -96,11 +104,57 @@ members = [
"crates/service/plugins/user_traits_component",
# Vendored code
"crates/vendor/candle/candle-core",
"crates/vendor/candle/candle-datasets",
"crates/vendor/candle/candle-examples",
"crates/vendor/candle/candle-flash-attn",
"crates/vendor/candle/candle-kernels",
"crates/vendor/candle/candle-metal-kernels",
"crates/vendor/candle/candle-nn",
"crates/vendor/candle/candle-onnx",
"crates/vendor/candle/candle-pyo3",
"crates/vendor/candle/candle-transformers",
"crates/vendor/candle/tensor-tools",
"crates/vendor/limitation",
# NB(bt,2023-12-12): Newrelic keeps breaking with upgrades. Kill for now.
#"crates/vendor/newrelic-telemetry-sdk-rust",
# cargo-hakari workspace hack (compilation speedup hack)
# see usage: https://crates.io/crates/cargo-hakari
"crates/my-workspace-hack",
"crates/workspace-build-acceleration",
]
[workspace.package]
# Common inheritable settings for monorepo packages
publish = false
[workspace.dependencies]
# Hakari workspace hack (faster compile times)
workspace-build-acceleration = { path = "crates/workspace-build-acceleration" }
# External libs
# Centrally pinned versions so that we can uniformly upgrade/downgrade versions across the workspace
# If a crate needs to use an older/newer version, the dependency should be managed in that crate's own Cargo.toml
anyhow = "1.0.97"
chrono = { version = "0.4.40", features = ["serde"] }
log = "0.4.27"
once_cell = "1.21.3"
rand = "0.9.0"
regex = "1.11.1"
reqwest = { version = "0.12.15", features = ["json", "cookies"] }
serde = { version = "1.0.219", features = ["derive"] }
serde_derive = "1.0.219"
serde_json = "1.0.140"
serde_yaml = "0.9.1"
toml = "0.7.4"
url = "2.5.4"
# Internal libs
crockford = { path = "crates/lib/data/crockford" }
errors = { path = "crates/lib/errors" }
filesys = { path = "crates/lib/files/filesys" }
memory_caching = { path = "crates/lib/caching/memory_caching" }
shared_service_components = { path = "crates/lib/shared_service_components" }
openai_sora_client = { path = "crates/lib/api_clients/openai_sora_client" }
primitives = { path = "crates/lib/data/primitives" }
subprocess_common = { path = "crates/lib/subprocess_common" }
+8
View File
@@ -0,0 +1,8 @@
image_width: 1024
image_height: 1024
#scheduler_steps: 15
# NB(bt): It looks like we can get down to 2 sampling steps, perhaps ?
scheduler_steps: 8
scheduler_samples: 1
cfg_scale: 15.0
seed: 42
@@ -13,7 +13,7 @@ name = "composite_identifiers"
path = "src/lib.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -21,7 +21,7 @@ enums = { path = "../enums" }
tokens = { path = "../tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../workspace-build-acceleration" }
# Serialization
serde = "1.0.125"
+2 -2
View File
@@ -13,13 +13,13 @@ name = "enums"
path = "src/lib.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../workspace-build-acceleration" }
# Serialization
serde = "1.0.125"
@@ -40,6 +40,10 @@ pub enum InferenceJobType {
#[serde(rename = "studio_gen2")]
StudioGen2,
/// Sora GPT 4o image gen
#[serde(rename = "image_gen_api")]
ImageGenApi,
/// A job that turns "FBX" game engine files into "GLTF" files (Bevy-compatible).
#[serde(rename = "convert_fbx_gltf")]
ConvertFbxToGltf,
@@ -108,6 +112,7 @@ impl InferenceJobType {
Self::GptSovits => "gpt_sovits",
Self::ComfyUi => "comfy_ui",
Self::StudioGen2 => "studio_gen2",
Self::ImageGenApi => "image_gen_api",
Self::ConvertFbxToGltf => "convert_fbx_gltf",
Self::MocapNet => "mocap_net",
Self::RvcV2 => "rvc_v2",
@@ -132,6 +137,7 @@ impl InferenceJobType {
"gpt_sovits" => Ok(Self::GptSovits),
"comfy_ui" => Ok(Self::ComfyUi),
"studio_gen2" => Ok(Self::StudioGen2),
"image_gen_api" => Ok(Self::ImageGenApi),
"convert_fbx_gltf" => Ok(Self::ConvertFbxToGltf),
"mocap_net" => Ok(Self::MocapNet),
"rvc_v2" => Ok(Self::RvcV2),
@@ -159,6 +165,7 @@ impl InferenceJobType {
Self::GptSovits,
Self::ComfyUi,
Self::StudioGen2,
Self::ImageGenApi,
Self::ConvertFbxToGltf,
Self::MocapNet,
Self::RvcV2,
@@ -197,6 +204,7 @@ mod tests {
assert_serialization(InferenceJobType::GptSovits, "gpt_sovits");
assert_serialization(InferenceJobType::ComfyUi, "comfy_ui");
assert_serialization(InferenceJobType::StudioGen2, "studio_gen2");
assert_serialization(InferenceJobType::ImageGenApi, "image_gen_api");
assert_serialization(InferenceJobType::ConvertFbxToGltf, "convert_fbx_gltf");
assert_serialization(InferenceJobType::MocapNet, "mocap_net");
assert_serialization(InferenceJobType::RvcV2, "rvc_v2");
@@ -220,6 +228,7 @@ mod tests {
assert_eq!(InferenceJobType::GptSovits.to_str(), "gpt_sovits");
assert_eq!(InferenceJobType::ComfyUi.to_str(), "comfy_ui");
assert_eq!(InferenceJobType::StudioGen2.to_str(), "studio_gen2");
assert_eq!(InferenceJobType::ImageGenApi.to_str(), "image_gen_api");
assert_eq!(InferenceJobType::ConvertFbxToGltf.to_str(), "convert_fbx_gltf");
assert_eq!(InferenceJobType::MocapNet.to_str(), "mocap_net");
assert_eq!(InferenceJobType::RvcV2.to_str(), "rvc_v2");
@@ -243,6 +252,7 @@ mod tests {
assert_eq!(InferenceJobType::from_str("gpt_sovits").unwrap(), InferenceJobType::GptSovits);
assert_eq!(InferenceJobType::from_str("comfy_ui").unwrap(), InferenceJobType::ComfyUi);
assert_eq!(InferenceJobType::from_str("studio_gen2").unwrap(), InferenceJobType::StudioGen2);
assert_eq!(InferenceJobType::from_str("image_gen_api").unwrap(), InferenceJobType::ImageGenApi);
assert_eq!(InferenceJobType::from_str("convert_fbx_gltf").unwrap(), InferenceJobType::ConvertFbxToGltf);
assert_eq!(InferenceJobType::from_str("mocap_net").unwrap(), InferenceJobType::MocapNet);
assert_eq!(InferenceJobType::from_str("rvc_v2").unwrap(), InferenceJobType::RvcV2);
@@ -261,7 +271,7 @@ mod tests {
fn all_variants() {
// Static check
let mut variants = InferenceJobType::all_variants();
assert_eq!(variants.len(), 16);
assert_eq!(variants.len(), 17);
assert_eq!(variants.pop_first(), Some(InferenceJobType::VideoRender));
assert_eq!(variants.pop_first(), Some(InferenceJobType::LivePortrait));
assert_eq!(variants.pop_first(), Some(InferenceJobType::FaceFusion));
@@ -269,6 +279,7 @@ mod tests {
assert_eq!(variants.pop_first(), Some(InferenceJobType::GptSovits));
assert_eq!(variants.pop_first(), Some(InferenceJobType::ComfyUi));
assert_eq!(variants.pop_first(), Some(InferenceJobType::StudioGen2));
assert_eq!(variants.pop_first(), Some(InferenceJobType::ImageGenApi));
assert_eq!(variants.pop_first(), Some(InferenceJobType::ConvertFbxToGltf));
assert_eq!(variants.pop_first(), Some(InferenceJobType::MocapNet));
assert_eq!(variants.pop_first(), Some(InferenceJobType::RvcV2));
@@ -33,6 +33,10 @@ pub enum InferenceModelType {
#[serde(rename = "seed_vc")]
SeedVc,
/// NB: This is for Sora GPT 4o image gen
#[serde(rename = "image_gen_api")]
ImageGenApi,
#[serde(rename = "tacotron2")]
Tacotron2,
#[serde(rename = "vits")]
@@ -70,6 +74,7 @@ impl InferenceModelType {
Self::VallEX => "vall_e_x",
Self::RerenderAVideo => "rerender_a_video",
Self::StableDiffusion => "stable_diffusion",
Self::ImageGenApi => "image_gen_api",
Self::SeedVc => "seed_vc",
Self::MocapNet => "mocap_net",
Self::StyleTTS2 => "styletts2",
@@ -90,6 +95,7 @@ impl InferenceModelType {
"vall_e_x" => Ok(Self::VallEX),
"rerender_a_video" => Ok(Self::RerenderAVideo),
"stable_diffusion" => Ok(Self::StableDiffusion),
"image_gen_api" => Ok(Self::ImageGenApi),
"mocap_net" => Ok(Self::MocapNet),
"styletts2" => Ok(Self::StyleTTS2),
"comfy_ui" => Ok(Self::ComfyUi),
@@ -113,6 +119,7 @@ impl InferenceModelType {
Self::VallEX,
Self::RerenderAVideo,
Self::StableDiffusion,
Self::ImageGenApi,
Self::MocapNet,
Self::StyleTTS2,
Self::ConvertFbxToGltf,
@@ -140,6 +147,7 @@ mod tests {
assert_serialization(InferenceModelType::VallEX, "vall_e_x");
assert_serialization(InferenceModelType::RerenderAVideo, "rerender_a_video");
assert_serialization(InferenceModelType::StableDiffusion, "stable_diffusion");
assert_serialization(InferenceModelType::ImageGenApi, "image_gen_api");
assert_serialization(InferenceModelType::MocapNet, "mocap_net");
assert_serialization(InferenceModelType::ComfyUi, "comfy_ui");
assert_serialization(InferenceModelType::StyleTTS2, "styletts2");
@@ -159,6 +167,7 @@ mod tests {
assert_eq!(InferenceModelType::VallEX.to_str(), "vall_e_x");
assert_eq!(InferenceModelType::RerenderAVideo.to_str(), "rerender_a_video");
assert_eq!(InferenceModelType::StableDiffusion.to_str(), "stable_diffusion");
assert_eq!(InferenceModelType::ImageGenApi.to_str(), "image_gen_api");
assert_eq!(InferenceModelType::MocapNet.to_str(), "mocap_net");
assert_eq!(InferenceModelType::StyleTTS2.to_str(), "styletts2");
assert_eq!(InferenceModelType::ConvertFbxToGltf.to_str(), "convert_fbx_gltf");
@@ -176,6 +185,7 @@ mod tests {
assert_eq!(InferenceModelType::from_str("vall_e_x").unwrap(), InferenceModelType::VallEX);
assert_eq!(InferenceModelType::from_str("rerender_a_video").unwrap(), InferenceModelType::RerenderAVideo);
assert_eq!(InferenceModelType::from_str("stable_diffusion").unwrap(), InferenceModelType::StableDiffusion);
assert_eq!(InferenceModelType::from_str("image_gen_api").unwrap(), InferenceModelType::ImageGenApi);
assert_eq!(InferenceModelType::from_str("mocap_net").unwrap(), InferenceModelType::MocapNet);
assert_eq!(InferenceModelType::from_str("styletts2").unwrap(), InferenceModelType::StyleTTS2);
assert_eq!(InferenceModelType::from_str("comfy_ui").unwrap(), InferenceModelType::ComfyUi);
@@ -187,17 +197,18 @@ mod tests {
fn all_variants() {
// Static check
let mut variants = InferenceModelType::all_variants();
assert_eq!(variants.len(), 13);
assert_eq!(variants.len(), 15);
assert_eq!(variants.pop_first(), Some(InferenceModelType::ComfyUi));
assert_eq!(variants.pop_first(), Some(InferenceModelType::RvcV2));
assert_eq!(variants.pop_first(), Some(InferenceModelType::SadTalker));
assert_eq!(variants.pop_first(), Some(InferenceModelType::SeedVc));
assert_eq!(variants.pop_first(), Some(InferenceModelType::SoVitsSvc));
assert_eq!(variants.pop_first(), Some(InferenceModelType::SeedVc));
assert_eq!(variants.pop_first(), Some(InferenceModelType::Tacotron2));
assert_eq!(variants.pop_first(), Some(InferenceModelType::Vits));
assert_eq!(variants.pop_first(), Some(InferenceModelType::VallEX));
assert_eq!(variants.pop_first(), Some(InferenceModelType::RerenderAVideo));
assert_eq!(variants.pop_first(), Some(InferenceModelType::StableDiffusion));
assert_eq!(variants.pop_first(), Some(InferenceModelType::ImageGenApi));
assert_eq!(variants.pop_first(), Some(InferenceModelType::MocapNet));
assert_eq!(variants.pop_first(), Some(InferenceModelType::StyleTTS2));
assert_eq!(variants.pop_first(), Some(InferenceModelType::ConvertFbxToGltf));
@@ -57,6 +57,9 @@ pub enum MediaFileOriginModelType {
#[serde(rename = "studio")]
StorytellerStudio,
#[serde(rename = "studio_ig")]
StorytellerStudioImageGen,
#[serde(rename = "vst")]
VideoStyleTransfer,
@@ -93,6 +96,7 @@ impl MediaFileOriginModelType {
Self::MocapNet => "mocap_net",
Self::StyleTTS2 => "styletts2",
Self::StableDiffusion15 => "stable_diffusion_1_5",
Self::StorytellerStudioImageGen => "studio_ig",
Self::GptSovits => "gpt_sovits",
Self::StorytellerStudio => "studio",
Self::VideoStyleTransfer => "vst",
@@ -115,6 +119,7 @@ impl MediaFileOriginModelType {
"mocap_net" => Ok(Self::MocapNet),
"styletts2" => Ok(Self::StyleTTS2),
"stable_diffusion_1_5" => Ok(Self::StableDiffusion15),
"studio_ig" => Ok(Self::StorytellerStudioImageGen),
"gpt_sovits" => Ok(Self::GptSovits),
"studio" => Ok(Self::StorytellerStudio),
"vst" => Ok(Self::VideoStyleTransfer),
@@ -142,6 +147,7 @@ impl MediaFileOriginModelType {
Self::StableDiffusion15,
Self::GptSovits,
Self::StorytellerStudio,
Self::StorytellerStudioImageGen,
Self::VideoStyleTransfer,
Self::ComfyUi,
Self::VallEX,
+2 -2
View File
@@ -13,11 +13,11 @@ name = "enums_public"
path = "src/lib.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../workspace-build-acceleration" }
# Internal
enums = { path = "../enums" }
@@ -91,6 +91,10 @@ pub enum PublicInferenceJobType {
/// Re-render a video is a video style transfer algorithm. We developed code
/// around it, but chose to develop AnimateDiff / ComfyUI support instead.
RerenderAVideo,
/// Sora GPT 4o image gen
#[serde(rename = "image_gen_api")]
ImageGenApi,
}
/// NB: Legacy API for older code.
@@ -119,6 +123,7 @@ impl PublicInferenceJobType {
InferenceJobType::Unknown => Self::Unknown,
InferenceJobType::BevyToWorkflow => Self::BevyToWorkflow,
InferenceJobType::RerenderAVideo => Self::RerenderAVideo,
InferenceJobType::ImageGenApi => Self::ImageGenApi,
}
}
@@ -140,6 +145,7 @@ impl PublicInferenceJobType {
Self::SeedVc => InferenceJobType::SeedVc,
Self::SoVitsSvc => InferenceJobType::SoVitsSvc,
Self::StableDiffusion => InferenceJobType::StableDiffusion,
Self::ImageGenApi => InferenceJobType::ImageGenApi,
Self::StyleTTS2 => InferenceJobType::StyleTTS2,
Self::Tacotron2 => InferenceJobType::Tacotron2,
Self::Unknown => InferenceJobType::Unknown,
@@ -64,6 +64,10 @@ pub enum PublicMediaFileModelType {
#[serde(rename = "studio")]
StorytellerStudio,
/// NB: This is GPT4o image generation
#[serde(rename = "studio_ig")]
StorytellerStudioImageGen,
#[serde(rename = "vst")]
VideoStyleTransfer,
@@ -89,6 +93,7 @@ impl PublicMediaFileModelType {
MediaFileOriginModelType::LivePortrait => Self::FaceMirror,
MediaFileOriginModelType::SadTalker => Self::FaceAnimator,
MediaFileOriginModelType::StyleTTS2 => Self::VoiceDesigner,
MediaFileOriginModelType::StorytellerStudioImageGen => Self::StorytellerStudioImageGen,
// Conserved variants
MediaFileOriginModelType::RvcV2 => Self::RvcV2,
MediaFileOriginModelType::SoVitsSvc => Self::SoVitsSvc,
@@ -113,6 +118,7 @@ impl PublicMediaFileModelType {
Self::FaceAnimator => MediaFileOriginModelType::SadTalker,
Self::Lipsync => MediaFileOriginModelType::FaceFusion,
Self::VoiceDesigner => MediaFileOriginModelType::StyleTTS2,
Self::StorytellerStudioImageGen => MediaFileOriginModelType::StorytellerStudioImageGen,
// Conserved variants
Self::RvcV2 => MediaFileOriginModelType::RvcV2,
Self::SoVitsSvc => MediaFileOriginModelType::SoVitsSvc,
+2 -2
View File
@@ -13,7 +13,7 @@ name = "reusable_types"
path = "src/lib.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
@@ -21,7 +21,7 @@ normal = ["my-workspace-hack"]
# (none)
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../workspace-build-acceleration" }
# External
chrono = { version = "0.4.22", features = ["serde"] }
+2 -2
View File
@@ -13,7 +13,7 @@ name = "tokens"
path = "src/lib.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
@@ -21,7 +21,7 @@ normal = ["my-workspace-hack"]
# (none)
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../workspace-build-acceleration" }
# Serialization
serde = "1.0.125"
@@ -13,7 +13,7 @@ name = "dev-database-seed"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -32,7 +32,7 @@ storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
log = "0.4.14"
@@ -13,7 +13,7 @@ name = "dev-upload-media-file"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -29,7 +29,7 @@ mysql_queries = { path = "../../../lib/mysql_queries" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
+2 -2
View File
@@ -13,7 +13,7 @@ name = "scrape-test"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
@@ -22,7 +22,7 @@ errors = { path = "../../../lib/errors" }
web_scrapers = { path = "../../../lib/web_scrapers" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
+2 -2
View File
@@ -13,11 +13,11 @@ name = "db-backfill"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# Internal
datetimes = { path = "../../../lib/data/datetimes" }
+2 -2
View File
@@ -13,7 +13,7 @@ name = "db-backup"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -21,7 +21,7 @@ easyenv = { path = "../../../lib/easyenv" }
errors = { path = "../../../lib/errors" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
anyhow = "1.0.86"
+2 -2
View File
@@ -13,7 +13,7 @@ name = "db-cleanup"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -26,7 +26,7 @@ storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
@@ -13,7 +13,7 @@ name = "elasticsearch-cli"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -29,7 +29,7 @@ storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
@@ -13,7 +13,7 @@ name = "kubectl-pod-cleanup"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -22,7 +22,7 @@ easyenv = { path = "../../../lib/easyenv" }
errors = { path = "../../../lib/errors" }
# Workspace hack (faster compile yytimes)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
log = "0.4.20"
@@ -13,7 +13,7 @@ name = "lru-delete-models"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -21,7 +21,7 @@ errors = { path = "../../../lib/errors" }
logging = { path = "../../../lib/logging" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = { version = "4.4.8", features = ["derive"] }
@@ -13,7 +13,7 @@ name = "migration-tool"
path = "src/main.rs"
[package.metadata.cargo-udeps.ignore]
normal = ["my-workspace-hack"]
normal = ["workspace-build-acceleration"]
[dependencies]
# Internal
@@ -30,7 +30,7 @@ storyteller_root = { path = "../../../lib/storyteller_root" }
tokens = { path = "../../../_types/tokens" }
# Workspace hack (faster compile times)
my-workspace-hack = { version = "0.1", path = "../../../my-workspace-hack" }
workspace-build-acceleration = { version = "0.1", path = "../../../workspace-build-acceleration" }
# External
clap = "4.4.8"
-28
View File
@@ -1,28 +0,0 @@
[package]
name = "app"
version = "0.1.0"
description = "A Tauri App"
authors = ["you"]
license = ""
repository = ""
edition = "2021"
rust-version = "1.77.2"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
name = "app_lib"
crate-type = ["staticlib", "cdylib", "rlib"]
[build-dependencies]
tauri-build = { version = "2.0.4", features = [] }
[dependencies]
serde_json = "1.0"
serde = { version = "1.0", features = ["derive"] }
log = "0.4"
tauri = { version = "2.2.4", features = [] }
tauri-plugin-log = "2.0.0-rc"
once_cell = "1.20.3"
image = "0.25.5"
base64 = "0.22.1"
-55
View File
@@ -1,55 +0,0 @@
mod image_endpoint;
use std::sync::{Arc, RwLock};
use once_cell::sync::Lazy;
use image_endpoint::infer_image;
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
tauri::Builder::default()
.setup(|app| {
if cfg!(debug_assertions) {
app.handle().plugin(
tauri_plugin_log::Builder::default()
.level(log::LevelFilter::Info)
.build(),
)?;
}
Ok(())
})
.invoke_handler(tauri::generate_handler![
my_custom_command,
test_round_trip,
infer_image,
])
.run(tauri::generate_context!())
.expect("error while running tauri application");
}
#[tauri::command]
fn my_custom_command() {
println!("I was invoked from JavaScript!");
}
static COUNTER : Lazy<Arc<RwLock<u64>>> = Lazy::new(|| Arc::new(RwLock::new(0)));
#[tauri::command]
fn test_round_trip() -> u64 {
let value : u64;
{
match COUNTER.write() {
Ok(mut counter) => {
*counter += 1;
value = *counter;
},
Err(_e) => {
value = 0;
}
}
}
println!("I was invoked from JavaScript! {:?}", value);
value
}
+4
View File
@@ -0,0 +1,4 @@
# Don't restart the dev server on Rust changes
crates/
Cargo.toml
Cargo.lock
+74
View File
@@ -0,0 +1,74 @@
[package]
name = "artcraft"
version = "0.1.0"
description = "Artcraft"
authors = ["you"]
license = ""
repository = ""
edition = "2021"
rust-version = "1.77.2"
publish = false
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
name = "app_lib"
crate-type = ["staticlib", "cdylib", "rlib"]
[build-dependencies]
tauri-build = { version = "2.1.1", features = [] }
[dependencies]
# Workspace hack (faster compile times)
workspace-build-acceleration = { workspace = true }
# Internal
crockford = { workspace = true }
errors = { workspace = true }
filesys = { workspace = true }
memory_caching = { workspace = true }
openai_sora_client = { workspace = true }
primitives = { workspace = true }
subprocess_common = { workspace = true }
# External
#openssl = { version = "0.10", features = ["vendored"] }
# NB(bt,2025-03-10): default-features = false on Windows while I debug openssl / rustls
anyhow = { workspace = true }
base64 = "0.22.1"
bytes = "1.10"
chrono = { workspace = true }
const_format = "0.2.34"
directories = "6.0.0"
dirs = "6.0.0"
futures = "0.3"
image = "0.25.5"
log = { workspace = true }
once_cell = { workspace = true }
rand = { workspace = true }
reqwest = { workspace = true, features = ["multipart", "stream"], default-features = false }
serde = { workspace = true }
serde_derive = { workspace = true }
serde_json = { workspace = true }
serde_yaml = { workspace = true }
tauri = { version = "2.4.1", features = ["unstable"] }
tauri-plugin-log = "2.3.1"
tempdir = "0.3.7"
tempfile = "3.18.0"
tokio = { version = "1.42", features = ["rt", "rt-multi-thread", "fs"] }
tokio-util = { version = "0.7", features = ["codec"] }
[dev-dependencies]
libloading = "0.8.6"
serial_test = "3.2.0"
assertor = "0.0.3"
[profile.dev]
incremental = true # Compile your binary in smaller steps.
[profile.release]
codegen-units = 1 # Allows LLVM to perform better optimization.
lto = true # Enables link-time-optimizations.
opt-level = 3 # `s` prioritizes small binary size. Use `3` if you prefer speed.
panic = "abort" # Higher performance by disabling panic handlers.
strip = true # Ensures debug symbols are removed.
@@ -6,6 +6,7 @@
"main"
],
"permissions": [
"core:default"
"core:default",
"log:default"
]
}
}

Before

Width:  |  Height:  |  Size: 11 KiB

After

Width:  |  Height:  |  Size: 11 KiB

Before

Width:  |  Height:  |  Size: 23 KiB

After

Width:  |  Height:  |  Size: 23 KiB

Before

Width:  |  Height:  |  Size: 2.2 KiB

After

Width:  |  Height:  |  Size: 2.2 KiB

Before

Width:  |  Height:  |  Size: 9.0 KiB

After

Width:  |  Height:  |  Size: 9.0 KiB

Before

Width:  |  Height:  |  Size: 12 KiB

After

Width:  |  Height:  |  Size: 12 KiB

Before

Width:  |  Height:  |  Size: 13 KiB

After

Width:  |  Height:  |  Size: 13 KiB

Before

Width:  |  Height:  |  Size: 25 KiB

After

Width:  |  Height:  |  Size: 25 KiB

Before

Width:  |  Height:  |  Size: 2.0 KiB

After

Width:  |  Height:  |  Size: 2.0 KiB

Before

Width:  |  Height:  |  Size: 28 KiB

After

Width:  |  Height:  |  Size: 28 KiB

Before

Width:  |  Height:  |  Size: 3.3 KiB

After

Width:  |  Height:  |  Size: 3.3 KiB

Before

Width:  |  Height:  |  Size: 5.9 KiB

After

Width:  |  Height:  |  Size: 5.9 KiB

Before

Width:  |  Height:  |  Size: 7.4 KiB

After

Width:  |  Height:  |  Size: 7.4 KiB

Before

Width:  |  Height:  |  Size: 3.9 KiB

After

Width:  |  Height:  |  Size: 3.9 KiB

Before

Width:  |  Height:  |  Size: 37 KiB

After

Width:  |  Height:  |  Size: 37 KiB

Before

Width:  |  Height:  |  Size: 49 KiB

After

Width:  |  Height:  |  Size: 49 KiB

@@ -1,11 +1,11 @@
use std::io::Cursor;
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use base64::prelude::{BASE64_STANDARD, BASE64_URL_SAFE};
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::ImageReader;
use std::io::Cursor;
#[tauri::command]
pub fn infer_image(image: &str) -> Result<String, String> {
pub fn flip_image(image: &str) -> Result<String, String> {
println!("infer_image called; processing image...");
let bytes = BASE64_STANDARD.decode(image)
@@ -18,17 +18,19 @@ pub fn infer_image(image: &str) -> Result<String, String> {
.map_err(|err| format!("Image decode error: {}", err))?;
let rotated_image = image.rotate180();
let mut buffer = Vec::new(); // TODO: Preallocate, better data structure
let mut writer = Cursor::new(&mut buffer);
let encoder = PngEncoder::new_with_quality
(&mut writer, CompressionType::Fast, FilterType::Adaptive);
let encoder = PngEncoder::new_with_quality(&mut writer, CompressionType::Fast, FilterType::Adaptive);
rotated_image.write_with_encoder(encoder)
.map_err(|err| format!("Image encoding error: {}", err))?;
let encoded = BASE64_STANDARD.encode(&buffer);
println!("Encoded: {:?}", encoded.split_at(10).0);
Ok(encoded)
}
@@ -0,0 +1,90 @@
use std::fs::read_to_string;
use base64::prelude::BASE64_STANDARD;
use errors::AnyhowResult;
use base64::Engine;
use image::codecs::png::{CompressionType, FilterType, PngEncoder};
use image::{DynamicImage, ImageReader};
use std::io::Cursor;
use log::{error, info};
use tauri::{AppHandle, Manager, State};
use openai_sora_client::credentials::SoraCredentials;
use openai_sora_client::image_gen::common::{ImageSize, NumImages};
use openai_sora_client::image_gen::sora_image_gen_remix::{sora_image_gen_remix, SoraImageGenRemixRequest};
use openai_sora_client::upload::upload_media_from_bytes::sora_media_upload_from_bytes;
use openai_sora_client::upload::upload_media_from_file::SoraMediaUploadRequest;
use crate::state::app_dir::AppDataRoot;
#[tauri::command]
pub async fn image_generation_command(
image: &str,
prompt: &str,
app: AppHandle,
app_data_root: State<'_, AppDataRoot>,
) -> Result<String, String> {
info!("image_generation_command called; processing image...");
let bytes = BASE64_STANDARD.decode(image)
.map_err(|err| format!("Base64 decode error: {}", err))?;
generate_image(bytes, prompt, &app_data_root)
.await
.map_err(|err| {
error!("error: {:?}", err);
"there was an error".to_string()
})?;
Ok("success".to_string())
}
pub async fn generate_image(file_bytes: Vec<u8>, prompt: &str, app_data_root: &AppDataRoot) -> AnyhowResult<()> {
let sora_credentials = get_credentials(app_data_root)?;
let filename = "image.png".to_string();
let response= sora_media_upload_from_bytes(file_bytes, filename, &sora_credentials)
.await
.map_err(|err| {
error!("Failed to upload image to Sora: {:?}", err);
err
})?;
let sora_media_tokens = vec![response.id];
let response = sora_image_gen_remix(SoraImageGenRemixRequest {
prompt: prompt.to_string(),
num_images: NumImages::One,
image_size: ImageSize::Square,
sora_media_tokens: sora_media_tokens.clone(),
credentials: &sora_credentials,
}).await
.map_err(|err| {
error!("Failed to call Sora image generation: {:?}", err);
err
})?;
println!(">> TASK ID: {:?} ", response.task_id);
Ok(())
}
fn get_credentials(app_data_root: &AppDataRoot) -> AnyhowResult<SoraCredentials> {
let cookie_file = app_data_root.get_sora_cookie_file_path();
let bearer = read_to_string("/Users/bt/dev/storyteller/storyteller-rust/test_data/temp/bearer.txt")?
.trim()
.to_string();
let cookie= read_to_string(cookie_file)?
.trim()
.to_string();
let sentinel = read_to_string("/Users/bt/dev/storyteller/storyteller-rust/test_data/temp/sentinel.txt")?
.trim()
.to_string();
Ok(SoraCredentials {
bearer_token: bearer,
cookie: cookie,
sentinel: Some(sentinel),
})
}
@@ -0,0 +1,3 @@
pub mod flip_image;
pub mod image_generation_command;
pub mod open_login_command;
@@ -0,0 +1,72 @@
use std::time::Duration;
use log::{error, info};
use once_cell::sync::Lazy;
use tauri::{AppHandle, Manager, Url, WebviewUrl, WebviewWindowBuilder, WindowBuilder};
use tokio::time::sleep_until;
use errors::AnyhowResult;
use crate::threads::login_thread::{LOGIN_WINDOW_NAME, SORA_LOGIN_URL};
use crate::utils::clear_all_webview_cookies::clear_all_webview_cookies;
pub const START_URL_STR: &str = "https://storyteller.ai/";
pub static START_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(START_URL_STR).expect("URL should parse")
});
pub const SORA_HOMEPAGE_URL_STR: &str = "https://sora.com/";
pub static SORA_HOMEPAGE_URL : Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_HOMEPAGE_URL_STR).expect("URL should parse")
});
#[tauri::command]
pub async fn open_login_command(
app: AppHandle,
) -> Result<String, String> {
info!("open_login_command called");
do_open_login(&app)
.await
.map_err(|err| {
error!("Error opening login: {:?}", err);
format!("Error opening login: {:?}", err)
})?;
Ok("result".to_string())
}
async fn do_open_login(app: &AppHandle) -> AnyhowResult<()> {
info!("Building login window...");
let url = WebviewUrl::External(START_URL.clone());
let window = WebviewWindowBuilder::new(app, LOGIN_WINDOW_NAME, url)
//.user_agent(openai_sora_client::credentials::USER_AGENT)
.always_on_top(true)
.title("Login to OpenAI")
.center()
.resizable(true)
.visible(true)
.closable(true)
.focused(true)
.devtools(true)
.build()?;
for (name, webview) in window.webviews() {
if name != LOGIN_WINDOW_NAME {
continue;
}
clear_all_webview_cookies(&webview)?;
// NB: We're starting to get Cloudflare protection screens. Let's try to avoid.
webview.navigate(SORA_HOMEPAGE_URL.clone())?;
tokio::time::sleep(Duration::from_millis(100)).await;
webview.navigate(SORA_LOGIN_URL.clone())?;
break;
}
info!("Done.");
Ok(())
}
@@ -0,0 +1 @@
pub mod notification_event;
@@ -0,0 +1,45 @@
use serde::Serialize;
#[derive(Clone, Copy, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum NotificationModelType {
ImageSegmentation,
TextEncoder,
TextTokenizer,
Unet,
Vae,
Json,
ClipEncoder,
FluxModel,
}
#[derive(Clone, Serialize)]
#[serde(rename_all = "snake_case")]
pub enum NotificationEvent<'a> {
ModelDownloadStarted {
model_name: &'a str,
model_type: NotificationModelType,
},
ModelDownloadProgress {
model_name: &'a str,
model_type: NotificationModelType,
currently_downloaded_bytes: usize,
total_file_bytes: usize,
},
ModelDownloadComplete {
model_name: &'a str,
model_type: NotificationModelType,
},
ModelLoading {
model_name: String,
model_type: NotificationModelType,
},
ModelInference {
model_name: String,
model_type: NotificationModelType,
},
ModelInferenceComplete {
model_name: String,
model_type: NotificationModelType,
},
}
+65
View File
@@ -0,0 +1,65 @@
pub mod commands;
pub mod events;
pub mod state;
pub mod threads;
pub mod transfer;
pub mod utils;
use crate::commands::flip_image::flip_image;
use crate::commands::image_generation_command::image_generation_command;
use crate::commands::open_login_command::open_login_command;
use crate::state::app_config::AppConfig;
use crate::threads::login_thread::login_thread;
use tauri_plugin_log::Target;
use tauri_plugin_log::TargetKind;
#[cfg_attr(mobile, tauri::mobile_entry_point)]
pub fn run() {
// NB: Tauri wants to install the logger itself, so we can't rely on the logger crate
// until the tauri runtime begins.
println!("Loading model config...");
let config = AppConfig::init()
.expect("config should load");
let app_data_root = config.app_data_root.clone();
let app_data_root2 = config.app_data_root.clone();
println!("Initializing backend runtime...");
tauri::Builder::default()
.plugin(tauri_plugin_log::Builder::new()
.level(log::LevelFilter::Info)
.targets(vec![
Target::new(TargetKind::Stdout),
Target::new(TargetKind::LogDir { file_name: Some(app_data_root.log_file_name_str().to_string()) }),
])
.build())
.setup(|app| {
// TODO(bt): This is broken on windows
// log_environment_details();
//if cfg!(debug_assertions) {
// app.handle().plugin(
// tauri_plugin_log::Builder::default()
// .level(log::LevelFilter::Info)
// .build(),
// )?;
//}
let app = app.handle().clone();
tauri::async_runtime::spawn(login_thread(app, app_data_root2));
Ok(())
})
.manage(config)
.manage(app_data_root)
.invoke_handler(tauri::generate_handler![
flip_image,
image_generation_command,
open_login_command,
])
.run(tauri::generate_context!("tauri.conf.json"))
.expect("error while running tauri application");
}
@@ -0,0 +1,58 @@
use crate::state::app_dir::AppDataRoot;
use crate::state::os_platform::OsPlatform;
use crate::state::yaml_config::YamlConfig;
const DEFAULT_SD_IMAGE_WIDTH: usize = 1024;
const DEFAULT_SD_IMAGE_HEIGHT: usize = 1024;
/// A central place to configure app-wide details.
pub struct AppConfig {
pub image_height: usize,
pub image_width: usize,
pub scheduler_steps: usize,
pub scheduler_samples: usize,
pub seed: Option<u64>,
pub cfg_scale: Option<f64>,
/// Root location for application data.
pub app_data_root: AppDataRoot,
}
impl AppConfig {
pub fn init() -> anyhow::Result<Self> {
let yaml_configs = YamlConfig::load_from_config_file_recursive();
println!("Configs: {:?}", yaml_configs);
let os_platform = OsPlatform::get();
let default_data_root = match os_platform {
OsPlatform::Linux => yaml_configs.linux_default_data_path.as_deref(),
OsPlatform::MacOs => yaml_configs.mac_default_data_path.as_deref(),
OsPlatform::Windows => yaml_configs.windows_default_data_path.as_deref(),
};
println!("Possible user-defined data root: {:?}", default_data_root);
let app_data_root = match default_data_root {
None => AppDataRoot::create_default()?,
Some(path) => AppDataRoot::create_existing(path)?,
};
println!("App data root: {:?}", app_data_root.path());
Ok(Self {
image_height: yaml_configs.image_height.unwrap_or(DEFAULT_SD_IMAGE_HEIGHT),
image_width: yaml_configs.image_width.unwrap_or(DEFAULT_SD_IMAGE_WIDTH),
scheduler_steps: yaml_configs.scheduler_steps.unwrap_or(4),
scheduler_samples: yaml_configs.scheduler_samples.unwrap_or(15),
seed: yaml_configs.seed,
cfg_scale: yaml_configs.cfg_scale,
app_data_root,
})
}
}
@@ -0,0 +1,219 @@
use crate::state::expanduser::expanduser;
use crate::state::os_platform::OsPlatform;
use anyhow::anyhow;
use directories::UserDirs;
use std::path::{Path, PathBuf};
use chrono::{DateTime, Local};
use tempdir::TempDir;
use tempfile::{Builder, NamedTempFile};
const DEFAULT_DATA_DIR : &str = "artcraft";
const ASSETS_SUBDIRECTORY : &str = "assets";
const WEIGHTS_SUBDIRECTORY : &str = "weights";
const TEMPORARY_SUBDIRECTORY : &str = "temp";
/// Note: Tauri appends ".log" to the end of the filename.
const LOG_FILE_NAME : &str = "application_debug";
/// The path to the application data directory, which includes "asset" and "weights" data.
#[derive(Clone)]
pub struct AppDataRoot {
path: PathBuf,
assets_dir: AppAssetsDir,
weights_dir: AppWeightsDir,
temp_dir: TemporaryDir,
log_file_name: PathBuf,
log_file_name_string: String,
}
#[derive(Clone)]
pub struct AppAssetsDir {
path: PathBuf,
}
#[derive(Clone)]
pub struct AppWeightsDir {
path: PathBuf,
}
#[derive(Clone)]
pub struct TemporaryDir {
path: PathBuf,
}
impl AppDataRoot {
pub fn create_default() -> anyhow::Result<Self> {
let directory = get_default_data_dir()?;
Self::create_existing(directory)
}
pub fn create_existing<P: AsRef<Path>>(dir: P) -> anyhow::Result<Self> {
let mut dir = dir.as_ref().to_path_buf();
match OsPlatform::get() {
OsPlatform::Linux | OsPlatform::MacOs => {
if let Some(d) = dir.as_os_str().to_str() {
dir = expanduser(d)?;
}
},
OsPlatform::Windows => {}
}
if !dir.is_dir() {
println!("Creating directory {:?}", dir);
std::fs::create_dir_all(&dir)?;
}
match dir.canonicalize() {
Ok(d) => dir = d,
Err(err) => {
println!("Error canonicalizing {:?}: {}", dir, err);
}
}
let assets_dir = AppAssetsDir::create_existing(dir.join(ASSETS_SUBDIRECTORY))?;
let weights_dir = AppWeightsDir::create_existing(dir.join(WEIGHTS_SUBDIRECTORY))?;
let temp_dir = TemporaryDir::create_existing(dir.join(TEMPORARY_SUBDIRECTORY))?;
let log_file_name = dir.join(LOG_FILE_NAME);
let log_file_name_string = log_file_name
.to_str()
.ok_or(anyhow!("couldn't convert log path to str"))?
.to_string();
Ok(Self {
path: dir,
assets_dir,
weights_dir,
temp_dir,
log_file_name,
log_file_name_string,
})
}
pub fn assets_dir(&self) -> &AppAssetsDir {
&self.assets_dir
}
pub fn weights_dir(&self) -> &AppWeightsDir {
&self.weights_dir
}
pub fn temp_dir(&self) -> &TemporaryDir {
&self.temp_dir
}
pub fn path(&self) -> &Path {
&self.path
}
pub fn log_file_name(&self) -> &Path {
&self.log_file_name
}
pub fn log_file_name_str(&self) -> &str {
&self.log_file_name_string
}
pub fn get_sora_cookie_file_path(&self) -> PathBuf {
self.path.join("sora_cookies.txt")
}
}
impl AppAssetsDir {
pub fn create_existing<P: AsRef<Path>>(dir: P) -> anyhow::Result<Self> {
let mut dir = dir.as_ref().to_path_buf();
match dir.canonicalize() {
Ok(d) => dir = d,
Err(err) => {
println!("Error canonicalizing {:?}: {}", dir, err);
}
}
if !dir.exists() {
println!("Creating directory {:?}", dir);
std::fs::create_dir(&dir)?;
}
Ok(Self {
path: dir,
})
}
pub fn path(&self) -> &Path {
&self.path
}
/// Return the current date asset directory
/// If it doesn't exist, create it.
pub fn make_or_get_current_date_dir(&self) -> anyhow::Result<PathBuf> {
let format: DateTime<Local> = Local::now();
let date_directory_name = format.format("%Y-%d-%m").to_string();
let full_path = self.path.join(date_directory_name);
if !full_path.exists() {
std::fs::create_dir(&full_path)?;
}
Ok(full_path)
}
}
impl AppWeightsDir {
pub fn create_existing<P: AsRef<Path>>(dir: P) -> anyhow::Result<Self> {
let mut dir = dir.as_ref().to_path_buf();
match dir.canonicalize() {
Ok(d) => dir = d,
Err(err) => {
println!("Error canonicalizing {:?}: {}", dir, err);
}
}
if !dir.exists() {
println!("Creating directory {:?}", dir);
std::fs::create_dir(&dir)?;
}
Ok(Self {
path: dir,
})
}
pub fn path(&self) -> &Path {
&self.path
}
}
impl TemporaryDir {
pub fn create_existing<P: AsRef<Path>>(dir: P) -> anyhow::Result<Self> {
let mut dir = dir.as_ref().to_path_buf();
match dir.canonicalize() {
Ok(d) => dir = d,
Err(err) => {
println!("Error canonicalizing {:?}: {}", dir, err);
}
}
if !dir.exists() {
println!("Creating directory {:?}", dir);
std::fs::create_dir(&dir)?;
}
Ok(Self {
path: dir,
})
}
pub fn with_prefix(&self, prefix: &str) -> anyhow::Result<TempDir> {
let tempdir = TempDir::new_in(&self.path, prefix)?;
Ok(tempdir)
}
pub fn new_named_temp_file(&self) -> anyhow::Result<NamedTempFile> {
let tempfile = Builder::new()
.prefix("temp_")
.suffix(".bin")
.tempfile_in(&self.path)?;
Ok(tempfile)
}
}
// eg. /home/bob/artcraft, /Users/bob/artcraft, or C:\Users\Alice\artcraft
fn get_default_data_dir() -> anyhow::Result<PathBuf> {
Ok(UserDirs::new()
.ok_or_else(|| anyhow!("could not determine user home directory"))?
.home_dir()
.join(DEFAULT_DATA_DIR))
}
@@ -0,0 +1,73 @@
use once_cell::sync::Lazy;
use std::io;
use std::path::{PathBuf, MAIN_SEPARATOR};
static PREFIX: Lazy<String> = Lazy::new(|| format!("~{}", MAIN_SEPARATOR));
/// The "expanduser" crate doesn't compile on Windows, so we replace its functionality slightly
pub fn expanduser<P: AsRef<str>>(path: P) -> io::Result<PathBuf> {
Ok(match path.as_ref() {
// matches an exact "~"
s if s == "~" => {
home_dir()?
},
// matches paths that start with `~/`
s if s.starts_with(&*PREFIX) => {
let home = home_dir()?;
home.join(&s[2..])
},
// // matches paths that start with `~` but not `~/`, might be a `~username/` path
// s if s.starts_with("~") => {
// let mut parts = s[1..].splitn(2, MAIN_SEPARATOR);
// let user = parts.next()
// .ok_or_else(|| io::Error::new(io::ErrorKind::Other, "malformed path"))?;
// let user = Passwd::from_name(&user)
// .map_err(|_| io::Error::new(io::ErrorKind::Other, "error searching for user"))?
// .ok_or_else(|| io::Error::new(io::ErrorKind::Other, format!("user '{}', does not exist", &user)))?;
// if let Some(ref path) = parts.next() {
// PathBuf::from(user.dir).join(&path)
// } else {
// PathBuf::from(user.dir)
// }
// },
// nothing to expand, just make a PathBuf
s => PathBuf::from(s)
})
}
pub fn home_dir() -> io::Result<PathBuf> {
dirs::home_dir()
.ok_or_else(|| io::Error::new(io::ErrorKind::Other, "no home directory is set"))
}
#[cfg(test)]
mod tests {
use crate::state::expanduser::expanduser;
use std::env;
use std::path::PathBuf;
#[test]
#[serial_test::serial]
fn test_success() {
let old_home = env::var("HOME").expect("no home dir set");
let new_home = "/home/foo";
env::set_var("HOME", new_home);
let path = expanduser("~/path/to/directory");
env::set_var("HOME", old_home);
assert_eq!(path.expect("io error"), PathBuf::from("/home/foo/path/to/directory"));
}
#[test]
#[serial_test::serial]
fn test_only_tilde() {
let old_home = env::var("HOME").expect("no home dir set");
let new_home = "/home/foo";
env::set_var("HOME", new_home);
let pathstr = "~";
let path = expanduser(pathstr);
env::set_var("HOME", old_home);
assert_eq!(path.expect("io error"), PathBuf::from("/home/foo"));
}
}
+5
View File
@@ -0,0 +1,5 @@
pub mod app_config;
pub mod app_dir;
pub mod os_platform;
pub mod yaml_config;
pub mod expanduser;
@@ -0,0 +1,20 @@
#[derive(Copy,Clone,Eq,PartialEq,Debug)]
pub enum OsPlatform {
Linux,
MacOs,
Windows,
}
impl OsPlatform {
pub fn get() -> OsPlatform {
// Totally scientific.
// https://doc.rust-lang.org/std/env/consts/constant.OS.html
match std::env::consts::OS {
"linux" => OsPlatform::Linux,
"windows" => OsPlatform::Windows,
"macos" | "apple" => OsPlatform::MacOs,
_ => OsPlatform::Linux,
}
}
}
@@ -0,0 +1,52 @@
use filesys::recursively_find_file_by_name::recursively_find_file_by_name;
use std::fs::File;
use std::io::BufReader;
use std::path::Path;
const CONFIG_FILENAME : &str = "app_config.yaml";
/// Configuration read or derived from a YAML config file at startup
/// This is not meant to contain any runtime config values.
/// This may not be exhaustive.
#[derive(serde::Deserialize, serde::Serialize, Clone, Debug, Default)]
pub struct YamlConfig {
pub image_height: Option<usize>,
pub image_width: Option<usize>,
pub scheduler_steps: Option<usize>,
pub scheduler_samples: Option<usize>,
pub seed: Option<u64>,
pub cfg_scale: Option<f64>,
/// Override the default location to store data on Linux
pub linux_default_data_path: Option<String>,
/// Override the default location to store data on Windows
pub windows_default_data_path: Option<String>,
/// Override the default location to store data on Mac
pub mac_default_data_path: Option<String>,
}
impl YamlConfig {
pub fn read_from_file<P: AsRef<Path>>(path: P) -> anyhow::Result<Self> {
let file = File::open(path)?;
let reader = BufReader::new(file);
let config = serde_yaml::from_reader(reader)?;
Ok(config)
}
pub fn load_from_config_file_recursive() -> YamlConfig {
let maybe_path = recursively_find_file_by_name(CONFIG_FILENAME, "../../../", 5)
.ok()
.flatten();
if let Some(path) = maybe_path {
let path = path.canonicalize().unwrap_or(path);
println!("Attempting to load configs from: {:?}", &path);
if let Ok(config) = YamlConfig::read_from_file(path) {
return config;
}
}
println!("Couldn't load config file. Using default configs.");
YamlConfig::default()
}
}
@@ -0,0 +1,103 @@
use std::fs::OpenOptions;
use std::io::Write;
use anyhow::anyhow;
use log::{error, info};
use once_cell::sync::Lazy;
use reqwest::Url;
use tauri::{AppHandle, Manager, Webview};
use errors::AnyhowResult;
use crate::state::app_dir::AppDataRoot;
use crate::utils::sora_webview_cookies::get_all_sora_cookies_as_string;
pub const LOGIN_WINDOW_NAME: &str = "login_window";
pub const SORA_LOGIN_URL_STR: &str = "https://sora.com/auth/login?callback_path=/";
pub static SORA_LOGIN_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_LOGIN_URL_STR).expect("URL should parse")
});
pub const SORA_ROOT_URL_STR: &str = "https://sora.com/";
pub static SORA_ROOT_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_ROOT_URL_STR).expect("URL should parse")
});
pub async fn login_thread(app: AppHandle, app_data_root: AppDataRoot) -> ! {
loop {
for (window_name, webview) in app.webviews() {
if window_name == LOGIN_WINDOW_NAME {
let result = check_login_window(&webview, &app_data_root).await;
if let Err(err) = result {
error!("Error checking login window: {:?}", err);
}
break;
}
}
tokio::time::sleep(std::time::Duration::from_millis(2_000)).await;
}
}
async fn check_login_window(webview: &Webview, app_data_root: &AppDataRoot) -> AnyhowResult<()> {
clear_browsing_data_on_test_domain(webview)?;
//keep_on_task(webview)?;
extract_cookies_to_file(webview, app_data_root)?;
// TODO: Other maintenance tasks ...
Ok(())
}
fn keep_on_task(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"auth.openai.com" => {},
"openai.com" => {},
"sora.com" => {},
// Third party SSO
"accounts.google.com" => {},
"accounts.youtube.com" => {},
"login.live.com" => {},
"appleid.apple.com" => {},
_ => {
info!("Non login hostname: {}", hostname);
webview.navigate(SORA_LOGIN_URL.clone())?;
}
}
Ok(())
}
/// This is just so we have a way to clear browsing data.
fn clear_browsing_data_on_test_domain(webview: &Webview) -> AnyhowResult<()> {
let url = webview.url()?;
let hostname= url.host()
.ok_or(anyhow!("no host in url"))?
.to_string();
match hostname.as_str() {
"storyteller.ai" => {
info!("Clearing all browsing data...");
webview.clear_all_browsing_data()?;
}
_ => {}
}
Ok(())
}
// TODO(bt,2025-04-07): Heuristic to detect when logged in. Only write when logged in.
fn extract_cookies_to_file(webview: &Webview, app_data_root: &AppDataRoot) -> AnyhowResult<()> {
let cookies = get_all_sora_cookies_as_string(webview)?;
let cookie_file = app_data_root.get_sora_cookie_file_path();
let mut file = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(cookie_file)?;
file.write_all(cookies.as_bytes())?;
file.flush()?;
Ok(())
}
@@ -0,0 +1 @@
pub mod login_thread;
@@ -0,0 +1,241 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use crate::transfer::error::Error;
use crate::transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use futures::stream::FuturesUnordered;
use futures::StreamExt;
use reqwest::header::{
HeaderMap, HeaderName, HeaderValue, AUTHORIZATION, CONTENT_RANGE,
RANGE,
};
use reqwest::Url;
use std::collections::HashMap;
use std::fmt::Display;
use std::fs::remove_file;
use std::io::SeekFrom;
use std::path::Path;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::sync::mpsc::Sender;
use std::time::Duration;
use log::info;
use tempfile::NamedTempFile;
use tokio::fs::OpenOptions;
use tokio::io::AsyncWriteExt;
use tokio::io::AsyncSeekExt;
use tokio::sync::Semaphore;
use tokio::time::sleep;
pub struct ProgressUpdate {
pub complete: usize,
pub total_length: usize,
}
/// max_files: Number of open file handles, which determines the maximum number of parallel downloads
/// parallel_failures: Number of maximum failures of different chunks in parallel (cannot exceed max_files)
/// max_retries: Number of maximum attempts per chunk. (Retries are exponentially backed off + jitter)
///
/// The number of threads can be tuned by the environment variable `TOKIO_WORKER_THREADS` as documented in
/// https://docs.rs/tokio/latest/tokio/runtime/struct.Builder.html#method.worker_threads
#[allow(clippy::too_many_arguments)]
pub async fn download_async<P: AsRef<Path>>(
url: String,
filename: NamedTempFile,
final_filename: P,
max_files: usize,
chunk_size: usize,
parallel_failures: usize,
max_retries: usize,
input_headers: Option<HashMap<String, String>>,
maybe_progress_sender: Option<Sender<ProgressUpdate>>,
) -> Result<(), Error> {
info!(">>> Downloading: {}", filename.as_ref().display());
let client = reqwest::Client::builder()
// https://github.com/hyperium/hyper/issues/2136#issuecomment-589488526
.http2_keep_alive_timeout(Duration::from_secs(15))
.build()?;
let mut headers = HeaderMap::new();
let mut auth_token = None;
if let Some(input_headers) = input_headers {
headers.reserve(input_headers.len());
for (k, v) in input_headers {
let name: HeaderName = k
.try_into()
.map_err(|err| Error::new_err(format!("Invalid header: {err}")))?;
let value: HeaderValue = AsRef::<str>::as_ref(&v)
.try_into()
.map_err(|err| Error::new_err(format!("Invalid header value: {err}")))?;
if name == AUTHORIZATION {
auth_token = Some(value);
} else {
headers.insert(name, value);
}
}
};
let response = if let Some(token) = auth_token.as_ref() {
client.get(&url).header(AUTHORIZATION, token)
} else {
client.get(&url)
}
.headers(headers.clone())
.header(RANGE, "bytes=0-0")
.send()
.await
.map_err(|err| Error::new_err(format!("Error while downloading: {err}")))?
.error_for_status()
.map_err(|err| Error::new_err(err.to_string()))?;
// Only call the final redirect URL to avoid overloading the Hub with requests and also
// altering the download count
let redirected_url = response.url();
if Url::parse(&url)
.map_err(|err| Error::new_err(format!("failed to parse url: {err}")))?
.host()
== redirected_url.host()
{
if let Some(token) = auth_token {
headers.insert(AUTHORIZATION, token);
}
}
let content_range = response
.headers()
.get(CONTENT_RANGE)
.ok_or(Error::new_err("No content length".to_string()))?
.to_str()
.map_err(|err| Error::new_err(format!("Error while downloading: {err}")))?;
let size: Vec<&str> = content_range.split('/').collect();
// Content-Range: bytes 0-0/702517648
// https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Range
let length: usize = size
.last()
.ok_or(Error::new_err(
"Error while downloading: No size was detected".to_string(),
))?
.parse()
.map_err(|err| Error::new_err(format!("Error while downloading: {err}")))?;
info!("File {:?} is {:?} length", filename.as_ref(), length);
let mut handles = FuturesUnordered::new();
let semaphore = Arc::new(Semaphore::new(max_files));
let parallel_failures_semaphore = Arc::new(Semaphore::new(parallel_failures));
let max_chunk_complete = Arc::new(AtomicUsize::new(0));
for start in (0..length).step_by(chunk_size) {
let url = redirected_url.to_string();
let filename = filename.as_ref().to_owned();
let client = client.clone();
let headers = headers.clone();
let stop = std::cmp::min(start + chunk_size - 1, length);
let semaphore = semaphore.clone();
let parallel_failures_semaphore = parallel_failures_semaphore.clone();
let maybe_progress = maybe_progress_sender.clone();
let max_chunk_complete = max_chunk_complete.clone();
handles.push(tokio::spawn(async move {
let permit = semaphore
.acquire_owned()
.await
.map_err(|err| Error::new_err(format!("Error while downloading: {err}")))?;
let mut chunk = download_chunk(&client, &url, &filename, start, stop, headers.clone()).await;
let mut i = 0;
if parallel_failures > 0 {
while let Err(dlerr) = chunk {
if i >= max_retries {
return Err(Error::new_err(format!(
"Failed after too many retries ({max_retries}): {dlerr}"
)));
}
let parallel_failure_permit = parallel_failures_semaphore.clone().try_acquire_owned().map_err(|err| {
Error::new_err(format!(
"Failed too many failures in parallel ({parallel_failures}): {dlerr} ({err})"
))
})?;
let wait_time = exponential_backoff(BASE_WAIT_TIME, i, MAX_WAIT_TIME);
sleep(Duration::from_millis(wait_time as u64)).await;
chunk = download_chunk(&client, &url, &filename, start, stop, headers.clone()).await;
i += 1;
drop(parallel_failure_permit);
if let Some(sender) = maybe_progress.as_ref() {
// NB: Calling fetch_max twice. First call will return the lower value.
let _old = max_chunk_complete.fetch_max(stop, Ordering::SeqCst);
let max_chunk = max_chunk_complete.fetch_max(stop, Ordering::SeqCst);
let progress_update = ProgressUpdate {
complete: max_chunk,
total_length: length,
};
if let Err(_err) = sender.send(progress_update) {
// Fail silently, not essential to issue progress updates
}
}
}
}
drop(permit);
chunk.map_err(|e| Error::new_err(format!("Downloading error {e}"))).and(Ok(stop - start))
}));
}
// Output the chained result
while let Some(result) = handles.next().await {
match result {
Ok(Ok(size)) => {
//if let Some(ref callback) = callback {
// callback.call((size,), None)?;
//}
}
Ok(Err(py_err)) => {
return Err(py_err);
}
Err(err) => {
return Err(Error::new_err(format!(
"Error while downloading: {err}"
)));
}
}
}
let file = filename.persist(final_filename)
.map_err(|e| Error::new_err(format!("Could not rename to final filename: {:?}", e)))?;
file.sync_all()?;
Ok(())
}
async fn download_chunk<P: AsRef<Path>>(
client: &reqwest::Client,
url: &str,
filename: P,
start: usize,
stop: usize,
headers: HeaderMap,
) -> Result<(), Error> {
// Process each socket concurrently.
let range = format!("bytes={start}-{stop}");
let mut file = OpenOptions::new()
.write(true)
.truncate(false)
.create(true)
.open(filename)
.await?;
file.seek(SeekFrom::Start(start as u64)).await?;
let response = client
.get(url)
.headers(headers)
.header(RANGE, range)
.send()
.await?
.error_for_status()?;
let content = response.bytes().await?;
file.write_all(&content).await?;
Ok(())
}
@@ -0,0 +1,49 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use std::fmt::Display;
use reqwest::header::ToStrError;
#[derive(Debug)]
pub enum Error {
Io(std::io::Error),
Request(reqwest::Error),
ToStrError(ToStrError),
Misc(String),
}
impl From<std::io::Error> for Error {
fn from(value: std::io::Error) -> Self {
Self::Io(value)
}
}
impl From<reqwest::Error> for Error {
fn from(value: reqwest::Error) -> Self {
Self::Request(value)
}
}
impl From<ToStrError> for Error {
fn from(value: ToStrError) -> Self {
Self::ToStrError(value)
}
}
impl Display for Error {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
Self::Io(io) => write!(f, "Io: {io}"),
Self::Request(req) => write!(f, "Request: {req}"),
Self::ToStrError(req) => write!(f, "Response non ascii: {req}"),
Self::Misc(err) => write!(f, "Misc error: {err}"),
}
}
}
impl std::error::Error for Error {}
impl Error {
pub fn new_err(reason: String) -> Self {
Self::Misc(reason)
}
}
@@ -0,0 +1,4 @@
pub mod download;
pub mod upload;
pub mod util;
pub mod error;
@@ -0,0 +1,189 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use crate::transfer::error::Error;
use crate::transfer::util::{exponential_backoff, BASE_WAIT_TIME, MAX_WAIT_TIME};
use futures::stream::FuturesUnordered;
use futures::StreamExt;
use reqwest::header::CONTENT_LENGTH;
use std::collections::HashMap;
use std::fmt::Display;
use std::io::SeekFrom;
use std::sync::Arc;
use std::time::Duration;
use tokio::fs::OpenOptions;
use tokio::io::{AsyncReadExt, AsyncSeekExt};
use tokio::sync::Semaphore;
use tokio::time::sleep;
use tokio_util::codec::{BytesCodec, FramedRead};
/// parts_urls: Dictionary consisting of part numbers as keys and the associated url as values
/// completion_url: The url that should be called when the upload is finished
/// max_files: Number of open file handles, which determines the maximum number of parallel uploads
/// parallel_failures: Number of maximum failures of different chunks in parallel (cannot exceed max_files)
/// max_retries: Number of maximum attempts per chunk. (Retries are exponentially backed off + jitter)
///
/// The number of threads can be tuned by the environment variable `TOKIO_WORKER_THREADS` as documented in
/// https://docs.rs/tokio/latest/tokio/runtime/struct.Builder.html#method.worker_threads
///
/// See https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html for more information
/// on the multipart upload
#[allow(clippy::too_many_arguments)]
fn multipart_upload(
file_path: String,
parts_urls: Vec<String>,
chunk_size: u64,
max_files: usize,
parallel_failures: usize,
max_retries: usize,
) -> Result<Vec<HashMap<String, String>>, Error> {
if parallel_failures > max_files {
return Err(Error::new_err(
"Error parallel_failures cannot be > max_files".to_string(),
));
}
if (parallel_failures == 0) != (max_retries == 0) {
return Err(Error::new_err(
"For retry mechanism you need to set both `parallel_failures` and `max_retries`"
.to_string(),
));
}
tokio::runtime::Builder::new_multi_thread()
.enable_all()
.build()?
.block_on(async {
upload_async(
file_path,
parts_urls,
chunk_size,
max_files,
parallel_failures,
max_retries,
)
.await
})
}
#[allow(clippy::too_many_arguments)]
async fn upload_async(
file_path: String,
parts_urls: Vec<String>,
chunk_size: u64,
max_files: usize,
parallel_failures: usize,
max_retries: usize,
) -> Result<Vec<HashMap<String, String>>, Error> {
let client = reqwest::Client::new();
let mut handles = FuturesUnordered::new();
let semaphore = Arc::new(Semaphore::new(max_files));
let parallel_failures_semaphore = Arc::new(Semaphore::new(parallel_failures));
for (part_number, part_url) in parts_urls.iter().enumerate() {
let url = part_url.to_string();
let path = file_path.to_owned();
let client = client.clone();
let start = (part_number as u64) * chunk_size;
let semaphore = semaphore.clone();
let parallel_failures_semaphore = parallel_failures_semaphore.clone();
handles.push(tokio::spawn(async move {
let permit = semaphore
.clone()
.acquire_owned()
.await
.map_err(|err| Error::new_err(format!("Error acquiring semaphore: {err}")))?;
let mut chunk = upload_chunk(&client, &url, &path, start, chunk_size).await;
let mut i = 0;
if parallel_failures > 0 {
while let Err(ul_err) = chunk {
if i >= max_retries {
return Err(Error::new_err(format!(
"Failed after too many retries ({max_retries}): {ul_err}"
)));
}
let parallel_failure_permit = parallel_failures_semaphore.clone().try_acquire_owned().map_err(|err| {
Error::new_err(format!(
"Failed too many failures in parallel ({parallel_failures}): {ul_err} ({err})"
))
})?;
let wait_time = exponential_backoff(BASE_WAIT_TIME, i, MAX_WAIT_TIME);
sleep(Duration::from_millis(wait_time as u64)).await;
chunk = upload_chunk(&client, &url, &path, start, chunk_size).await;
i += 1;
drop(parallel_failure_permit);
}
}
drop(permit);
chunk.map_err(|e|{
//match e {
// Error::Io(io) => Error::new_err(format!("Io error {io}")),
// Error::Request(req) => Error::new_err(format!("Error while sending chunk {req}")),
// Error::ToStrError(req) => Error::new_err(format!("Response header contains non ASCII chars: {req}")),
//}
e
}).map(|chunk| (part_number, chunk, chunk_size))
}));
}
let mut results: Vec<HashMap<String, String>> = vec![HashMap::default(); parts_urls.len()];
while let Some(result) = handles.next().await {
match result {
Ok(Ok((part_number, headers, size))) => {
//if let Some(ref callback) = callback {
// callback.call((size,), None)?;
//}
results[part_number] = headers;
}
Ok(Err(py_err)) => {
return Err(py_err);
}
Err(err) => {
return Err(Error::new_err(format!(
"Error occurred while uploading: {err}"
)));
}
}
}
Ok(results)
}
async fn upload_chunk(
client: &reqwest::Client,
url: &str,
path: &str,
start: u64,
chunk_size: u64,
) -> Result<HashMap<String, String>, Error> {
let mut options = OpenOptions::new();
let mut file = options.read(true).open(path).await?;
let file_size = file.metadata().await?.len();
let bytes_transferred = std::cmp::min(file_size - start, chunk_size);
file.seek(SeekFrom::Start(start)).await?;
let chunk = file.take(chunk_size);
let response = client
.put(url)
.header(CONTENT_LENGTH, bytes_transferred)
.body(reqwest::Body::wrap_stream(FramedRead::new(
chunk,
BytesCodec::new(),
)))
.send()
.await?;
let response = response.error_for_status()?;
let mut headers = HashMap::new();
for (name, value) in response.headers().into_iter() {
headers.insert(name.to_string(), value.to_str()?.to_owned());
}
Ok(headers)
}
@@ -0,0 +1,15 @@
//! Implemented from https://github.com/huggingface/hf_transfer/blob/main/src/lib.rs
use rand::{thread_rng, Rng};
use std::fmt::Display;
pub const BASE_WAIT_TIME: usize = 300;
pub const MAX_WAIT_TIME: usize = 10_000;
fn jitter() -> usize {
thread_rng().gen_range(0..=500)
}
pub fn exponential_backoff(base_wait_time: usize, n: usize, max: usize) -> usize {
(base_wait_time + n.pow(2) + jitter()).min(max)
}
@@ -0,0 +1,7 @@
use tauri::Webview;
use errors::AnyhowResult;
pub fn clear_all_webview_cookies(webview: &Webview) -> AnyhowResult<()> {
webview.clear_all_browsing_data()?;
Ok(())
}
@@ -0,0 +1,14 @@
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use image::{DynamicImage, ImageReader};
use std::io::Cursor;
pub fn decode_base64_image(base64_image: &str) -> anyhow::Result<DynamicImage> {
let bytes = BASE64_STANDARD.decode(base64_image)?;
let image = ImageReader::new(Cursor::new(bytes))
.with_guessed_format()?
.decode()?;
Ok(image)
}
@@ -0,0 +1,14 @@
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use image::{DynamicImage, ImageFormat};
use std::io::Cursor;
pub fn encode_dynamic_image_base64_png(image: DynamicImage) -> anyhow::Result<String> {
let mut bytes = Vec::with_capacity(1024*1024);
image.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)?;
let bytes = BASE64_STANDARD.encode(bytes);
Ok(bytes)
}
@@ -0,0 +1,16 @@
use base64::prelude::BASE64_STANDARD;
use base64::Engine;
use image::{ImageFormat, RgbImage};
use std::io::Cursor;
pub fn encode_rgb_image_base64_png(
image: RgbImage
) -> anyhow::Result<String> {
let mut bytes = Vec::with_capacity(1024*1024);
image.write_to(&mut Cursor::new(&mut bytes), ImageFormat::Png)?;
let bytes = BASE64_STANDARD.encode(bytes);
Ok(bytes)
}
@@ -0,0 +1,3 @@
pub mod decode_base64_image;
pub mod encode_dynamic_image_base64_png;
pub mod encode_rgb_image_base64_png;
+3
View File
@@ -0,0 +1,3 @@
pub mod clear_all_webview_cookies;
pub mod image;
pub mod sora_webview_cookies;
@@ -0,0 +1,26 @@
use once_cell::sync::Lazy;
use reqwest::Url;
use tauri::Webview;
use tauri::webview::Cookie;
use errors::AnyhowResult;
const SORA_ROOT_COOKIE_URL_STR: &str = "https://sora.com/";
static SORA_ROOT_COOKIE_URL: Lazy<Url> = Lazy::new(|| {
Url::parse(SORA_ROOT_COOKIE_URL_STR).expect("URL should parse")
});
pub fn get_all_sora_cookies(webview: &Webview) -> AnyhowResult<Vec<Cookie>> {
let cookies = webview.cookies_for_url(SORA_ROOT_COOKIE_URL.clone())?;
Ok(cookies)
}
pub fn get_all_sora_cookies_as_string(webview: &Webview) -> AnyhowResult<String> {
let cookies = get_all_sora_cookies(webview)?;
let cookie_string = cookies
.iter()
.map(|cookie| format!("{}={}", cookie.name(), cookie.value()))
.collect::<Vec<String>>()
.join("; ");
Ok(cookie_string)
}
+37
View File
@@ -0,0 +1,37 @@
{
"$schema": "https://schema.tauri.app/config/2.0.0-rc",
"productName": "ArtCraft",
"version": "0.1.0",
"identifier": "com.unique.value.thing.why",
"build": {
"frontendDist": "../../../frontend/realtime/dist",
"devUrl": "http://localhost:5174",
"beforeDevCommand": "npm run dev",
"beforeBuildCommand": "npm run build2"
},
"app": {
"windows": [
{
"title": "ArtCraft",
"width": 2400,
"height": 1300,
"resizable": true,
"fullscreen": false
}
],
"security": {
"csp": null
}
},
"bundle": {
"active": true,
"targets": "all",
"icon": [
"icons/32x32.png",
"icons/128x128.png",
"icons/128x128@2x.png",
"icons/icon.icns",
"icons/icon.ico"
]
}
}
@@ -0,0 +1,7 @@
#[test]
fn test_dynamic_library_loading() {
let name = libloading::library_filename("cublas");
assert_eq!(name, "libcublas.so");
}
+4
View File
@@ -0,0 +1,4 @@
# Generated by Cargo
# will have compiled files and executables
/target/
/gen/schemas
@@ -0,0 +1,4 @@
# Don't restart the dev server on Rust changes
crates/
Cargo.toml
Cargo.lock
+86
View File
@@ -0,0 +1,86 @@
[package]
name = "tauri-realtime"
version = "0.1.0"
description = "A Tauri App"
authors = ["you"]
license = ""
repository = ""
edition = "2021"
rust-version = "1.77.2"
publish = false
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[lib]
name = "app_lib"
crate-type = ["staticlib", "cdylib", "rlib"]
[build-dependencies]
tauri-build = { version = "2.1.1", features = [] }
[dependencies]
# Internal
crockford = { path = "../../lib/data/crockford" }
filesys = { path = "../../lib/files/filesys" }
memory_caching = { path = "../../lib/caching/memory_caching" }
ml_weights_registry = { path = "../../lib/machine_learning/ml_weights_registry" }
primitives = { path = "../../lib/data/primitives" }
# Internal ML
# TODO: Only on Windows and Linux
#ml_models = { path = "../ml_models", optional = true }
# Bypass cudarc issues on Windows https://github.com/coreylowman/cudarc/issues/332
cudarc = { version = "0.13.5", features = ["std", "cublas", "cublaslt", "curand", "driver", "nvrtc", "f16", "cuda-version-from-build-system", "dynamic-linking"], default-features=false }
# External
#openssl = { version = "0.10", features = ["vendored"] }
# NB(bt,2025-03-10): default-features = false on Windows while I debug openssl / rustls
anyhow = "1.0"
base64 = "0.22.1"
bytes = "1.10"
chrono = "0.4.40"
const_format = "0.2.34"
directories = "6.0.0"
dirs = "6.0.0"
futures = "0.3"
half = { version = "2.3.1", features = ["num-traits", "use-intrinsics", "rand_distr"] }
image = "0.25.5"
log = "0.4"
once_cell = "1.20.3"
ort = { version = "1.16.2", features = ["cuda", "load-dynamic"] }
rand = "0.8.5"
reqwest = { version = "0.12.12", features = ["stream"], default-features = false }
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
serde_yaml = "0.9"
tauri = { version = "2.4.1", features = [] }
tauri-plugin-log = "2.3.1"
tempdir = "0.3.7"
tempfile = "3.18.0"
tokio = { version = "1.42", features = ["rt", "rt-multi-thread", "fs"] }
tokio-util = { version = "0.7", features = ["codec"] }
# tracing-subscriber = { version = "0.3", features = [ "env-filter", "fmt" ] }
[target.'cfg(not(target_os = "macos"))'.dependencies]
ml_models = { path = "../../lib/machine_learning/ml_models" }
[dev-dependencies]
libloading = "0.8.6"
serial_test = "3.2.0"
assertor = "0.0.3"
#[target.'cfg(target_os = "macos")'.features]
#default = []
#ml_models = ["dep:ml_models"]
[profile.dev]
incremental = true # Compile your binary in smaller steps.
[profile.release]
codegen-units = 1 # Allows LLVM to perform better optimization.
lto = true # Enables link-time-optimizations.
opt-level = 3 # `s` prioritizes small binary size. Use `3` if you prefer speed.
panic = "abort" # Higher performance by disabling panic handlers.
strip = true # Ensures debug symbols are removed.
Binary file not shown.

After

Width:  |  Height:  |  Size: 8.4 KiB

+3
View File
@@ -0,0 +1,3 @@
fn main() {
tauri_build::build()
}
@@ -0,0 +1,12 @@
{
"$schema": "../gen/schemas/desktop-schema.json",
"identifier": "default",
"description": "enables the default permissions",
"windows": [
"main"
],
"permissions": [
"core:default",
"log:default"
]
}
Binary file not shown.

After

Width:  |  Height:  |  Size: 11 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 23 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.2 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 9.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 13 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 25 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 2.0 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 28 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.3 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.9 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 7.4 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 3.9 KiB

Binary file not shown.
Binary file not shown.

After

Width:  |  Height:  |  Size: 37 KiB

Some files were not shown because too many files have changed in this diff Show More