Compare commits

...
Author SHA1 Message Date
Tanish Manku f5b80ea982 Dev infra: local filesystem uploads + fully-local pagescene media
Closes the two remaining fully-local-stack seams found after the last round.

Uploads (previously every upload endpoint returned 500 with dummy S3 creds -
the bucket put happens before the DB insert and rust-s3 is fail-on-err):
- LegacyBucketClient gains an internal backend enum (S3 unchanged | LocalDir)
  and a create_local(root, bucket_name, root_prefix) constructor. All upload/
  download methods route LocalDir to the filesystem; bucket_name() reports the
  configured name. Objects map disk = root + rooted object path via the new
  pub local_object_disk_path (rejects ".." and backslash segments - PMX zips
  feed user-controlled names), unit-tested. The fake-generation resolver now
  uses the same function instead of its private copy so the layouts can't
  drift.
- build_dependencies: when SERVER_ENVIRONMENT=Development and
  LOCAL_MEDIA_ROOT is set, all three bucket clients are constructed in local
  mode (private/gc have zero callers in storyteller-web, so this is purely
  uniform); the server_environment read moved above client construction.
  Uploads land under .devstack/media and are immediately viewable through
  the existing /media mount + CDN_BASE_URL - no S3/R2/MinIO.

Frontend (pagescene was the only surface actually broken against a local
backend; moodboard/library/create-* already consume media_links correctly):
- The vite /__cdn dev proxy assumed https upstreams, so localhost media
  (http) died on a TLS handshake with a 502 - breaking every pagescene
  GLB/splat/animation fetch. fetchAsset no longer proxies localhost at all
  (the dev backend sends CORS for any localhost origin), and the proxy
  itself now picks http for localhost hosts as a backstop.
- GetCdnOrigin() returns the API host when it is localhost (the local
  backend serves media itself; CDN origin == API origin), production CDN
  otherwise. The pagescene adapter's getCdnUrl skips the Cloudflare
  /cdn-cgi/image resize segment on localhost - the local backend has no
  resizer, and dev serves originals anyway.
- vite /v1 fallback proxy now targets http://localhost:12345 instead of
  production, so nothing can silently fall back to prod traffic in dev;
  fixed the stale main.tsx comment claiming setDevelopment() is disabled.

Docs: seams table updated (uploads + vite proxy + GetCdnOrigin closed;
API_TARGETS/SegmentationApi noted as dead code; the earlier moodboard claim
was wrong - it never had a hardcoded host), LOCAL_MEDIA_ROOT now documents
the storage mode, verification checklist gains the upload check.

Verified E2E on this machine: POST /v1/media_files/upload/image (multipart,
real endpoint) returns success, the object lands under .devstack/media at
the canonical fan-out path, lists in the library with media_class=image,
and serves 200 image/jpeg from http://localhost:12345/media/...;
cloud_storage unit tests pass; storyteller-web + webapp dev server boot
clean with "Local filesystem storage mode is ON".
2026-07-29 21:10:42 -07:00
Tanish Manku 335015d9e2 Dev infra: fully-local generation and gallery for the dev stack
Closes the two gaps found smoke-testing the local stack: an empty gallery
(zero media_files rows, media URLs hardcoded to the remote r2.dev bucket)
and generation flows that could not complete (omni_gen calls providers
synchronously in the handler, so dummy keys meant a 500 after billing;
the webapp blocked submits earlier anyway because signup grants no wallet).

Backend (all development-only):
- CDN_BASE_URL env override in cdn_link.rs (host-only, validated loudly;
  precedence over the compiled-in hosts for every domain/env) and a
  dev-gated actix-files mount serving LOCAL_MEDIA_ROOT/media at /media,
  mirroring the public bucket layout - media URLs in API responses resolve
  against the backend itself
- DEV_FAKE_GENERATION flag on ServerState (easyenv bool, forced off outside
  Development): the omni_gen image/video pipelines skip the external
  provider call and return a synthetic Fal enqueue (external id fake_<uuid>),
  so billing, prompt insert, and the pending generic_inference_jobs row all
  stay real
- dev_fake_generation_resolver_thread: polls pending fake_ jobs (new
  runtime-query list_pending_fake_generation_jobs in mysql_queries) and
  after DEV_FAKE_GENERATION_RESOLVE_SECS (default 6s) marks them
  complete_success with a media_files row backed by a test_data asset
  copied under LOCAL_MEDIA_ROOT (mp4 results get -thumb.jpg/-thumb.gif
  siblings since the thumbnail job never runs locally), or complete_failure
  via the existing simulate_artcraft_failure prompt triggers (category word
  supported, e.g. "simulate_artcraft_failure face_not_detected")

Bootstrap/seeds (PS + bash kept in lockstep):
- seed_demo_user seeds an artcraft wallet (DEMO_CREDITS, default 100k
  banked + ledger rows; idempotent top-up) - signup creates no wallet and
  the webapp gates generate on balance, which is why submits went nowhere
- seed_demo_user also seeds 6 gallery images (test_data jpgs) with objects
  under .devstack/media in the exact bucket fan-out layout
- fixed the already-exists detection in seed_demo_user: create_account
  returns error_type, not error_code, so re-runs died on UsernameTaken
- generated secrets env gains DEV_FAKE_GENERATION/CDN_BASE_URL/
  LOCAL_MEDIA_ROOT (appended to existing files on bootstrap re-runs);
  stale "providers are only contacted by workers" comments corrected
- doctor checks wallet balance, media rows, and the new secrets keys

Docs: dev_setup_local_stack.md documents the new flags, corrects the
sync-provider-call and upload-500 claims, updates the seams table and
verification checklist; storyteller_web CLAUDE.md video model examples
fixed (seedance_2p0_fast_g -> seedance_2p0_fast).

Verified E2E on this machine: gallery lists 8 items on localhost URLs;
image and video submits go pending ~6s then complete with locally-served
results (mp4 + gif preview 200); failure trigger yields complete_failure
with category face_not_detected; credits genuinely deducted (100000 ->
99932); doctor 0 failures / 0 warnings.
2026-07-29 20:48:31 -07:00
Tanish Manku f23a6df80b Dev infra: harden Windows bootstrap after two adversarial debug passes
Pass 1 (correctness sweep):
- All native invocations (mysql, mysqld init, winget, curl downloads,
  cargo run, seed curl) now go through Invoke-Native, so a caller capturing
  the scripts' output can never turn ordinary stderr (warnings, progress)
  into a terminating NativeCommandError under Stop preference
- Provisioning DDL feeds through a temp file + `source` and prints the mysql
  output when it fails (with the MYSQL_ROOT_PASSWORD hint)
- Get-MySqlCount distinguishes "table empty" (0) from "query failed" (-1);
  bootstrap and doctor no longer misread a failed COUNT as unseeded
- diesel_cli that is present but broken (moved libmysql.dll) is detected via
  an exit-code probe and reinstalled with --force
- Shared Add-BuildToolsToSession maps winget install dirs + LIBCLANG_PATH in
  bootstrap, doctor, and run_backend_dev (cargo run may recompile); the tools
  check maps PATH before probing so re-runs skip winget entirely
- my.ini values quoted (clone paths with spaces); service-start timeouts
  point at the .devstack log files

Pass 2 (live retest matrix, all green on this machine):
- Full re-run with output captured in-PowerShell: exit 0, no NativeCommandError
- Wiped-state rebuild (mysql-data/my.ini/redis/secrets deleted): re-init,
  re-provision, 281 migrations replayed, seeds + secrets regenerated, backend
  booted, demo user recreated via /v1/create_account, doctor 0 fail / 0 warn
- Failure paths: seed without backend and doctor without services both die
  cleanly with actionable messages and exit 1; stop_dev_services frees both
  ports gracefully
2026-07-28 22:37:03 -07:00
Tanish Manku 345cbe766a Dev infra: Windows-first bootstrap - portable MySQL/Redis stack, no WSL required
Adds script/bootstrap/windows/*.ps1 (PowerShell 5.1-compatible) as the primary
bootstrap path, validated end-to-end on a real Windows 10 machine:

- Portable stack under gitignored .devstack/: MySQL 8.4.11 (official
  cdn.mysql.com zip, --initialize-insecure, bound to 127.0.0.1) and Redis
  5.0.14.1 (tporadowski build, REDIS_ZIP_URL-overridable) run as plain user
  processes - no admin, no services, no WSL, no Docker
- bootstrap_dev_stack.ps1: winget-installs missing build tools (Strawberry
  Perl, NASM, LLVM; wreq's BoringSSL + aws-lc-sys need them), compiles
  diesel_cli against the portable MySQL's own client lib
  (MYSQLCLIENT_LIB_DIR), runs all 281 migrations, seeds roles/badges
  idempotently, generates the dummy secrets env (plus TEMP_DIR for Windows),
  builds storyteller-web, installs frontend deps
- start/stop_dev_services.ps1, run_backend_dev.ps1, seed_demo_user.ps1,
  dev_stack_doctor.ps1 mirror the bash variants
- Invoke-Native (cmd /c ... 2>&1) wraps long-running native calls so PS 5.1
  cannot turn harmless stderr warnings into terminating NativeCommandErrors
  when output is captured; my.ini uses forward slashes (MySQL escape rules)

Verified on this machine: 281/281 migrations applied, backend built (3m05s)
and booted on :12345 with dummy secrets, POST /v1/create_account +
/v1/login round-trip succeeded, webapp dev server served on :4201, doctor
reports 0 failures. The bash scripts remain the Linux/WSL2/CI path.
2026-07-28 21:23:06 -07:00
Tanish Manku 537409eb3e Dev infra: fresh-clone bootstrap for the fully-local dev stack (backend + webapp)
Adds script/bootstrap/ — idempotent Ubuntu 22.04+/WSL2/CI automation that takes
a fresh clone to a running local stack with zero production traffic:

- bootstrap_dev_stack.sh: apt packages, MySQL 8 + Redis up, storyteller DB/user,
  281 diesel migrations, idempotent role/badge seeds, generated dummy
  development-secrets.env (19 required-at-boot vars no checked-in config
  supplies), SQLX_OFFLINE backend build, frontend npm install
- run_backend_dev.sh / seed_demo_user.sh: run the API from the repo root and
  create+verify a demo login through the real POST /v1/create_account path
- dev_stack_doctor.sh: PASS/WARN/FAIL diagnosis of every prerequisite

_docs/dev_setup_local_stack.md records the requirements the scripts encode and
corrects stale claims (server reads MYSQL_URL, not DATABASE_URL; repo-root .env
is not loaded by the server; config precedence is first-value-wins;
SQLX_OFFLINE is compile-time only).

.gitattributes now forces LF for *.sh/*.env so Windows clones run under WSL2.
2026-07-28 20:51:19 -07:00
35 changed files with 3029 additions and 59 deletions
+5
View File
@@ -1,3 +1,8 @@
# Shell scripts and dotenv config files are executed/parsed on Linux (WSL2, CI)
# even when the repo is cloned on Windows — they must stay LF in the worktree.
*.sh text eol=lf
*.env text eol=lf
# Ignore certain code for Github language statistics
# https://github.com/github/linguist/blob/master/docs/overrides.md
old_code/trumpet-trumped.com/dictionary/cmudict-0.7b linguist-vendored
+3
View File
@@ -104,3 +104,6 @@ windows/sqlite/sqlite3.lib
crates/desktop/src-tauri/*.dll
windows_dlls/
# Portable local dev stack (script/bootstrap/windows) - MySQL/Redis binaries, data, logs
.devstack/
Generated
+31 -1
View File
@@ -34,6 +34,29 @@ dependencies = [
"smallvec",
]
[[package]]
name = "actix-files"
version = "0.6.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "df8c4f30e3272d7c345f88ae0aac3848507ef5ba871f9cc2a41c8085a0f0523b"
dependencies = [
"actix-http",
"actix-service",
"actix-utils",
"actix-web",
"bitflags 2.11.0",
"bytes",
"derive_more 2.0.1",
"futures-core",
"http-range",
"log",
"mime",
"mime_guess",
"percent-encoding",
"pin-project-lite",
"v_htmlescape",
]
[[package]]
name = "actix-http"
version = "3.12.1"
@@ -7673,7 +7696,7 @@ version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "681030a937600a36906c185595136d26abfebb4aa9c65701cefcaf8578bb982b"
dependencies = [
"proc-macro-crate 2.0.0",
"proc-macro-crate 3.5.0",
"proc-macro2",
"quote",
"syn 2.0.117",
@@ -11577,6 +11600,7 @@ checksum = "ef5430c8e36b713e13b48a9f709cc21e046723fe44ce34587b73a830203b533e"
name = "storyteller-web"
version = "0.0.1"
dependencies = [
"actix-files",
"actix-http",
"actix-multipart",
"actix-service",
@@ -13894,6 +13918,12 @@ dependencies = [
"wasm-bindgen",
]
[[package]]
name = "v_htmlescape"
version = "0.15.8"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4e8257fbc510f0a46eb602c10215901938b5c2a7d5e70fc11483b1d3c9b5b18c"
[[package]]
name = "value-bag"
version = "1.12.0"
+1
View File
@@ -307,6 +307,7 @@ worldlabs_consumer_client = { path = "crates/api_clients/worldlabs_consumer_clie
# Centrally pinned versions so that we can uniformly upgrade/downgrade versions across the workspace
# If a crate needs to use an older/newer version, the dependency should be managed in that crate's own Cargo.toml
actix-cors = "0.7.1"
actix-files = "0.6.9"
actix-http = "3.12.1"
actix-multipart = "0.7.2"
actix-rt = "2.11.0"
+294
View File
@@ -0,0 +1,294 @@
# Local Dev Stack — Requirements & Bootstrap
How to run the full ArtCraft dev stack **entirely locally** — the
`storyteller-web` backend plus the `artcraft-webapp` frontend — with no traffic
to production servers. Scope: webapp-first; the Tauri desktop app is a later
phase.
The automated path on **Windows** (primary platform, native — no WSL, no
Docker, no admin rights for the stack itself):
```powershell
.\script\bootstrap\windows\bootstrap_dev_stack.ps1 # one-time setup, idempotent, re-run anytime
.\script\bootstrap\windows\run_backend_dev.ps1 # terminal 1: API on http://localhost:12345
.\script\bootstrap\windows\seed_demo_user.ps1 # terminal 2: create demo login (localdev1 / localdev1pass)
cd frontend; npx nx dev artcraft-webapp # terminal 2: webapp on http://localhost:4201
.\script\bootstrap\windows\dev_stack_doctor.ps1 # health check, any time
.\script\bootstrap\windows\stop_dev_services.ps1 # stop MySQL/Redis when done
```
The equivalent on **Linux** (Ubuntu 22.04+ — native, WSL2, or CI):
```bash
./script/bootstrap/bootstrap_dev_stack.sh # one-time setup, idempotent, re-run anytime
./script/bootstrap/run_backend_dev.sh # terminal 1: API on http://localhost:12345
./script/bootstrap/seed_demo_user.sh # terminal 2: create demo login (localdev1 / localdev1pass)
cd frontend && npx nx dev artcraft-webapp # terminal 2: webapp on http://localhost:4201
./script/bootstrap/dev_stack_doctor.sh # health check, any time
```
This document records the underlying requirements so the scripts aren't a
black box, and corrects some stale claims in older docs.
## Supported platforms
| Platform | Status |
|---------------------|---------------------------------------------------------------------------|
| Windows 10/11 | Primary. `script/bootstrap/windows/*.ps1` run a **portable stack**: MySQL 8.4 (official zip) and Redis live under the gitignored `.devstack/` and run as plain user processes — no admin, no services, no WSL, no Docker. Build tools (cmake, perl, nasm, llvm) are offered via winget |
| Ubuntu 22.04+ | Supported by `script/bootstrap/*.sh` (native, WSL2, or CI runner/container); services via apt |
| macOS | Not covered by the scripts; follow `_docs/dev_setup_server.md` manually |
### Windows specifics
- Everything stateful lives in `.devstack/` (gitignored): `mysql/` (extracted
zip), `mysql-data/`, `redis/`, `logs/`, `pids/`, `downloads/` (cached zips),
`my.ini`. Delete the directory to fully reset; re-run bootstrap to rebuild.
- Pinned services: MySQL **8.4.11** from `cdn.mysql.com` (the 8.x series is
required; only some patch versions exist on the CDN) and Redis **5.0.14.1**
(tporadowski build — old but sufficient for the backend's keepalive/TTL/
rate-limit usage; override `REDIS_ZIP_URL` to use Memurai/Garnet instead).
- The portable MySQL root user has **no password** (`--initialize-insecure`) —
local dev only, bound to 127.0.0.1.
- diesel-cli is compiled against the portable MySQL's own client library
(`MYSQLCLIENT_LIB_DIR`/`MYSQLCLIENT_VERSION`); `libmysql.dll` must be on
`PATH` to *run* it — the scripts handle both.
- Compiling the backend needs cmake, Strawberry Perl, NASM, and LLVM/libclang
(the `wreq` HTTP client builds BoringSSL, and `aws-lc-sys`/`ring` need
NASM/clang on MSVC). The bootstrap checks and offers winget installs.
- The generated secrets file additionally sets `TEMP_DIR` (the upload temp-dir
helper defaults to `/tmp`, which doesn't exist on Windows).
## Requirements matrix
| Requirement | Version / detail | Needed for |
|-------------------|--------------------------------------------------|---------------------------------------------------|
| MySQL server | 8.x series (9.x unsupported by tooling) | Backend boot (eager connect) + migrations |
| Redis server | any recent | Backend boot (r2d2 pool connects eagerly) |
| Rust (stable) | no pinned MSRV; Docker builds use 1.93.0 | Building `storyteller-web`, diesel_cli |
| diesel_cli | `cargo install diesel_cli --no-default-features --features mysql,sqlite` | Running the 281 migrations |
| Node.js | 20+ (enforced by `script/common/frontend_preflight.sh`) | Frontend (Nx 21 / Vite 6) |
| npm | ships with Node; **pnpm must not be used** | Frontend install (`frontend/package-lock.json`) |
| System libs (apt) | build-essential, cmake, libssl-dev, libclang-dev, libfontconfig1-dev, fontconfig, perl, pkg-config, ffmpeg, libmysqlclient-dev, libsqlite3-dev, curl, jq | Compiling the workspace + diesel_cli; seeding scripts |
| Elasticsearch | optional (7.x) | Only 4 search endpoints; server boots without it |
| Docker | **not required** — no docker-compose exists | — |
Database identity (used consistently by docs, `.env`, and defaults):
database `storyteller`, user `storyteller`, password `password`, host
`localhost`.
## Backend: what `storyteller-web` needs to boot
Binary: `storyteller-web` in `crates/service/web/storyteller_web`
(binaries: `storyteller-web`, `docs` — Swagger UI on :8989, `docs-cli` — emits
`api.json`). Binds `0.0.0.0:12345` (`BIND_ADDRESS`), plain HTTP.
**Run it from the repo root** — the config search path and `includes/` paths
are cwd-relative.
### Config loading (corrected facts)
- Mode is selected by the `SERVER_ENVIRONMENT` **process env var only**
(default: Development). Setting it inside an env file does nothing — it is
read before any file loads.
- In Development mode, files load from
`crates/service/web/storyteller_web/config/` in this order:
`storyteller-web.common.env` → `storyteller-web.development.env` →
`storyteller-web.development-secrets.env` (gitignored; the bootstrap
generates it).
- **Precedence is first-value-wins** (dotenv never overrides an existing key):
real process env > common.env > development.env > development-secrets.env.
The secrets file therefore cannot override anything already set in
`development.env`.
- The repo-root `.env` is **NOT loaded by the server** (it passes
`ignore_legacy_dot_env_file: true`). Root `.env`'s `DATABASE_URL` feeds
diesel-cli/sqlx-cli only; the server reads **`MYSQL_URL`** (default
`mysql://storyteller:password@localhost/storyteller`).
### Hard requirements at boot
| Requirement | Behavior if missing |
|----------------------------------|--------------------------------------------------------|
| MySQL reachable via `MYSQL_URL` | Boot fails (sqlx pool connects eagerly; the DB must exist and be migrated — health-check and ban-poller threads query immediately) |
| Redis reachable via `REDIS_0_URL` (default `redis://localhost/0`) | Boot fails (r2d2 pool connects eagerly) |
| ~30 `get_env_string_required` vars present | Boot fails on the first absent var |
Of the required-present vars, 19 are supplied by **no** checked-in file and
live in the generated secrets file: `ACCESS_KEY`, `SECRET_KEY`, `REGION_NAME`,
`W2L_PRIVATE_DOWNLOAD_BUCKET_NAME`, `W2L_PUBLIC_DOWNLOAD_BUCKET_NAME`,
`RESEND_API_KEY`, `FAL_API_KEY`, `GMICLOUD_API_KEY`, `GROK_API_KEY`,
`BEEBLE_API_KEY`, `OPENAI_API_KEY`, `WORLDLABS_API_KEY`,
`SEEDANCE2PRO_COOKIES`, `SEEDANCE2PRO_WHITELIST_COOKIES`,
`SEEDANCE2PRO_BYTEPLUS_ULTRA_COOKIES`, `STRIPE_FAKEYOU_ACCOUNT_ID`,
`STRIPE_ARTCRAFT_ACCOUNT_ID`, `STRIPE_ARTCRAFT_SECRET_KEY`,
`STRIPE_ARTCRAFT_SECRET_WEBHOOK_KEY`.
**Placeholder values are safe at boot**: none of these services is contacted
at boot (clients are constructed offline; Elasticsearch, S3/R2, Stripe,
Resend, and Google-cert fetches are all lazy). But two earlier claims here
were wrong and are corrected:
- **The omni_gen endpoints call providers synchronously in the HTTP handler**
(`/v1/omni_gen/generate/{image,video}` — the path the webapp uses). With
dummy keys the provider call fails and the submit returns HTTP 500 *after*
billing the wallet (the image pipeline has no refund; video refunds only
Kinovi). Only the legacy polling providers (GmiCloud, Grok, Kinovi orders,
WorldLabs) are worker-driven. `DEV_FAKE_GENERATION` (below) exists so local
submits never reach a real provider.
- **Uploads would fail hard with dummy S3 creds** (every media upload
endpoint uploads to the bucket *before* inserting the DB row, and `rust-s3`
is built with `fail-on-err`) — which is why `LOCAL_MEDIA_ROOT` (below) also
switches the bucket clients into a local filesystem mode: uploads write
under `.devstack/media/` and are immediately viewable via the same `/media`
mount. No S3/R2/MinIO needed.
### Fully-local generation and media (`DEV_FAKE_GENERATION`)
The generated secrets file enables three development-only settings (appended
automatically on bootstrap re-runs for older stacks):
| Env var | Effect |
|-----------------------|----------------------------------------------------------------------------|
| `DEV_FAKE_GENERATION` | omni_gen image/video submits skip the external provider and insert a normal **pending** job with a `fake_…` external id. A resolver thread inside `storyteller-web` completes each one after `DEV_FAKE_GENERATION_RESOLVE_SECS` (default 6s, poll every `DEV_FAKE_GENERATION_POLL_SECS`, default 2s) with a placeholder result copied from `test_data/`. Prompts containing `simulate_artcraft_failure` (or `test_artcraft_failure`) fail instead, optionally with a `FrontendFailureCategory` word in the prompt (e.g. `simulate_artcraft_failure face_not_detected`). Hard-disabled outside `SERVER_ENVIRONMENT=Development` |
| `CDN_BASE_URL` | Overrides the compiled-in CDN hosts in `cdn_link.rs` for all media URLs in API responses. Must be scheme+host only (no path) — set to `http://localhost:12345` so media resolves against the backend itself |
| `LOCAL_MEDIA_ROOT` | Directory the backend serves at `/media` (dev-only actix-files mount), mirroring the public bucket layout: object path `/media/{…}` lives at `{LOCAL_MEDIA_ROOT}/media/{…}`. Also switches all three `LegacyBucketClient`s (public/private/gc) into a **local filesystem mode**, so every media upload endpoint writes here instead of S3/R2 — uploads work with dummy cloud creds. The bootstrap points it at `.devstack/media/` |
With all three set, the full loop works offline: submit → pending job →
polling (`/v1/jobs/session`) → success with a `media_files` row whose URL the
backend itself serves — or a failed job with a real failure category. Credits
are genuinely deducted from the seeded wallet (no refund on fake results —
the demo wallet is large enough not to care).
### Auth on localhost
- `POST /v1/login` takes `{"username_or_email": "...", "password": "..."}`;
success returns `{"success": true, "signed_session": "<jwt>"}` **and** a
`session` cookie.
- With the checked-in dev config (`COOKIE_DOMAIN=.jungle.horse`), the session
cookie is issued non-Secure and host-only (the Domain attribute is never
set), so plain-HTTP localhost login works as-is. The frontend additionally
stores `signed_session` in localStorage and sends it as a `session` header.
- Dev-mode CORS allows any `localhost`/`127.0.0.1` origin on any port, with
credentials.
- Health endpoint for scripts/CI: `GET /_status`.
### Build
- Compile with `SQLX_OFFLINE=true` (uses the checked-in `.sqlx/` cache; 483
queries). Without it, the sqlx macros open a **live DB connection at compile
time** using `DATABASE_URL` from the root `.env`. `SQLX_OFFLINE` has no
runtime effect — ignore older notes saying not to set it when running.
- First build takes several minutes; the crate has ~370 pre-existing warnings.
## Database: migrations and seed data
- 281 Diesel migrations in `_database/sql/migrations/` (`diesel.toml` at repo
root points there). Run from the repo root: `diesel migration run`.
`"Encountered unknown type for Mysql: enum"` warnings are harmless.
`migrations_squashed/` is documentation only — not executable.
- Seed data: `_database/sql/seed/sql/system_roles.sql` (roles `user`, `mod`,
`admin`) and `user_badges.sql`. **The `user` role is mandatory** — account
creation hardcodes `user_role_slug='user'`. The legacy wrapper
`bootstrap_inserts_roles_etc.sh` must run with cwd `_database/sql/` and is
not idempotent; the bootstrap script applies the same SQL files with
idempotency guards instead.
- Demo user: created through **`POST /v1/create_account`** (no captcha, no
invite code, no email-verification gate; password just needs 6+ chars). This
exercises the real signup path instead of hand-rolled SQL. Gotcha: usernames
like `demo`, `admin`, `test`, `dev`, `artcraft` are reserved
(`includes/binary_includes/usernames/reserved_usernames.txt`) — the default
demo user is `localdev1`. The Rust seeder
(`cargo run --bin dev-database-seed`) exists but hard-requires a
`.env-secrets` file and also seeds model weights, so the HTTP path is
preferred.
- Demo credits: signup creates **no wallet**, and the webapp gates the
generate button on wallet balance — so `seed_demo_user` also seeds an
`artcraft`-namespace wallet with `DEMO_CREDITS` (default 100,000) banked
credits, plus matching ledger rows. Idempotent: re-runs top the balance
back up to `DEMO_CREDITS` but never lower it. (Raw SQL, because the only
real credit-granting paths are Stripe webhooks and moderator tooling.)
- Demo gallery: `seed_demo_user` also seeds a handful of `media_files` image
rows (from `test_data/image/`) with their objects copied under
`.devstack/media/`, so the library isn't empty on a fresh stack. Rows mirror
what a real image upload writes (`media_class='image'` matters — the
library filters on it, and the column default `'unknown'` is invisible in
every tab).
## Frontend: `artcraft-webapp`
- Nx workspace at repo-root `frontend/` (npm workspaces; `npm install` from
`frontend/`). The FontAwesome Pro registry token is committed in
`frontend/.npmrc` — no manual auth step.
- Serve: `npx nx dev artcraft-webapp` → `http://localhost:4201`. No lib
pre-build is needed in dev (libs resolve to TS source via the
`"development"` export condition); production `nx build` does need lib
builds.
- **In dev the webapp already targets the local backend**: `src/main.tsx`
calls `StorytellerApiHostStore.setDevelopment()` under
`import.meta.env.DEV`, pointing all API traffic at
`http://localhost:12345`. (The adjacent comment claiming this is disabled is
stale — the call is live, and it bypasses the Vite `/v1` →
`api.storyteller.ai` proxy.) So: local backend running → fully local stack;
local backend **not** running → dev webapp is broken, not silently on prod.
- Optional: `VITE_GOOGLE_CLIENT_ID` in `frontend/apps/artcraft-webapp/.env`
enables the Google SSO button; email/password login needs nothing.
- Other dev ports if you run more apps: website 4200, dashboard 4202 (separate
pnpm app, excluded from Nx), video-info-website 4250, genhub 4300, Tauri
frontend 5173.
## What still touches production (known, deliberate seams)
Cleaning these up is the next phase of the infra work; the bootstrap does not
change runtime behavior:
| Seam | Today |
|-----------------------------------------------|--------------------------------------------------------------|
| Media/CDN URLs in API responses | **Closed** when `CDN_BASE_URL` + `LOCAL_MEDIA_ROOT` are set (the bootstrap default): media URLs point at the backend's own `/media` mount. Unset, the compiled-in `cdn_link.rs` constants still point at the public `pub-….r2.dev` bucket |
| Media uploads | **Closed** when `LOCAL_MEDIA_ROOT` is set: the bucket clients run in local filesystem mode and uploads land under `.devstack/media/`. Unset (or non-Development), uploads still 500 with dummy creds |
| Vite `/v1` proxy in webapp | **Closed**: now targets `http://localhost:12345` (it was already bypassed in dev by `setDevelopment()`, but the fallback could silently hit production). The `/__cdn` dev proxy also no longer intercepts localhost media (it assumed https and broke pagescene GLB/splat loads against a local backend) |
| `GetCdnOrigin` (pagescene asset thumbnails) | **Closed** for local dev: returns the API host when it's localhost (the local backend serves media itself), production CDN otherwise; the pagescene adapter also skips the Cloudflare `/cdn-cgi/image` resize path on localhost. `enums/Api.ts` `API_TARGETS` and `SegmentationApi` still hardcode production hosts but are dead code (not exported/reachable from the webapp); moodboard never had a hardcoded host — it consumes `media_links` directly |
| Elasticsearch-backed search | `search_featured` / `search_session` media files, `weights/search`, legacy `tts/search` fail without a local ES + `elasticsearch-cli` reindex (optional) |
## CI usage
The same bootstrap works on a fresh Ubuntu runner/container:
```bash
export MYSQL_ROOT_PASSWORD=root # GitHub Actions' preinstalled MySQL uses password auth
./script/bootstrap/bootstrap_dev_stack.sh --yes
./script/bootstrap/run_backend_dev.sh &
./script/bootstrap/seed_demo_user.sh # waits for /_status
./script/bootstrap/dev_stack_doctor.sh
```
`--yes` answers all prompts; `--skip-frontend` / `--skip-rust-build` /
`--skip-packages` slice the work for focused jobs.
## Verification checklist
- `./script/bootstrap/dev_stack_doctor.sh` exits 0 with no FAILs.
- `POST /v1/login` with the demo user returns `success: true` plus a
`signed_session` (done automatically by `seed_demo_user.sh`).
- Webapp at `http://localhost:4201` can log in as `localdev1`, has credits in
the top bar, and sees the seeded images in the library.
- A generation submit returns `{"success": true, "inference_job_token": …}`,
shows as pending for a few seconds, then completes with a placeholder
result (`DEV_FAKE_GENERATION`) — no external provider is contacted. A
prompt containing `simulate_artcraft_failure` fails with a real error
toast/category instead.
- Uploading an image (e.g. a reference image in the prompt box) succeeds and
the file lands under `.devstack/media/` — no S3/R2 credentials involved.
- Nothing hits `api.storyteller.ai` or the r2.dev CDN during the above
(verify in the browser network tab — media URLs should point at
`http://localhost:12345/media/…`).
## Corrections to older docs
- `crates/service/web/storyteller_web/CLAUDE.md`: the repo-root `.env` is
*not* auto-loaded by the server, and `SQLX_OFFLINE=true` is fine (and
recommended) when running — it only affects compile-time query checking.
- `_docs/dev_setup_server.md`: `./script/dev_mysql_connect.sh` doesn't exist —
it's `./script/dev_database_connect.sh`.
- `diesel.toml`'s comment about needing an unreleased diesel CLI is stale —
any current `diesel_cli` works.
- `TESTING_INFRA_HANDOFF.md` (untracked): the server env var is `MYSQL_URL`,
not `DATABASE_URL`; migration count is 281; crate path is
`crates/service/web/storyteller_web`.
@@ -1,5 +1,5 @@
use std::error::Error;
use std::path::Path;
use std::path::{Path, PathBuf};
use std::time::Duration;
use anyhow::anyhow;
@@ -18,11 +18,26 @@ use errors::AnyhowResult;
#[derive(Clone)]
pub struct LegacyBucketClient {
bucket: Bucket,
backend: LegacyBucketBackend,
/// If set, put all files under this root path.
optional_bucket_root: Option<String>,
}
#[derive(Clone)]
enum LegacyBucketBackend {
S3 {
bucket: Bucket,
},
/// Development-only: objects live on the local filesystem under `root`,
/// with the rooted object path (e.g. "/media/a/b/…/file.jpg") mapped
/// segment-by-segment beneath it — the same layout the dev server's
/// static /media mount serves. Lets the local dev stack run every upload
/// endpoint without S3/R2 credentials.
LocalDir {
root: PathBuf,
bucket_name: String,
},
}
#[derive(Debug)]
pub enum LegacyBucketClientError {
@@ -40,10 +55,6 @@ impl Error for LegacyBucketClientError {}
impl LegacyBucketClient {
pub fn bucket_name(&self) -> String {
self.bucket.name().to_string()
}
pub fn create(
access_key: &str,
secret_key: &str,
@@ -85,15 +96,34 @@ impl LegacyBucketClient {
let optional_bucket_root = optional_bucket_root.map(|s| s.to_string());
Ok(Self {
bucket: *bucket, // NB: We don't need to keep this boxed on the heap.
backend: LegacyBucketBackend::S3 {
bucket: *bucket, // NB: We don't need to keep this boxed on the heap.
},
optional_bucket_root,
})
}
fn get_rooted_object_name(&self, object_name: &str) -> String {
match &self.optional_bucket_root {
None => object_name.to_string(),
Some(root) => format!("{}/{}", root, object_name),
/// Development-only: a "bucket" backed by a local directory (see
/// `LegacyBucketBackend::LocalDir`). `bucket_name` is only reported back
/// via `bucket_name()`; no cloud service is ever contacted.
pub fn create_local(
local_root: impl Into<PathBuf>,
bucket_name: &str,
optional_bucket_root: Option<&str>,
) -> Self {
Self {
backend: LegacyBucketBackend::LocalDir {
root: local_root.into(),
bucket_name: bucket_name.to_string(),
},
optional_bucket_root: optional_bucket_root.map(|s| s.to_string()),
}
}
pub fn bucket_name(&self) -> String {
match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket.name().to_string(),
LegacyBucketBackend::LocalDir { bucket_name, .. } => bucket_name.clone(),
}
}
@@ -103,7 +133,14 @@ impl LegacyBucketClient {
let object_name = self.get_rooted_object_name(object_name);
debug!("Rooted filename for bucket: {}", object_name);
let response = self.bucket.put_object(&object_name, bytes).await?;
let bucket = match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket,
LegacyBucketBackend::LocalDir { root, .. } => {
return write_local_object(root, &object_name, bytes).await;
},
};
let response = bucket.put_object(&object_name, bytes).await?;
let body_bytes = response.bytes();
let code = response.status_code();
@@ -124,7 +161,17 @@ impl LegacyBucketClient {
info!("Filename for bucket: {}", object_name);
let object_name = self.get_rooted_object_name(object_name);
info!("Rooted filename for bucket: {}", object_name);
let response = self.bucket.put_object_with_content_type(&object_name, bytes, content_type).await?;
let bucket = match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket,
LegacyBucketBackend::LocalDir { root, .. } => {
// Content type is derived from the file extension when the dev
// static mount serves it back; nothing to store here.
return write_local_object(root, &object_name, bytes).await;
},
};
let response = bucket.put_object_with_content_type(&object_name, bytes, content_type).await?;
let body_bytes = response.bytes();
let code = response.status_code();
info!("upload code: {}", code);
@@ -145,10 +192,17 @@ impl LegacyBucketClient {
let object_name = self.get_rooted_object_name(object_name);
info!("Rooted filename for bucket: {}", object_name);
let response = self.bucket.put_object_with_content_type(&object_name, bytes, content_type).await;
let bucket = match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket,
LegacyBucketBackend::LocalDir { root, .. } => {
return write_local_object(root, &object_name, bytes).await;
},
};
let response = bucket.put_object_with_content_type(&object_name, bytes, content_type).await;
if let Err(err) = &response {
error!("S3 Upload Error for bucket name {}: {:?}", &self.bucket.name, err);
error!("S3 Upload Error for bucket name {}: {:?}", bucket.name, err);
}
let response = response?;
@@ -204,13 +258,23 @@ impl LegacyBucketClient {
info!("Uploading with content type...");
#[allow(deprecated)]
self.upload_file_with_content_type(&object_path_str, &buffer, content_type).await
}
pub async fn download_file(&self, path: &str) -> anyhow::Result<Vec<u8>> {
info!("downloading from bucket: {}", path);
let response = self.bucket.get_object(path).await?;
let bucket = match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket,
LegacyBucketBackend::LocalDir { root, .. } => {
let disk_path = local_object_disk_path(root, path)?;
return tokio::fs::read(&disk_path).await
.map_err(|err| anyhow!("File not found in local bucket ({}): {}", disk_path.display(), err));
},
};
let response = bucket.get_object(path).await?;
let bytes = response.bytes();
let code = response.status_code();
@@ -234,24 +298,34 @@ impl LegacyBucketClient {
.map(|s| s.to_string())
.ok_or(anyhow!("could not convert object path to string"))?;
let bucket = match &self.backend {
LegacyBucketBackend::S3 { bucket } => bucket,
LegacyBucketBackend::LocalDir { root, .. } => {
let disk_path = local_object_disk_path(root, &object_path_str)?;
tokio::fs::copy(&disk_path, filesystem_path.as_ref()).await
.map_err(|err| anyhow!("File not found in local bucket ({}): {}", disk_path.display(), err))?;
return Ok(());
},
};
info!("creating file for bucket download: {:?}", filesystem_path.as_ref());
let mut output_file = File::create(filesystem_path).await?;
let result = self.bucket.get_object_to_writer(&object_path_str, &mut output_file).await;
let result = bucket.get_object_to_writer(&object_path_str, &mut output_file).await;
info!("downloading from bucket (named '{}'), path: {}", &self.bucket.name, &object_path_str);
info!("downloading from bucket (named '{}'), path: {}", bucket.name, &object_path_str);
let status_code = match result {
Ok(status_code) => status_code,
Err(err) => {
return bail!("Error downloading from bucket (named '{}'): {:?}", &self.bucket.name, err)
return bail!("Error downloading from bucket (named '{}'): {:?}", bucket.name, err)
}
};
match status_code {
404 => {
error!("File not found in bucket (named '{}'), path: {}", &self.bucket.name, &object_path_str);
error!("File not found in bucket (named '{}'), path: {}", bucket.name, &object_path_str);
bail!("File not found in bucket: {}", &object_path_str)
},
_ => {
@@ -261,4 +335,70 @@ impl LegacyBucketClient {
Ok(())
}
fn get_rooted_object_name(&self, object_name: &str) -> String {
match &self.optional_bucket_root {
None => object_name.to_string(),
Some(root) => format!("{}/{}", root, object_name),
}
}
}
/// Map a rooted object path (e.g. "/media/a/b/…/file.jpg") to a disk path
/// under `root`, segment by segment. This is the single source of truth for
/// the local-dir layout — the dev server's static /media mount and the fake
/// generation resolver must agree with it (disk = root + rooted path).
pub fn local_object_disk_path(root: &Path, rooted_object_path: &str) -> anyhow::Result<PathBuf> {
let mut path = root.to_path_buf();
for segment in rooted_object_path.split('/').filter(|s| !s.is_empty()) {
// Object names can come from user-controlled archives (e.g. PMX zip
// entries); never let them escape the root.
if segment == ".." || segment.contains('\\') {
bail!("refusing suspicious object path segment: {}", rooted_object_path);
}
path.push(segment);
}
Ok(path)
}
async fn write_local_object(root: &Path, rooted_object_path: &str, bytes: &[u8]) -> anyhow::Result<()> {
let disk_path = local_object_disk_path(root, rooted_object_path)?;
if let Some(parent) = disk_path.parent() {
tokio::fs::create_dir_all(parent).await?;
}
tokio::fs::write(&disk_path, bytes).await?;
info!("Wrote local bucket object: {}", disk_path.display());
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
mod local_object_disk_path_tests {
use super::*;
#[test]
fn maps_rooted_path_under_root() {
let path = local_object_disk_path(Path::new("/data"), "/media/a/b/file.jpg").unwrap();
assert_eq!(path, Path::new("/data").join("media").join("a").join("b").join("file.jpg"));
}
#[test]
fn ignores_leading_slash_and_empty_segments() {
let with_slash = local_object_disk_path(Path::new("/data"), "/media//x").unwrap();
let without_slash = local_object_disk_path(Path::new("/data"), "media/x").unwrap();
assert_eq!(with_slash, without_slash);
}
#[test]
fn rejects_parent_dir_traversal() {
assert!(local_object_disk_path(Path::new("/data"), "/media/../../etc/passwd").is_err());
}
#[test]
fn rejects_backslash_segments() {
assert!(local_object_disk_path(Path::new("/data"), "/media/a\\b").is_err());
}
}
}
@@ -0,0 +1,64 @@
use sqlx::{MySqlPool, Row};
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
use tokens::tokens::prompts::PromptToken;
use tokens::tokens::users::UserToken;
/// A pending job created by the DEV_FAKE_GENERATION short-circuit
/// (external_third_party_id starts with `fake_`), joined with its prompt text
/// so the dev resolver can honor the synthetic-failure prompt triggers.
pub struct PendingFakeGenerationJob {
pub job_token: InferenceJobToken,
/// e.g. "image_generation", "video_generation".
pub inference_category: String,
pub maybe_prompt_token: Option<PromptToken>,
pub maybe_positive_prompt: Option<String>,
pub maybe_creator_user_token: Option<UserToken>,
}
/// List pending fake-generation jobs older than `min_age_seconds`, oldest
/// first. Development-only: production never writes `fake_` external ids.
///
/// NB: runtime query (not the compile-time macro) because this dev-only SELECT
/// isn't in the sqlx offline cache.
pub async fn list_pending_fake_generation_jobs(
mysql_pool: &MySqlPool,
min_age_seconds: u32,
limit: u32,
) -> Result<Vec<PendingFakeGenerationJob>, sqlx::Error> {
let rows = sqlx::query(
r#"
SELECT
j.token AS job_token,
j.inference_category,
j.maybe_prompt_token,
j.maybe_creator_user_token,
p.maybe_positive_prompt
FROM generic_inference_jobs AS j
LEFT JOIN prompts AS p
ON p.token = j.maybe_prompt_token
WHERE j.status = 'pending'
AND j.maybe_external_third_party_id LIKE 'fake\_%'
AND j.created_at <= NOW() - INTERVAL ? SECOND
ORDER BY j.created_at ASC
LIMIT ?
"#)
.bind(min_age_seconds)
.bind(limit)
.fetch_all(mysql_pool)
.await?;
let jobs = rows.into_iter()
.map(|row| PendingFakeGenerationJob {
job_token: InferenceJobToken::new(row.get::<String, _>("job_token")),
inference_category: row.get::<String, _>("inference_category"),
maybe_prompt_token: row.get::<Option<String>, _>("maybe_prompt_token")
.map(PromptToken::new),
maybe_positive_prompt: row.get::<Option<String>, _>("maybe_positive_prompt"),
maybe_creator_user_token: row.get::<Option<String>, _>("maybe_creator_user_token")
.map(UserToken::new),
})
.collect();
Ok(jobs)
}
@@ -8,6 +8,7 @@ pub mod job_status;
pub mod kill_generic_inference_jobs;
pub mod list_user_jobs_for_moderation;
pub mod kill_jobs_in_development;
pub mod list_pending_fake_generation_jobs;
pub mod list_session_jobs;
pub mod mark_generic_inference_job_cancelled_by_user;
pub mod mark_generic_inference_job_successfully_done_by_token;
+4 -4
View File
@@ -75,23 +75,23 @@ curl -s -c /tmp/cookies.txt -X POST http://localhost:12345/v1/login \
-H "Content-Type: application/json" \
-d '{"username_or_email":"newuser4","password":"newuser4"}'
# 2. Generate video (GmiCloud Seedance 2.0 Fast Global):
# 2. Generate video (Seedance 2.0 Fast):
# Requires a UUID-format idempotency_token.
curl -s -b /tmp/cookies.txt -X POST http://localhost:12345/v1/omni_gen/generate/video \
-H "Content-Type: application/json" \
-d '{
"model": "seedance_2p0_fast_g",
"model": "seedance_2p0_fast",
"prompt": "a corgi running through a field",
"duration_seconds": 5,
"aspect_ratio": "wide_sixteen_by_nine",
"idempotency_token": "a1b2c3d4-e5f6-7890-abcd-ef1234567890"
}'
# 3. Generate video (GmiCloud Seedance 2.0 Global, 720p):
# 3. Generate video (Seedance 2.0, 720p):
curl -s -b /tmp/cookies.txt -X POST http://localhost:12345/v1/omni_gen/generate/video \
-H "Content-Type: application/json" \
-d '{
"model": "seedance_2p0_g",
"model": "seedance_2p0",
"prompt": "a corgi running through a field",
"duration_seconds": 5,
"resolution": "seven_twenty_p",
@@ -167,6 +167,7 @@ tempfile.workspace = true
# https://github.com/durch/rust-s3/pull/254
rust-s3.workspace = true
actix-files.workspace = true
actix-http.workspace = true
actix-multipart.workspace = true
actix-service.workspace = true
@@ -9,6 +9,34 @@ const STORYTELLER_CDN_STR: &str = "https://cdn-2.fakeyou.com";
const FAKEYOU_DEVELOPMENT_CDN_STR: &str = "https://pub-c8a4a5bdbdb048f286b77bdf9f786ff2.r2.dev";
const STORYTELLER_DEVELOPMENT_CDN_STR: &str = "https://pub-c8a4a5bdbdb048f286b77bdf9f786ff2.r2.dev";
// Runtime override for the CDN base, taking precedence over every constant
// above for all domains and environments. The fully-local dev stack sets
// CDN_BASE_URL=http://localhost:12345 so media URLs resolve against the
// backend's own static /media mount (see add_service_routes) instead of the
// shared r2.dev bucket. Unset (production, and dev-against-remote) keeps the
// compiled-in hosts.
//
// The override must be scheme+host only: some consumers append the rooted
// object path with Url::set_path, which would silently drop a path component
// in the base, so a base with a path is rejected loudly instead.
// Leaked once so get_cdn_host can stay &'static str.
static CDN_BASE_URL_OVERRIDE_STR: Lazy<Option<&'static str>> = Lazy::new(|| {
let base = std::env::var("CDN_BASE_URL")
.ok()
.map(|s| s.trim().trim_end_matches('/').to_string())
.filter(|s| !s.is_empty())?;
let url = Url::parse(&base).expect("CDN_BASE_URL must be a valid absolute URL");
if url.path() != "/" && !url.path().is_empty() {
panic!("CDN_BASE_URL must not include a path component (got '{}'): \
media URL builders overwrite the path", url.path());
}
Some(&*Box::leak(base.into_boxed_str()))
});
static CDN_BASE_URL_OVERRIDE: Lazy<Option<Url>> = Lazy::new(|| {
CDN_BASE_URL_OVERRIDE_STR.map(|s| Url::parse(s).expect("validated above"))
});
const FAKEYOU_CDN: Lazy<Url> = Lazy::new(|| Url::parse(FAKEYOU_CDN_STR)
.expect("should never fail"));
@@ -25,6 +53,9 @@ const STORYTELLER_DEVELOPMENT_CDN: Lazy<Url> = Lazy::new(|| Url::parse(STORYTELL
// TODO(bt,2025-01-31): Perhaps this should be config driven and configurable at runtime instead of hardcoded.
pub fn get_cdn_host(media_domain: MediaDomain, server_environment: ServerEnvironment) -> &'static str {
if let Some(base) = *CDN_BASE_URL_OVERRIDE_STR {
return base;
}
match (media_domain, server_environment) {
(MediaDomain::FakeYou, ServerEnvironment::Development) => FAKEYOU_DEVELOPMENT_CDN_STR,
(MediaDomain::FakeYou, ServerEnvironment::Production) => FAKEYOU_CDN_STR,
@@ -34,6 +65,9 @@ pub fn get_cdn_host(media_domain: MediaDomain, server_environment: ServerEnviron
}
pub fn new_cdn_url(media_domain: MediaDomain, server_environment: ServerEnvironment) -> Url {
if let Some(url) = CDN_BASE_URL_OVERRIDE.as_ref() {
return url.clone();
}
match (media_domain, server_environment) {
(MediaDomain::FakeYou, ServerEnvironment::Development) => FAKEYOU_DEVELOPMENT_CDN.clone(),
(MediaDomain::FakeYou, ServerEnvironment::Production) => FAKEYOU_CDN.clone(),
@@ -7,13 +7,14 @@ use artcraft_router::client::router_client::RouterClient;
use artcraft_router::client::router_fal_client::RouterFalClient;
use artcraft_router::client::router_seedance2pro_client::RouterSeedance2ProClient;
use artcraft_router::generate::generate_image::generate_image_request_builder::GenerateImageRequestBuilder;
use artcraft_router::generate::generate_image::generate_image_response::GenerateImageResponse;
use artcraft_router::generate::generate_image::generate_image_response::{FalImageResponsePayload, GenerateImageResponse};
use artcraft_router::generate::generate_image::image_generation_draft_context::ImageGenerationDraftContext;
use artcraft_router::generate::generate_image::image_generation_draft_or_request::ImageGenerationDraftOrRequest;
use artcraft_router::generate::generate_image::image_generation_request::ImageGenerationRequest;
use seedance2pro_client::creds::seedance2pro_session::Seedance2ProSession;
use tokens::tokens::generic_inference_jobs::InferenceJobToken;
use tokens::tokens::users::UserToken;
use uuid_utils::uuid::generate_random_uuid;
use crate::billing::wallets::attempt_wallet_deduction::attempt_wallet_deduction_else_common_web_error;
use crate::http_server::common_responses::common_web_error::CommonWebError;
@@ -173,6 +174,21 @@ async fn finalize_and_generate(
server_state: &ServerState,
resolved_media: &MediaFilesAsCdnUrlListAndMap,
) -> Result<GenerateImageResponse, CommonWebError> {
// Dev-only: skip the external provider entirely and hand back a synthetic
// Fal enqueue. The handler then inserts a normal pending job (external id
// `fake_…`), which the dev fake-job resolver thread later completes.
if server_state.dev_fake_generation {
let fake_request_id = format!("fake_{}", generate_random_uuid());
info!("DEV_FAKE_GENERATION: skipping image provider call, request_id={}", fake_request_id);
return Ok(GenerateImageResponse::Fal(FalImageResponsePayload {
request_id: Some(fake_request_id),
gateway_request_id: None,
maybe_status_url: None,
maybe_response_url: None,
maybe_outbound_request: None,
}));
}
let provider = draft_or_request.get_provider();
let client = build_router_client(provider, server_state)?;
@@ -6,12 +6,13 @@ use log::{error, info, warn};
use artcraft_router::api::router_video_model::RouterVideoModel;
use artcraft_router::api::router_provider::RouterProvider;
use artcraft_router::generate::generate_video::generate_video_request_builder::GenerateVideoRequestBuilder;
use artcraft_router::generate::generate_video::generate_video_response::GenerateVideoResponse;
use artcraft_router::generate::generate_video::generate_video_response::{FalVideoResponsePayload, GenerateVideoResponse};
use artcraft_router::generate::generate_video::video_generation_draft_context::VideoGenerationDraftContext;
use artcraft_router::generate::generate_video::video_generation_draft_or_request::VideoGenerationDraftOrRequest;
use tokens::tokens::characters::CharacterToken;
use tokens::tokens::media_files::MediaFileToken;
use tokens::tokens::users::UserToken;
use uuid_utils::uuid::generate_random_uuid;
use crate::http_server::common_responses::common_web_error::CommonWebError;
use crate::http_server::endpoint_helpers::refund_wallet_after_api_failure::refund_wallet_after_api_failure;
@@ -220,6 +221,22 @@ async fn upload_and_generate(
kinovi_account: KinoviAccount,
) -> Result<GenerateVideoResponse, CommonWebError> {
// Dev-only: skip the external provider entirely (including draft media
// uploads) and hand back a synthetic Fal enqueue. The handler then inserts
// a normal pending job (external id `fake_…`), which the dev fake-job
// resolver thread later completes.
if server_state.dev_fake_generation {
let fake_request_id = format!("fake_{}", generate_random_uuid());
info!("DEV_FAKE_GENERATION: skipping video provider call, request_id={}", fake_request_id);
return Ok(GenerateVideoResponse::Fal(FalVideoResponsePayload {
request_id: Some(fake_request_id),
gateway_request_id: None,
maybe_status_url: None,
maybe_response_url: None,
maybe_outbound_request: None,
}));
}
let provider = draft_or_request.get_provider();
let client = build_router_client(provider, server_state, kinovi_account)?;
@@ -4,6 +4,7 @@ use actix_web::dev::{ServiceRequest, ServiceResponse};
use actix_web::error::Error;
use actix_web::App;
use log::{info, warn};
use server_environment::ServerEnvironment;
use crate::http_server::routes::application_routes::add_application_routes::add_application_routes;
@@ -22,13 +23,45 @@ pub fn add_routes<T, B> (app: App<T>, server_environment: ServerEnvironment) ->
>,
{
let mut app = app;
app = add_legacy_routes(app); // various legacy routes, mostly deprecated
app = add_application_routes(app); // Primary product service area routes
app = add_local_media_route(app, server_environment); // Dev-only static media (fully-local stack)
app = add_service_routes(app); // Essential service routes (status, health, info, etc.)
app
}
/// Dev-only: serve media files from a local directory at /media, mirroring the
/// public bucket's layout, so a fully-local stack (CDN_BASE_URL pointed at this
/// server) can render seeded and fake-generated media without any cloud bucket.
/// LOCAL_MEDIA_ROOT is the bucket root: an object with rooted path
/// /media/{...} is read from {LOCAL_MEDIA_ROOT}/media/{...}.
/// No-op in production and when LOCAL_MEDIA_ROOT is unset.
fn add_local_media_route<T, B> (app: App<T>, server_environment: ServerEnvironment) -> App<T>
where
B: MessageBody,
T: ServiceFactory<
ServiceRequest,
Config = (),
Response = ServiceResponse<B>,
Error = Error,
InitError = (),
>,
{
if !server_environment.is_development() {
return app;
}
let root = match std::env::var("LOCAL_MEDIA_ROOT") {
Ok(root) if !root.trim().is_empty() => root,
_ => return app,
};
let media_dir = std::path::Path::new(&root).join("media");
if let Err(err) = std::fs::create_dir_all(&media_dir) {
// Fall back to remote CDN behavior rather than refusing to boot.
warn!("LOCAL_MEDIA_ROOT set but '{}' is not creatable ({}); /media not mounted", media_dir.display(), err);
return app;
}
info!("Dev local media: serving {} at /media", media_dir.display());
app.service(actix_files::Files::new("/media", media_dir))
}
@@ -55,6 +55,7 @@ use crate::startup::setup_disabled_endpoints::read_disabled_endpoints;
use crate::startup::setup_metrics::build_metrics;
use crate::state::server_state::ServerState;
use crate::threads::db_health_checker_thread::db_health_checker_thread::db_health_checker_thread;
use crate::threads::dev_fake_generation_resolver_thread::dev_fake_generation_resolver_thread;
use crate::threads::poll_ip_banlist_thread::poll_ip_bans;
use crate::threads::poll_model_token_info_thread::poll_model_token_info_thread;
@@ -148,6 +149,16 @@ async fn main() -> AnyhowResult<()> {
poll_model_token_info_thread(model_token_info_cache_clone, mysql_pool_clone).await;
});
if server_state.dev_fake_generation {
info!("Spawning dev fake-generation resolver thread (DEV_FAKE_GENERATION=true).");
let mysql_pool_clone = server_state.mysql_pool.clone();
tokio_runtime.spawn(async {
dev_fake_generation_resolver_thread(mysql_pool_clone).await;
});
}
// ==================== Metrics worker ==================== //
let (metrics_collector, maybe_metrics_worker) =
@@ -9,7 +9,7 @@ use cloud_storage::legacy_bucket_client::LegacyBucketClient;
use elasticsearch::http::transport::Transport;
use elasticsearch::Elasticsearch;
use errors::AnyhowResult;
use log::info;
use log::{info, warn};
use memory_caching::arc_ttl_sieve::ArcTtlSieve;
use memory_caching::single_item_ttl_cache::SingleItemTtlCache;
use mysql_queries::mediators::badge_granter::BadgeGranter;
@@ -124,20 +124,38 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
let s3_compatible_endpoint_url = easyenv::get_env_string_or_default("S3_COMPATIBLE_ENDPOINT_URL", "https://storage.googleapis.com");
let bucket_timeout = easyenv::get_env_duration_seconds_or_default("BUCKET_TIMEOUT_SECONDS", Duration::from_secs(60 * 5));
let private_bucket_client = LegacyBucketClient::create(
&access_key, &secret_key, &region_name, &private_bucket_name,
&s3_compatible_endpoint_url, None, Some(bucket_timeout),
)?;
// NB: read here (before the bucket clients) rather than at first use below —
// the fully-local dev storage mode is gated on the environment.
let server_environment = ServerEnvironment::from_str(&easyenv::get_env_string_required("SERVER_ENVIRONMENT")?)
.ok_or(anyhow!("invalid server environment"))?;
let public_bucket_client = LegacyBucketClient::create(
&access_key, &secret_key, &region_name, &public_bucket_name,
&s3_compatible_endpoint_url, None, Some(bucket_timeout),
)?;
// Dev-only: when LOCAL_MEDIA_ROOT is set (the fully-local dev stack), back
// all bucket "uploads" with the local filesystem instead of S3/R2. Objects
// land under the same directory the dev server serves at /media, so
// uploaded media is immediately viewable via the CDN_BASE_URL override.
// Production and dev-against-real-buckets are unaffected (env-gated).
let maybe_local_media_root = if server_environment.is_development() {
std::env::var("LOCAL_MEDIA_ROOT").ok().filter(|root| !root.trim().is_empty())
} else {
None
};
if let Some(root) = maybe_local_media_root.as_deref() {
warn!("Local filesystem storage mode is ON: bucket uploads write under {} (LOCAL_MEDIA_ROOT).", root);
}
let auto_gc_bucket_client = LegacyBucketClient::create(
&access_key, &secret_key, &region_name, &gc_enabled_public_bucket_name,
&s3_compatible_endpoint_url, None, Some(bucket_timeout),
)?;
let make_bucket_client = |bucket_name: &str| -> AnyhowResult<LegacyBucketClient> {
match maybe_local_media_root.as_deref() {
Some(root) => Ok(LegacyBucketClient::create_local(root, bucket_name, None)),
None => LegacyBucketClient::create(
&access_key, &secret_key, &region_name, bucket_name,
&s3_compatible_endpoint_url, None, Some(bucket_timeout),
),
}
};
let private_bucket_client = make_bucket_client(&private_bucket_name)?;
let public_bucket_client = make_bucket_client(&public_bucket_name)?;
let auto_gc_bucket_client = make_bucket_client(&gc_enabled_public_bucket_name)?;
// In-Memory Caches
let voice_list_cache_ttl = easyenv::get_env_duration_seconds_or_default("VOICE_LIST_CACHE_TTL_SECONDS", Duration::from_secs(60));
@@ -175,9 +193,6 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
let health_check_status = HealthCheckStatus::new();
let server_environment = ServerEnvironment::from_str(&easyenv::get_env_string_required("SERVER_ENVIRONMENT")?)
.ok_or(anyhow!("invalid server environment"))?;
let (pager, pager_worker, paging_flags) = build_pager(server_environment, server_hostname);
let service_feature_flags = setup_static_feature_flags(paging_flags)?;
@@ -192,6 +207,15 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
let inference_providers = setup_inference_providers()?;
// Dev-only fake generation (see the field docs on ServerState). The
// environment gate is enforced here so the flag can never arm in production
// even if the env var leaks into a deployed config.
let dev_fake_generation = server_environment.is_development()
&& easyenv::get_env_bool_or_default("DEV_FAKE_GENERATION", false);
if dev_fake_generation {
warn!("DEV_FAKE_GENERATION is ON: omni_gen requests will not contact real providers.");
}
let resend_api_key = easyenv::get_env_string_required("RESEND_API_KEY")?;
let startup_time = Utc::now();
@@ -213,6 +237,7 @@ pub async fn setup_dependencies(server_hostname: &str) -> AnyhowResult<SetupResu
hostname: server_hostname.to_string(),
startup_time,
server_environment,
dev_fake_generation,
flags: service_feature_flags,
third_party_url_redirector,
health_check_status,
@@ -57,6 +57,13 @@ pub struct ServerState {
/// Knowing if we're in production will allow us to turn off development-only functionalities.
pub server_environment: ServerEnvironment,
/// Dev-only (DEV_FAKE_GENERATION=true, forced off outside Development):
/// omni_gen pipelines skip the external provider call and insert a pending
/// job with a synthetic `fake_` external id, which the dev fake-job
/// resolver thread later completes or fails. Lets a local stack with dummy
/// provider secrets exercise the full submit -> pending -> resolve flow.
pub dev_fake_generation: bool,
/// Feature flags will allow us to restart the service with different conditions embedded in the code.
pub flags: StaticFeatureFlags,
@@ -0,0 +1,194 @@
use std::path::Path;
use std::time::Duration;
use cloud_storage::legacy_bucket_client::local_object_disk_path;
use log::{error, info};
use sqlx::MySqlPool;
use bucket_paths::legacy::typified_paths::public::media_files::bucket_file_path::MediaFileBucketPath;
use bucket_paths::path_conventions::video_thumbnail_suffixes::{VIDEO_ANIMATED_GIF_THUMBNAIL_SUFFIX, VIDEO_STATIC_JPG_THUMBNAIL_SUFFIX};
use enums::by_table::generic_inference_jobs::inference_result_type::InferenceResultType;
use enums::by_table::media_files::media_file_class::MediaFileClass;
use enums::by_table::media_files::media_file_origin_category::MediaFileOriginCategory;
use enums::by_table::media_files::media_file_origin_product_category::MediaFileOriginProductCategory;
use enums::by_table::media_files::media_file_type::MediaFileType;
use enums::common::generation_provider::GenerationProvider;
use errors::AnyhowResult;
use hashing::sha256::sha256_hash_bytes::sha256_hash_bytes;
use mysql_queries::queries::generic_inference::job::mark_job_failed_by_token::{mark_job_failed_by_token, MarkJobFailedByTokenArgs};
use mysql_queries::queries::generic_inference::web::list_pending_fake_generation_jobs::{list_pending_fake_generation_jobs, PendingFakeGenerationJob};
use mysql_queries::queries::generic_inference::web::mark_generic_inference_job_successfully_done_by_token_with_executor::{
mark_generic_inference_job_successfully_done_by_token_with_executor,
MarkGenericInferenceJobSuccessfullyDoneByTokenWithExecutorArgs,
};
use mysql_queries::queries::media_files::create::insert_builder::media_file_insert_builder::MediaFileInsertBuilder;
use tokens::tokens::media_files::MediaFileToken;
use crate::http_server::endpoints::generate::common::job_failure_test::test_synthetic_failure_reason;
/// Placeholder result assets, relative to the repo root (the server's working
/// directory — its config search path already depends on that).
const FAKE_IMAGE_RESULT_SOURCE: &str = "test_data/image/juno.jpg";
const FAKE_VIDEO_RESULT_SOURCE: &str = "test_data/video/mp4/golden_sun_garoh.mp4";
const JOBS_PER_TICK: u32 = 25;
/// Dev-only companion to DEV_FAKE_GENERATION (see ServerState): polls for
/// pending jobs whose external id is `fake_…` and, once they are older than
/// DEV_FAKE_GENERATION_RESOLVE_SECS, resolves them the way a real provider
/// callback would — a `media_files` row backed by a placeholder asset under
/// LOCAL_MEDIA_ROOT for success, or a synthetic failure when the prompt
/// contains the `simulate_artcraft_failure` / `test_artcraft_failure`
/// triggers (with an optional FrontendFailureCategory word, same as the
/// legacy handlers).
pub async fn dev_fake_generation_resolver_thread(mysql_pool: MySqlPool) {
let resolve_after = easyenv::get_env_duration_seconds_or_default(
"DEV_FAKE_GENERATION_RESOLVE_SECS", Duration::from_secs(6));
let poll_interval = easyenv::get_env_duration_seconds_or_default(
"DEV_FAKE_GENERATION_POLL_SECS", Duration::from_secs(2));
info!("Dev fake-generation resolver running: jobs resolve after ~{}s (poll every {}s).",
resolve_after.as_secs(), poll_interval.as_secs());
loop {
tokio::time::sleep(poll_interval).await;
let jobs = match list_pending_fake_generation_jobs(
&mysql_pool,
resolve_after.as_secs() as u32,
JOBS_PER_TICK,
).await {
Ok(jobs) => jobs,
Err(err) => {
error!("Dev fake-generation resolver failed to list pending jobs: {:?}", err);
continue;
}
};
for job in jobs {
let job_token = job.job_token.clone();
if let Err(err) = resolve_job(&mysql_pool, &job, resolve_after).await {
error!("Dev fake-generation resolver failed on job {}: {:?}", job_token.as_str(), err);
// Fail the job so the frontend sees a terminal state instead of an
// eternally-pending spinner (and so we don't retry it every tick).
let mark_result = mark_job_failed_by_token(MarkJobFailedByTokenArgs {
pool: &mysql_pool,
job_token: &job_token,
maybe_public_failure_reason: Some("Dev fake generation could not produce a result."),
internal_debugging_failure_reason: &format!("dev fake resolver error: {:?}", err),
maybe_frontend_failure_category: None,
}).await;
if let Err(mark_err) = mark_result {
error!("Also failed to mark job {} failed: {:?}", job_token.as_str(), mark_err);
}
}
}
}
}
async fn resolve_job(
mysql_pool: &MySqlPool,
job: &PendingFakeGenerationJob,
simulated_duration: Duration,
) -> AnyhowResult<()> {
let prompt = job.maybe_positive_prompt.as_deref().unwrap_or("");
if let Some(failure) = test_synthetic_failure_reason(prompt) {
mark_job_failed_by_token(MarkJobFailedByTokenArgs {
pool: mysql_pool,
job_token: &job.job_token,
maybe_public_failure_reason: failure.frontend_failure_message.as_deref(),
internal_debugging_failure_reason: "DEV_FAKE_GENERATION synthetic failure (prompt trigger)",
maybe_frontend_failure_category: Some(failure.frontend_failure_category),
}).await?;
info!("Dev fake-generation: job {} failed synthetically ({:?}).",
job.job_token.as_str(), failure.frontend_failure_category);
return Ok(());
}
let media_token = insert_fake_result_media(mysql_pool, job).await?;
mark_generic_inference_job_successfully_done_by_token_with_executor(
MarkGenericInferenceJobSuccessfullyDoneByTokenWithExecutorArgs {
executor: mysql_pool,
token: &job.job_token,
maybe_entity_type: Some(InferenceResultType::MediaFile),
maybe_entity_token: Some(media_token.as_str()),
total_job_duration: Some(simulated_duration),
inference_duration: Some(simulated_duration),
},
).await?;
info!("Dev fake-generation: job {} completed with media file {}.",
job.job_token.as_str(), media_token.as_str());
Ok(())
}
/// Copy a placeholder asset to a fresh bucket path under LOCAL_MEDIA_ROOT and
/// insert the `media_files` row, mirroring what the Fal webhook success path
/// does with a real provider result (minus the R2 upload).
async fn insert_fake_result_media(
mysql_pool: &MySqlPool,
job: &PendingFakeGenerationJob,
) -> AnyhowResult<MediaFileToken> {
let is_video = job.inference_category.contains("video");
let (source_path, extension, media_class, media_type, mime_type, product_category) = if is_video {
(FAKE_VIDEO_RESULT_SOURCE, ".mp4", MediaFileClass::Video, MediaFileType::Mp4,
"video/mp4", MediaFileOriginProductCategory::VideoGeneration)
} else {
(FAKE_IMAGE_RESULT_SOURCE, ".jpg", MediaFileClass::Image, MediaFileType::Jpg,
"image/jpeg", MediaFileOriginProductCategory::ImageGeneration)
};
let media_root = std::env::var("LOCAL_MEDIA_ROOT")
.ok()
.filter(|root| !root.trim().is_empty())
.ok_or_else(|| anyhow::anyhow!(
"LOCAL_MEDIA_ROOT must be set for DEV_FAKE_GENERATION results to be viewable"))?;
let file_bytes = tokio::fs::read(source_path).await
.map_err(|err| anyhow::anyhow!(
"cannot read placeholder asset {} (server must run from the repo root): {}", source_path, err))?;
let bucket_path = MediaFileBucketPath::generate_new(Some("fake_"), Some(extension));
let disk_path = local_object_disk_path(Path::new(&media_root), &bucket_path.get_full_object_path_str())?;
if let Some(parent) = disk_path.parent() {
tokio::fs::create_dir_all(parent).await?;
}
tokio::fs::write(&disk_path, &file_bytes).await?;
if is_video {
// The video thumbnail job never runs locally, so drop the thumbnail
// siblings the gallery expects next to the object. The animated "gif" is
// really JPEG bytes — browsers decode <img> content by magic bytes, and a
// still preview is plenty for dev.
let thumb_bytes = tokio::fs::read(FAKE_IMAGE_RESULT_SOURCE).await?;
let base = disk_path.to_string_lossy().to_string();
tokio::fs::write(format!("{base}{VIDEO_STATIC_JPG_THUMBNAIL_SUFFIX}"), &thumb_bytes).await?;
tokio::fs::write(format!("{base}{VIDEO_ANIMATED_GIF_THUMBNAIL_SUFFIX}"), &thumb_bytes).await?;
}
let checksum = sha256_hash_bytes(&file_bytes)?;
let media_token = MediaFileInsertBuilder::new()
.checksum_sha2(&checksum)
.creator_ip_address("127.0.0.1")
.file_size_bytes(file_bytes.len() as u64)
.maybe_creator_user(job.maybe_creator_user_token.as_ref())
.maybe_generation_provider(Some(GenerationProvider::Artcraft))
.maybe_prompt_token(job.maybe_prompt_token.as_ref())
.media_file_class(media_class)
.media_file_origin_category(MediaFileOriginCategory::Inference)
.media_file_origin_product_category(product_category)
.media_file_type(media_type)
.mime_type(mime_type)
.public_bucket_directory_hash(&bucket_path)
.insert_pool(mysql_pool)
.await
.map_err(|err| anyhow::anyhow!("failed to insert fake result media file: {:?}", err))?;
Ok(media_token)
}
@@ -1,3 +1,4 @@
pub mod db_health_checker_thread;
pub mod dev_fake_generation_resolver_thread;
pub mod poll_ip_banlist_thread;
pub mod poll_model_token_info_thread;
pub mod poll_model_token_info_thread;
+4 -4
View File
@@ -20,10 +20,10 @@ if (import.meta.env.DEV) {
StorytellerApiHostStore.getInstance().setApiSchemeAndHost(
window.location.origin,
);
// NB: This is for Brandon to test with storyteller-web locally:
// (disabled — it overrides the origin above and points at localhost:12345,
// which breaks dev unless a local backend is running. Re-enable only when
// testing against a local storyteller-web.)
// NB: This overrides the origin above and points all API traffic at the
// local storyteller-web on http://localhost:12345. Dev REQUIRES the local
// backend (see _docs/dev_setup_local_stack.md) — without it the webapp is
// broken in dev, not silently on production.
StorytellerApiHostStore.getInstance().setDevelopment();
} catch (e) {
console.warn("Failed to set dev API host override", e);
@@ -47,11 +47,15 @@ const apiHost = () =>
// through the Vite `/__cdn` proxy so they aren't blocked by CORS — the CDN only
// sends CORS headers for the production origin. In production this is a no-op:
// the asset is fetched directly from its real URL.
// Localhost media (the fully-local dev stack's backend serving /media) is
// fetched directly: the dev backend sends CORS headers for any localhost
// origin, and the proxy would break it anyway (it assumes https).
function maybeProxyCdnUrl(url: string): string {
if (!import.meta.env.DEV) return url;
try {
const parsed = new URL(url, window.location.origin);
if (parsed.origin === window.location.origin) return url;
if (parsed.hostname === "localhost" || parsed.hostname === "127.0.0.1") return url;
return `/__cdn/${parsed.host}${parsed.pathname}${parsed.search}`;
} catch {
return url;
@@ -334,6 +338,11 @@ export const useWebAppPageSceneAdapter = (
const path = bucketPath?.startsWith("/")
? bucketPath
: `/${bucketPath ?? ""}`;
// The fully-local dev backend serves media itself and has no
// Cloudflare /cdn-cgi/image resizer — serve the original.
if (base.includes("localhost") || base.includes("127.0.0.1")) {
return `${base}${path}`;
}
const params: string[] = [];
if (width) params.push(`width=${width}`);
if (quality) params.push(`quality=${quality}`);
+12 -7
View File
@@ -42,7 +42,12 @@ function cdnProxyPlugin() {
return;
}
const [, host, assetPath = '/', query = ''] = match;
const upstream = await fetch(`https://${host}${assetPath}${query}`, {
// Localhost upstreams (the fully-local dev backend) are plain
// HTTP; everything else (real CDNs) is HTTPS. The web adapter
// fetches localhost directly and never proxies it, but don't let
// a stray localhost request die on a TLS handshake.
const scheme = /^(localhost|127\.0\.0\.1)(:\d+)?$/.test(host) ? 'http' : 'https';
const upstream = await fetch(`${scheme}://${host}${assetPath}${query}`, {
method: req.method === 'HEAD' ? 'HEAD' : 'GET',
headers: req.headers.range
? { range: String(req.headers.range) }
@@ -93,14 +98,14 @@ export default defineConfig(() => ({
port: 4201,
host: 'localhost',
proxy: {
// Forward API calls to production API to avoid CORS during local dev
// Fallback only: main.tsx calls setDevelopment() in DEV, which points
// all API traffic at http://localhost:12345 directly, bypassing this
// proxy entirely. Target the local backend too so nothing can silently
// fall back to production traffic during local dev.
'/v1': {
target: 'https://api.storyteller.ai',
target: 'http://localhost:12345',
changeOrigin: true,
secure: true,
headers: {
Origin: 'https://api.storyteller.ai',
},
secure: false,
},
// NB: `/__cdn` asset proxying lives in `cdnProxyPlugin` above — it
// needs per-request host routing, which `server.proxy` can't do.
+12
View File
@@ -1,4 +1,16 @@
import { StorytellerApiHostStore } from "./config/StorytellerApiHostStore";
// TODO(bt,2025-05-15): Remove this code
//
// Legacy CDN origin for callers that rebuild media URLs from raw bucket
// paths instead of using the `media_links` the API returns (pagescene's
// asset thumbnails are the last live consumer). When the API host is a
// local dev backend, that backend serves media itself (CDN_BASE_URL +
// /media mount), so the CDN origin IS the API origin.
export function GetCdnOrigin(): string {
const apiHost = StorytellerApiHostStore.getInstance().getApiSchemeAndHost();
if (apiHost.includes("localhost") || apiHost.includes("127.0.0.1")) {
return apiHost;
}
return "https://cdn-2.fakeyou.com";
}
+388
View File
@@ -0,0 +1,388 @@
#!/usr/bin/env bash
# Bootstrap the full local dev stack (backend + frontend) on a fresh clone.
#
# Target platform: Ubuntu 22.04+ — native, WSL2, or a CI runner/container.
# (Windows users: the native PowerShell variant in script/bootstrap/windows/
# is the primary path — no WSL needed. macOS users: follow
# _docs/dev_setup_server.md manually. See _docs/dev_setup_local_stack.md.)
#
# Idempotent: safe to re-run at any time; each step detects work already done.
#
# Usage:
# ./script/bootstrap/bootstrap_dev_stack.sh [options]
#
# Options:
# --yes, -y Non-interactive: assume "yes" for prompts (CI mode)
# --skip-packages Don't apt-install system packages
# --skip-rust-build Don't build the storyteller-web binary
# --skip-frontend Don't run the frontend npm install
#
# After it succeeds:
# ./script/bootstrap/run_backend_dev.sh # start the API on :12345
# ./script/bootstrap/seed_demo_user.sh # create the demo login
# cd frontend && npx nx dev artcraft-webapp # webapp on :4201
set -euo pipefail
root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
source "${root_dir}/script/bootstrap/common.sh"
SECRETS_ENV_FILE="${root_dir}/crates/service/web/storyteller_web/config/storyteller-web.development-secrets.env"
# System packages: MySQL + Redis servers, the native libs needed to compile the
# Rust workspace (per build/service_cpu.Dockerfile) and diesel_cli
# (libmysqlclient-dev, libsqlite3-dev), plus curl/jq for the seeding scripts.
SYSTEM_PACKAGES=(
build-essential
ca-certificates
cmake
curl
ffmpeg
fontconfig
git
jq
libclang-dev
libfontconfig1-dev
libmysqlclient-dev
libsqlite3-dev
libssl-dev
mysql-server
perl
pkg-config
redis-server
)
ASSUME_YES=false
SKIP_PACKAGES=false
SKIP_RUST_BUILD=false
SKIP_FRONTEND=false
main() {
parse_args "$@"
ensure_supported_platform
install_system_packages
ensure_mysql_running
provision_mysql_database
ensure_redis_running
ensure_rust_toolchain
ensure_diesel_cli
run_migrations
seed_roles_and_badges
write_secrets_env_if_missing
build_backend
setup_frontend
print_next_steps
}
parse_args() {
for arg in "$@"; do
case "${arg}" in
--yes|-y) ASSUME_YES=true ;;
--skip-packages) SKIP_PACKAGES=true ;;
--skip-rust-build) SKIP_RUST_BUILD=true ;;
--skip-frontend) SKIP_FRONTEND=true ;;
--help|-h) sed -n '2,22p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) die "Unknown option: ${arg} (try --help)" ;;
esac
done
}
ensure_supported_platform() {
case "$(uname -s)" in
Linux) ;;
Darwin)
die "This script targets Ubuntu/WSL2. On macOS, follow _docs/dev_setup_server.md."
;;
*)
die "Unsupported platform '$(uname -s)'. On Windows use script/bootstrap/windows/bootstrap_dev_stack.ps1 (native), or WSL2 Ubuntu for this script."
;;
esac
if grep -qi microsoft /proc/version 2>/dev/null && [[ "${root_dir}" == /mnt/* ]]; then
warn "This checkout lives on the Windows filesystem (${root_dir})."
warn "Rust + npm builds are dramatically slower through /mnt. Consider cloning"
warn "into the WSL filesystem (e.g. ~/storyteller-rust) instead."
fi
}
install_system_packages() {
step "System packages"
if [ "${SKIP_PACKAGES}" = "true" ]; then
log "Skipped (--skip-packages)."
return
fi
local missing=()
for pkg in "${SYSTEM_PACKAGES[@]}"; do
dpkg -s "${pkg}" >/dev/null 2>&1 || missing+=("${pkg}")
done
if [ "${#missing[@]}" -eq 0 ]; then
log "All required apt packages are already installed."
return
fi
log "Missing packages: ${missing[*]}"
confirm "Install them with apt-get?" || die "Cannot continue without system packages (or pass --skip-packages if you manage them yourself)."
maybe_sudo env DEBIAN_FRONTEND=noninteractive apt-get update
maybe_sudo env DEBIAN_FRONTEND=noninteractive apt-get install -y "${missing[@]}"
}
ensure_mysql_running() {
step "MySQL server"
if mysqladmin ping --silent 2>/dev/null; then
log "MySQL is running."
else
log "Starting MySQL..."
maybe_sudo service mysql start 2>/dev/null \
|| maybe_sudo systemctl start mysql 2>/dev/null \
|| die "Could not start MySQL (tried 'service mysql start' and 'systemctl start mysql')."
mysqladmin ping --silent 2>/dev/null || sleep 3
mysqladmin ping --silent 2>/dev/null || die "MySQL did not come up."
fi
# The tooling supports the MySQL 8.x series; 9.x is known-unsupported
# (_docs/dev_setup_server.md).
local version
version="$(mysqld --version 2>/dev/null || true)"
case "${version}" in
*" 8."*) ;;
"") warn "Could not determine MySQL server version." ;;
*) warn "MySQL 8.x is expected; found: ${version}" ;;
esac
}
provision_mysql_database() {
step "MySQL database + user (${DEV_MYSQL_DB} / ${DEV_MYSQL_USER}@localhost)"
if mysql_app_reachable; then
log "Database and app user already provisioned."
return
fi
# Same DDL as _docs/dev_setup_server.md, made idempotent.
mysql_admin <<SQL
CREATE DATABASE IF NOT EXISTS ${DEV_MYSQL_DB};
CREATE USER IF NOT EXISTS '${DEV_MYSQL_USER}'@'localhost' IDENTIFIED BY '${DEV_MYSQL_PASSWORD}';
GRANT ALL PRIVILEGES ON ${DEV_MYSQL_DB}.* TO '${DEV_MYSQL_USER}'@'localhost';
FLUSH PRIVILEGES;
SQL
mysql_app_reachable || die "Provisioning ran but '${DEV_MYSQL_USER}' still cannot reach '${DEV_MYSQL_DB}'."
log "Provisioned."
}
ensure_redis_running() {
step "Redis server"
if redis_reachable; then
log "Redis is running."
return
fi
log "Starting Redis..."
maybe_sudo service redis-server start 2>/dev/null \
|| maybe_sudo systemctl start redis-server 2>/dev/null \
|| die "Could not start Redis."
redis_reachable || die "Redis did not come up."
}
ensure_rust_toolchain() {
step "Rust toolchain"
ensure_cargo_on_path
if command -v cargo >/dev/null 2>&1; then
log "Found $(rustc --version 2>/dev/null || echo 'cargo (rustc missing?)')."
return
fi
confirm "Rust is not installed. Install stable via rustup?" || die "Rust is required to build the backend."
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y --default-toolchain stable
ensure_cargo_on_path
command -v cargo >/dev/null 2>&1 || die "rustup install finished but cargo is not on PATH."
}
ensure_diesel_cli() {
step "diesel_cli (migration runner)"
if command -v diesel >/dev/null 2>&1; then
log "Found $(diesel --version)."
return
fi
# Same install command as _docs/dev_setup_server.md. Needs libmysqlclient-dev
# and libsqlite3-dev (installed above).
log "Installing diesel_cli (this compiles from source; a few minutes)..."
cargo install diesel_cli --no-default-features --features mysql,sqlite
}
run_migrations() {
step "Database migrations (_database/sql/migrations)"
# diesel-cli reads DATABASE_URL; the repo-root .env provides it, but export a
# default so this also works if that file is ever absent (e.g. sparse CI).
export DATABASE_URL="${DATABASE_URL:-mysql://${DEV_MYSQL_USER}:${DEV_MYSQL_PASSWORD}@${DEV_MYSQL_HOST}/${DEV_MYSQL_DB}}"
cd "${root_dir}"
# NB: "Encountered unknown type for Mysql: enum" warnings are harmless
# (_docs/dev_setup_server.md).
diesel migration run
log "Migrations up to date."
}
seed_roles_and_badges() {
step "Seed data (system roles + badges)"
# Same data as _database/sql/seed/bootstrap_inserts_roles_etc.sh, but guarded
# so re-runs don't hit duplicate-key errors. The 'user' role is mandatory:
# account creation hardcodes user_role_slug='user'.
local role_count badge_count
role_count="$(mysql_app -N -e "SELECT COUNT(*) FROM user_roles" 2>/dev/null || echo 0)"
if [ "${role_count}" -gt 0 ]; then
log "user_roles already seeded (${role_count} rows)."
else
mysql_app < "${root_dir}/_database/sql/seed/sql/system_roles.sql"
log "Inserted system roles (user, mod, admin)."
fi
badge_count="$(mysql_app -N -e "SELECT COUNT(*) FROM badges" 2>/dev/null || echo 0)"
if [ "${badge_count}" -gt 0 ]; then
log "badges already seeded (${badge_count} rows)."
else
mysql_app < "${root_dir}/_database/sql/seed/sql/user_badges.sql"
log "Inserted badges."
fi
}
write_secrets_env_if_missing() {
step "Development secrets file"
if [ -f "${SECRETS_ENV_FILE}" ]; then
log "Already exists: ${SECRETS_ENV_FILE} (leaving it untouched)."
add_dev_stack_keys_to_existing_secrets_env
return
fi
# Every env var below is read with get_env_string_required at server startup
# (build_dependencies.rs / setup_inference_providers.rs / setup_stripe_*.rs)
# but is NOT supplied by the checked-in config files. None of them is
# contacted at boot — placeholder values are enough to run the server; the
# endpoints that would use them fail lazily if exercised.
#
# NOTE: config loading is FIRST-VALUE-WINS (dotenv never overrides an
# already-set key), so values here cannot override anything already set in
# storyteller-web.development.env — only real process env vars can.
cat > "${SECRETS_ENV_FILE}" <<'EOF'
# Local-development secrets for storyteller-web. Generated by
# script/bootstrap/bootstrap_dev_stack.sh — gitignored; edit freely.
#
# Placeholders below satisfy required-at-boot checks without enabling any
# external integration. Replace individual values with real dev credentials
# only if you need that specific integration locally.
# R2/S3 object storage (clients are constructed offline)
ACCESS_KEY=dummy-local-dev
SECRET_KEY=dummy-local-dev
REGION_NAME=us-east-1
W2L_PRIVATE_DOWNLOAD_BUCKET_NAME=dummy-local-dev-private
W2L_PUBLIC_DOWNLOAD_BUCKET_NAME=dummy-local-dev-public
# Email (Resend)
RESEND_API_KEY=dummy-local-dev
# Generation providers. NB: the omni_gen endpoints (webapp image/video
# generation) contact providers SYNCHRONOUSLY in the request handler, so with
# dummy keys a real submit would 500 — DEV_FAKE_GENERATION below avoids that.
# The legacy polling providers are only contacted by worker binaries, which
# local dev does not run.
FAL_API_KEY=dummy-local-dev
GMICLOUD_API_KEY=dummy-local-dev
GROK_API_KEY=dummy-local-dev
BEEBLE_API_KEY=dummy-local-dev
OPENAI_API_KEY=dummy-local-dev
WORLDLABS_API_KEY=dummy-local-dev
SEEDANCE2PRO_COOKIES=dummy-local-dev
SEEDANCE2PRO_WHITELIST_COOKIES=dummy-local-dev
SEEDANCE2PRO_BYTEPLUS_ULTRA_COOKIES=dummy-local-dev
# Stripe account ids (test-mode secret keys are already checked into
# storyteller-web.development.env; only the account ids are missing there)
STRIPE_FAKEYOU_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_SECRET_KEY=sk_test_dummylocaldev
STRIPE_ARTCRAFT_SECRET_WEBHOOK_KEY=whsec_dummylocaldev
EOF
dev_stack_keys_block >> "${SECRETS_ENV_FILE}"
log "Wrote ${SECRETS_ENV_FILE}"
}
# The fully-local media + fake-generation settings (development-only; the
# server hard-ignores DEV_FAKE_GENERATION outside SERVER_ENVIRONMENT=Development).
dev_stack_keys_block() {
cat <<EOF
# Fully-local dev stack (see _docs/dev_setup_local_stack.md):
# - DEV_FAKE_GENERATION: omni_gen image/video submits skip real providers and
# a resolver thread completes them a few seconds later with a placeholder
# result (or fails them when the prompt contains simulate_artcraft_failure).
# - CDN_BASE_URL: media URLs in API responses point at this server instead of
# the shared r2.dev bucket. Host only — no path suffix.
# - LOCAL_MEDIA_ROOT: directory served at /media (mirrors the bucket layout);
# seeded gallery media and fake generation results live here.
DEV_FAKE_GENERATION=true
CDN_BASE_URL=http://localhost:12345
LOCAL_MEDIA_ROOT=${DEV_MEDIA_ROOT}
EOF
}
# Bootstrap re-runs on a stack whose secrets file predates the fully-local
# media/fake-generation support: append the new keys (first-value-wins never
# lets these override anything the user already set).
add_dev_stack_keys_to_existing_secrets_env() {
if grep -q '^[[:space:]]*LOCAL_MEDIA_ROOT=' "${SECRETS_ENV_FILE}"; then
return
fi
dev_stack_keys_block >> "${SECRETS_ENV_FILE}"
log "Appended fully-local dev stack keys (DEV_FAKE_GENERATION, CDN_BASE_URL, LOCAL_MEDIA_ROOT) to ${SECRETS_ENV_FILE}"
}
build_backend() {
step "Backend build (storyteller-web)"
if [ "${SKIP_RUST_BUILD}" = "true" ]; then
log "Skipped (--skip-rust-build)."
return
fi
cd "${root_dir}"
# SQLX_OFFLINE only affects compile-time query verification (uses the
# checked-in .sqlx/ cache instead of a live DB); it has no runtime effect.
log "Building (first build takes several minutes)..."
SQLX_OFFLINE=true cargo build -p storyteller-web --bin storyteller-web
log "Built target/debug/storyteller-web."
}
setup_frontend() {
step "Frontend install (frontend/)"
if [ "${SKIP_FRONTEND}" = "true" ]; then
log "Skipped (--skip-frontend)."
return
fi
source "${root_dir}/script/common/frontend_preflight.sh"
frontend_preflight "${root_dir}/frontend"
pushd "${root_dir}/frontend" >/dev/null
frontend_npm_install
popd >/dev/null
log "Frontend dependencies installed."
}
print_next_steps() {
step "Done"
cat <<EOF
The dev stack is bootstrapped. Next steps:
1. Start the backend (binds ${DEV_BACKEND_URL}):
./script/bootstrap/run_backend_dev.sh
2. In another terminal, create the demo login (idempotent):
./script/bootstrap/seed_demo_user.sh
Credentials: ${DEMO_USERNAME} / ${DEMO_PASSWORD}
3. Start the webapp (binds http://localhost:4201; in dev it talks to
${DEV_BACKEND_URL} automatically):
cd frontend && npx nx dev artcraft-webapp
Health checks at any time:
./script/bootstrap/dev_stack_doctor.sh
EOF
}
main "$@"
+121
View File
@@ -0,0 +1,121 @@
#!/usr/bin/env bash
# Shared helpers for the dev-stack bootstrap scripts.
#
# Usage (from a script that has set ${root_dir}):
#
# source "${root_dir}/script/bootstrap/common.sh"
#
# See _docs/dev_setup_local_stack.md for the full picture.
# Canonical local-dev database identity. These match _docs/dev_setup_server.md,
# the diesel DATABASE_URL in the repo-root .env, and the server's MYSQL_URL
# default (crates/schema/config/shared_env_var_config/src/mysql.rs). Override
# via environment only if your local MySQL differs — most tooling assumes them.
DEV_MYSQL_DB="${DEV_MYSQL_DB:-storyteller}"
DEV_MYSQL_USER="${DEV_MYSQL_USER:-storyteller}"
DEV_MYSQL_PASSWORD="${DEV_MYSQL_PASSWORD:-password}"
DEV_MYSQL_HOST="${DEV_MYSQL_HOST:-localhost}"
# Where the backend listens (BIND_ADDRESS default in
# storyteller_web/src/startup/build_dependencies.rs).
DEV_BACKEND_URL="${DEV_BACKEND_URL:-http://localhost:12345}"
# Default demo account. The username must NOT be on the reserved-usernames list
# (includes/binary_includes/usernames/reserved_usernames.txt — "demo", "admin",
# "test", and "dev" are all reserved) and must match ^[A-Za-z0-9_\-]{3,16}$.
DEMO_USERNAME="${DEMO_USERNAME:-localdev1}"
DEMO_PASSWORD="${DEMO_PASSWORD:-localdev1pass}"
DEMO_EMAIL="${DEMO_EMAIL:-localdev1@example.com}"
# Banked credits the demo user's artcraft wallet is seeded/topped-up to.
# Generation submits are gated on wallet balance, so 0 credits means the
# generate button silently refuses in the webapp.
DEMO_CREDITS="${DEMO_CREDITS:-100000}"
# Local media served by the backend at /media when LOCAL_MEDIA_ROOT points
# here (fully-local gallery + fake generation results). Mirrors the public
# bucket layout: object path /media/{...} lives at .devstack/media/media/{...}.
# NB: callers must have set ${root_dir} before sourcing this file.
DEV_MEDIA_ROOT="${DEV_MEDIA_ROOT:-${root_dir}/.devstack/media}"
log() { echo "[bootstrap] $*"; }
warn() { echo "[bootstrap] WARNING: $*" >&2; }
die() { echo "[bootstrap] ERROR: $*" >&2; exit 1; }
step() {
echo ""
echo "=== $* ==="
}
# sudo unless we already are root (CI containers often run as root without sudo).
maybe_sudo() {
if [ "$(id -u)" -eq 0 ]; then
"$@"
else
sudo "$@"
fi
}
confirm() {
local prompt="$1"
if [ "${ASSUME_YES:-false}" = "true" ]; then
return 0
fi
local reply
read -r -p "${prompt} [y/N] " reply
[[ "${reply}" =~ ^[Yy] ]]
}
# Generate a token in the backend's format: {prefix}{crockford-lower entropy}
# padded to the same TOTAL length the Rust generators use (tokens crate,
# impl_crockford_generator!). Alphabet excludes i/l/o/u like Crockford base32.
new_dev_token() {
local prefix="$1" total_length="$2"
local charset="0123456789abcdefghjkmnpqrstvwxyz"
local count=$((total_length - ${#prefix}))
local entropy=""
local i
for ((i = 0; i < count; i++)); do
entropy+="${charset:RANDOM%32:1}"
done
echo "${prefix}${entropy}"
}
# Run a statement (or stdin) as the storyteller app user against the dev DB.
# MYSQL_PWD keeps the password off the process list.
mysql_app() {
MYSQL_PWD="${DEV_MYSQL_PASSWORD}" mysql \
-u "${DEV_MYSQL_USER}" -h "${DEV_MYSQL_HOST}" -D "${DEV_MYSQL_DB}" "$@"
}
# Run a statement as an administrator, for provisioning the DB and user.
# On a fresh Ubuntu/WSL install, root uses auth_socket, so `sudo mysql` works.
# On GitHub Actions runners the preinstalled MySQL root password is "root" —
# export MYSQL_ROOT_PASSWORD to use password auth instead.
mysql_admin() {
if [ -n "${MYSQL_ROOT_PASSWORD:-}" ]; then
MYSQL_PWD="${MYSQL_ROOT_PASSWORD}" mysql -u root -h "${DEV_MYSQL_HOST}" "$@"
else
maybe_sudo mysql "$@"
fi
}
mysql_app_reachable() {
mysql_app -e "SELECT 1" >/dev/null 2>&1
}
redis_reachable() {
redis-cli ping 2>/dev/null | grep -q PONG
}
backend_reachable() {
curl -sf -o /dev/null --max-time 3 "${DEV_BACKEND_URL}/_status"
}
# Make ~/.cargo/bin visible when rustup was installed but the shell profile
# hasn't been re-sourced (fresh installs, CI).
ensure_cargo_on_path() {
if ! command -v cargo >/dev/null 2>&1 && [ -x "${HOME}/.cargo/bin/cargo" ]; then
export PATH="${HOME}/.cargo/bin:${PATH}"
fi
}
+156
View File
@@ -0,0 +1,156 @@
#!/usr/bin/env bash
# Diagnose the local dev stack: checks every prerequisite the bootstrap sets up
# and reports PASS / WARN / FAIL per item.
#
# FAIL = the stack cannot work until fixed (exit code 1).
# WARN = optional or run-time-only (e.g. backend not currently running).
set -uo pipefail
root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
source "${root_dir}/script/bootstrap/common.sh"
SECRETS_ENV_FILE="${root_dir}/crates/service/web/storyteller_web/config/storyteller-web.development-secrets.env"
failures=0
warnings=0
pass() { printf ' PASS %s\n' "$1"; }
fail() { printf ' FAIL %s\n' "$1"; failures=$((failures + 1)); }
warn_check() { printf ' WARN %s\n' "$1"; warnings=$((warnings + 1)); }
echo "Dev stack doctor (${root_dir})"
echo ""
echo "--- Toolchains ---"
ensure_cargo_on_path
if command -v cargo >/dev/null 2>&1; then
pass "rust: $(rustc --version 2>/dev/null | head -1)"
else
fail "rust: cargo not on PATH (bootstrap installs via rustup)"
fi
if command -v diesel >/dev/null 2>&1; then
pass "diesel_cli: $(diesel --version 2>/dev/null | head -1)"
else
fail "diesel_cli: not installed (cargo install diesel_cli --no-default-features --features mysql,sqlite)"
fi
if command -v node >/dev/null 2>&1; then
node_major="$(node --version | sed -E 's/^v([0-9]+).*/\1/')"
if [ "${node_major}" -ge 20 ]; then
pass "node: $(node --version) (>= 20 required by Nx 21 / Vite 6)"
else
fail "node: $(node --version) is too old — Node 20+ required"
fi
else
fail "node: not installed (Node 20+ required for the frontend)"
fi
echo ""
echo "--- Services ---"
if mysql_app_reachable; then
pass "mysql: reachable as '${DEV_MYSQL_USER}' on database '${DEV_MYSQL_DB}'"
migration_dirs="$(find "${root_dir}/_database/sql/migrations" -mindepth 1 -maxdepth 1 -type d | wc -l | tr -d ' ')"
applied="$(mysql_app -N -e "SELECT COUNT(*) FROM __diesel_schema_migrations" 2>/dev/null || echo 0)"
if [ "${applied}" -ge "${migration_dirs}" ] && [ "${applied}" -gt 0 ]; then
pass "migrations: ${applied} applied (${migration_dirs} in _database/sql/migrations)"
elif [ "${applied}" -gt 0 ]; then
fail "migrations: only ${applied}/${migration_dirs} applied — run 'diesel migration run'"
else
fail "migrations: none applied — run 'diesel migration run'"
fi
role_count="$(mysql_app -N -e "SELECT COUNT(*) FROM user_roles" 2>/dev/null || echo 0)"
if [ "${role_count}" -ge 3 ]; then
pass "seed: user_roles has ${role_count} rows (user/mod/admin present)"
else
fail "seed: user_roles has ${role_count} rows — account creation needs the 'user' role (re-run bootstrap)"
fi
badge_count="$(mysql_app -N -e "SELECT COUNT(*) FROM badges" 2>/dev/null || echo 0)"
if [ "${badge_count}" -gt 0 ]; then
pass "seed: badges has ${badge_count} rows"
else
warn_check "seed: badges table is empty (cosmetic; re-run bootstrap to fill)"
fi
demo_count="$(mysql_app -N -e "SELECT COUNT(*) FROM users WHERE username='${DEMO_USERNAME}'" 2>/dev/null || echo 0)"
if [ "${demo_count}" -gt 0 ]; then
pass "demo user: '${DEMO_USERNAME}' exists"
credits="$(mysql_app -N -e "SELECT COALESCE(MAX(banked_credits + monthly_credits), 0) FROM wallets WHERE wallet_namespace='artcraft' AND owner_user_token=(SELECT token FROM users WHERE username='${DEMO_USERNAME}')" 2>/dev/null || echo 0)"
if [ "${credits}" -gt 0 ]; then
pass "demo wallet: '${DEMO_USERNAME}' has ${credits} credits"
else
warn_check "demo wallet: '${DEMO_USERNAME}' has no credits — generation flows are blocked (re-run seed_demo_user.sh)"
fi
media_count="$(mysql_app -N -e "SELECT COUNT(*) FROM media_files WHERE maybe_creator_user_token=(SELECT token FROM users WHERE username='${DEMO_USERNAME}')" 2>/dev/null || echo 0)"
if [ "${media_count}" -gt 0 ]; then
pass "demo media: '${DEMO_USERNAME}' has ${media_count} media rows"
else
warn_check "demo media: gallery is empty (re-run seed_demo_user.sh)"
fi
else
warn_check "demo user: '${DEMO_USERNAME}' not created yet (./script/bootstrap/seed_demo_user.sh, backend must be running)"
fi
else
fail "mysql: cannot connect as '${DEV_MYSQL_USER}'@'${DEV_MYSQL_HOST}' to '${DEV_MYSQL_DB}' (is MySQL running? run bootstrap)"
fi
if redis_reachable; then
pass "redis: PONG"
else
fail "redis: not reachable on localhost (the backend's r2d2 pool connects eagerly at boot)"
fi
echo ""
echo "--- Backend ---"
if [ -f "${SECRETS_ENV_FILE}" ]; then
pass "secrets env: ${SECRETS_ENV_FILE#"${root_dir}"/} exists"
if grep -q '^[[:space:]]*LOCAL_MEDIA_ROOT=' "${SECRETS_ENV_FILE}"; then
pass "secrets env: fully-local dev stack keys present (DEV_FAKE_GENERATION, CDN_BASE_URL, LOCAL_MEDIA_ROOT)"
else
warn_check "secrets env: fully-local dev stack keys missing — generation/gallery use remote CDN and real providers (re-run bootstrap to append)"
fi
else
fail "secrets env: missing — the server aborts at boot on ~19 required vars (re-run bootstrap)"
fi
if [ -x "${root_dir}/target/debug/storyteller-web" ] || [ -x "${root_dir}/target/release/storyteller-web" ]; then
pass "binary: storyteller-web is built"
else
warn_check "binary: storyteller-web not built yet (bootstrap builds it; cargo run will build on demand)"
fi
if backend_reachable; then
pass "server: ${DEV_BACKEND_URL}/_status responds"
else
warn_check "server: not running (./script/bootstrap/run_backend_dev.sh)"
fi
echo ""
echo "--- Frontend ---"
if [ -d "${root_dir}/frontend/node_modules" ]; then
pass "frontend: node_modules present"
else
warn_check "frontend: node_modules missing (cd frontend && npm install)"
fi
if [ -e "${root_dir}/frontend/pnpm-lock.yaml" ] || [ -d "${root_dir}/frontend/node_modules/.pnpm" ]; then
fail "frontend: stale pnpm artifacts detected (see frontend/README.md — the workspace uses npm)"
else
pass "frontend: no stale pnpm artifacts"
fi
echo ""
echo "Summary: ${failures} failure(s), ${warnings} warning(s)."
if [ "${failures}" -gt 0 ]; then
echo "Fix failures with: ./script/bootstrap/bootstrap_dev_stack.sh (safe to re-run)"
exit 1
fi
+26
View File
@@ -0,0 +1,26 @@
#!/usr/bin/env bash
# Run the storyteller-web backend for local development (binds 0.0.0.0:12345).
#
# Must run from a bootstrapped environment (./script/bootstrap/bootstrap_dev_stack.sh).
# Always runs from the repo root: the server's config search path
# (crates/service/web/storyteller_web/config/*.env) and its includes/ defaults
# are cwd-relative.
set -euo pipefail
root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
source "${root_dir}/script/bootstrap/common.sh"
SECRETS_ENV_FILE="${root_dir}/crates/service/web/storyteller_web/config/storyteller-web.development-secrets.env"
mysql_app_reachable || die "MySQL is not reachable as '${DEV_MYSQL_USER}'. Run ./script/bootstrap/bootstrap_dev_stack.sh first."
redis_reachable || die "Redis is not reachable. Run ./script/bootstrap/bootstrap_dev_stack.sh first."
[ -f "${SECRETS_ENV_FILE}" ] || die "Missing ${SECRETS_ENV_FILE}. Run ./script/bootstrap/bootstrap_dev_stack.sh first."
ensure_cargo_on_path
cd "${root_dir}"
# SERVER_ENVIRONMENT defaults to Development when unset (bootstrap.rs), and the
# dev config files are picked up from the search path automatically.
# SQLX_OFFLINE only affects compile-time query checking, never runtime.
log "Starting storyteller-web on ${DEV_BACKEND_URL} (Ctrl-C to stop)..."
exec env SQLX_OFFLINE=true cargo run -p storyteller-web --bin storyteller-web
+171
View File
@@ -0,0 +1,171 @@
#!/usr/bin/env bash
# Create (or verify) the local demo user via the running backend, then verify
# that login works. Idempotent: an already-existing user is a success.
#
# Requires the backend to be up (./script/bootstrap/run_backend_dev.sh) — this
# script waits up to --wait-seconds (default 60) for it.
#
# The account is created through POST /v1/create_account rather than raw SQL so
# it exercises the real signup path (bcrypt hash, session, firehose row) and
# stays correct if the users schema evolves.
#
# Override the credentials via DEMO_USERNAME / DEMO_PASSWORD / DEMO_EMAIL.
# Username rules: 3-16 chars of [A-Za-z0-9_-], not on the reserved list
# (includes/binary_includes/usernames/reserved_usernames.txt); password >= 6 chars.
set -euo pipefail
root_dir="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
source "${root_dir}/script/bootstrap/common.sh"
WAIT_SECONDS=60
for arg in "$@"; do
case "${arg}" in
--wait-seconds=*) WAIT_SECONDS="${arg#*=}" ;;
--help|-h) sed -n '2,15p' "${BASH_SOURCE[0]}" | sed 's/^# \{0,1\}//'; exit 0 ;;
*) die "Unknown option: ${arg} (try --help)" ;;
esac
done
command -v jq >/dev/null 2>&1 || die "jq is required (apt-get install jq)."
log "Waiting for backend at ${DEV_BACKEND_URL} (up to ${WAIT_SECONDS}s)..."
deadline=$((SECONDS + WAIT_SECONDS))
until backend_reachable; do
if [ "${SECONDS}" -ge "${deadline}" ]; then
die "Backend not reachable at ${DEV_BACKEND_URL}/_status. Start it with ./script/bootstrap/run_backend_dev.sh"
fi
sleep 2
done
log "Backend is up."
log "Creating account '${DEMO_USERNAME}'..."
create_response="$(curl -sS -X POST "${DEV_BACKEND_URL}/v1/create_account" \
-H "Content-Type: application/json" \
-d "$(jq -n \
--arg u "${DEMO_USERNAME}" \
--arg p "${DEMO_PASSWORD}" \
--arg e "${DEMO_EMAIL}" \
'{username: $u, password: $p, password_confirmation: $p, email_address: $e}')")"
if [ "$(echo "${create_response}" | jq -r '.success // false')" = "true" ]; then
log "Account created."
else
error_code="$(echo "${create_response}" | jq -r '.error_type // .error_code // .error_code_str // empty')"
case "${error_code}" in
*[Uu]sername[Tt]aken*|*[Ee]mail[Tt]aken*)
log "Account already exists (${error_code}) — continuing."
;;
*)
die "Account creation failed: ${create_response}"
;;
esac
fi
log "Verifying login..."
login_response="$(curl -sS -X POST "${DEV_BACKEND_URL}/v1/login" \
-H "Content-Type: application/json" \
-d "$(jq -n \
--arg u "${DEMO_USERNAME}" \
--arg p "${DEMO_PASSWORD}" \
'{username_or_email: $u, password: $p}')")"
if [ "$(echo "${login_response}" | jq -r '.success // false')" != "true" ]; then
die "Login verification failed: ${login_response}"
fi
if [ -z "$(echo "${login_response}" | jq -r '.signed_session // empty')" ]; then
die "Login succeeded but returned no signed_session: ${login_response}"
fi
# --- Credits wallet ---------------------------------------------------------
# Generation submits deduct from the user's artcraft wallet and the webapp
# gates the generate button on the balance, so a user without credits cannot
# exercise any generation flow. Signup does not create a wallet; seed one.
# Raw SQL (not an endpoint): the only credit-granting paths are Stripe
# webhooks and moderator tooling, neither of which works locally.
log "Seeding credits wallet (${DEMO_CREDITS} banked credits)..."
user_token="$(mysql_app -N -e "SELECT token FROM users WHERE username = '${DEMO_USERNAME}' LIMIT 1")"
[ -n "${user_token}" ] || die "Could not find users row for '${DEMO_USERNAME}'."
wallet_row="$(mysql_app -N -e "SELECT token, banked_credits FROM wallets WHERE wallet_namespace = 'artcraft' AND owner_user_token = '${user_token}' LIMIT 1")"
if [ -z "${wallet_row}" ]; then
wallet_token="$(new_dev_token "wallet_" 32)"
create_ledger_token="$(new_dev_token "wle_" 32)"
credit_ledger_token="$(new_dev_token "wle_" 32)"
mysql_app -e "
INSERT INTO wallets SET token='${wallet_token}', wallet_namespace='artcraft', owner_user_token='${user_token}', banked_credits=${DEMO_CREDITS}, monthly_credits=0;
INSERT INTO wallet_ledger_entries SET token='${create_ledger_token}', wallet_token='${wallet_token}', entry_type='create', credits_delta=0, banked_credits_before=0, banked_credits_after=0, monthly_credits_before=0, monthly_credits_after=0;
INSERT INTO wallet_ledger_entries SET token='${credit_ledger_token}', wallet_token='${wallet_token}', entry_type='credit_banked', maybe_entity_ref='dev_bootstrap_seed', credits_delta=${DEMO_CREDITS}, banked_credits_before=0, banked_credits_after=${DEMO_CREDITS}, monthly_credits_before=0, monthly_credits_after=0;
" || die "Failed to insert the demo wallet."
log "Wallet created with ${DEMO_CREDITS} banked credits."
else
wallet_token="$(echo "${wallet_row}" | cut -f1)"
balance="$(echo "${wallet_row}" | cut -f2)"
if [ "${balance}" -lt "${DEMO_CREDITS}" ]; then
credit_ledger_token="$(new_dev_token "wle_" 32)"
delta=$((DEMO_CREDITS - balance))
mysql_app -e "
UPDATE wallets SET banked_credits=${DEMO_CREDITS}, version=version+1 WHERE token='${wallet_token}' LIMIT 1;
INSERT INTO wallet_ledger_entries SET token='${credit_ledger_token}', wallet_token='${wallet_token}', entry_type='credit_banked', maybe_entity_ref='dev_bootstrap_topup', credits_delta=${delta}, banked_credits_before=${balance}, banked_credits_after=${DEMO_CREDITS}, monthly_credits_before=0, monthly_credits_after=0;
" || die "Failed to top up the demo wallet."
log "Wallet topped up: ${balance} -> ${DEMO_CREDITS} banked credits."
else
log "Wallet already has ${balance} banked credits — leaving as-is."
fi
fi
# --- Gallery media ----------------------------------------------------------
# Seed a few images so the library/gallery isn't empty on a fresh stack. The
# files live under .devstack/media (served by the backend at /media when
# LOCAL_MEDIA_ROOT + CDN_BASE_URL are set — bootstrap writes both into the
# generated secrets env). Rows mirror what a real image upload would write.
log "Seeding demo gallery media..."
seed_sources=(
"test_data/image/juno.jpg|Dev seed - Juno 1"
"test_data/image/mochi.jpg|Dev seed - Mochi 1"
"test_data/image/juno.jpg|Dev seed - Juno 2"
"test_data/image/mochi.jpg|Dev seed - Mochi 2"
"test_data/image/juno.jpg|Dev seed - Juno 3"
"test_data/image/mochi.jpg|Dev seed - Mochi 3"
)
existing_media="$(mysql_app -N -e "SELECT COUNT(*) FROM media_files WHERE maybe_creator_user_token='${user_token}' AND maybe_title LIKE 'Dev seed %'" 2>/dev/null || echo 0)"
if [ "${existing_media}" -ge "${#seed_sources[@]}" ]; then
log "Gallery already has ${existing_media} seeded media rows — leaving as-is."
else
for entry in "${seed_sources[@]}"; do
source_rel="${entry%%|*}"
title="${entry#*|}"
source_path="${root_dir}/${source_rel}"
[ -f "${source_path}" ] || die "Seed image missing: ${source_path}"
# Mirror MediaFileBucketPath: /media/{h0..h4}/{hash}/image_{hash}.jpg
hash="$(new_dev_token "" 32)"
fanout="${hash:0:1}/${hash:1:1}/${hash:2:1}/${hash:3:1}/${hash:4:1}"
object_dir="${DEV_MEDIA_ROOT}/media/${fanout}/${hash}"
mkdir -p "${object_dir}"
cp "${source_path}" "${object_dir}/image_${hash}.jpg"
sha="$(sha256sum "${source_path}" | cut -d' ' -f1)"
size="$(stat -c%s "${source_path}" 2>/dev/null || stat -f%z "${source_path}")"
media_token="$(new_dev_token "m_" 32)"
mysql_app -e "
INSERT INTO media_files SET token='${media_token}', maybe_creator_user_token='${user_token}',
origin_category='upload', origin_product_category='unknown', media_class='image', media_type='jpg',
maybe_mime_type='image/jpeg', file_size_bytes=${size}, checksum_sha2='${sha}',
public_bucket_directory_hash='${hash}', maybe_public_bucket_prefix='image_', maybe_public_bucket_extension='.jpg',
creator_ip_address='127.0.0.1', creator_set_visibility='public', is_user_upload=1, is_intermediate_system_file=0,
maybe_title='${title}';
" || die "Failed to insert seeded media row for ${title}."
done
log "Seeded ${#seed_sources[@]} gallery images under ${DEV_MEDIA_ROOT}."
fi
log "Demo user is ready."
log " Username: ${DEMO_USERNAME}"
log " Password: ${DEMO_PASSWORD}"
log "Log in at the webapp /login page, or via curl:"
log " curl -c /tmp/cookies.txt -X POST ${DEV_BACKEND_URL}/v1/login -H 'Content-Type: application/json' -d '{\"username_or_email\":\"${DEMO_USERNAME}\",\"password\":\"${DEMO_PASSWORD}\"}'"
@@ -0,0 +1,503 @@
# Bootstrap the full local dev stack (backend + frontend) on native Windows.
#
# Everything service-shaped is PORTABLE and lives under the gitignored
# .devstack\ directory: MySQL 8.4 (official zip) and Redis run as plain user
# processes - no admin rights, no Windows services, no WSL, no Docker.
# Build tools (cmake, perl, nasm, llvm) are checked and offered via winget.
#
# Idempotent: safe to re-run at any time; each step detects work already done.
#
# Usage (from anywhere; paths are self-resolving):
# .\script\bootstrap\windows\bootstrap_dev_stack.ps1 [options]
#
# Options:
# -Yes Non-interactive: assume "yes" for prompts (CI mode)
# -SkipTools Don't check/install build tools via winget
# -SkipRustBuild Don't build the storyteller-web binary
# -SkipFrontend Don't run the frontend npm install
#
# After it succeeds:
# .\script\bootstrap\windows\run_backend_dev.ps1 # API on :12345
# .\script\bootstrap\windows\seed_demo_user.ps1 # demo login
# cd frontend; npx nx dev artcraft-webapp # webapp on :4201
param(
[switch]$Yes,
[switch]$SkipTools,
[switch]$SkipRustBuild,
[switch]$SkipFrontend
)
. (Join-Path $PSScriptRoot "common.ps1")
. (Join-Path $PSScriptRoot "start_dev_services.ps1")
$script:AssumeYes = [bool]$Yes
# Tools needed to compile the Rust workspace on Windows (wreq's BoringSSL build
# needs cmake + nasm + perl + libclang; see the workspace Cargo.toml notes).
# Each entry: name, exe to probe, winget id, dirs the installer typically uses
# (appended to this session's PATH after install).
$BuildTools = @(
@{ Name = "CMake"; Exe = "cmake"; WingetId = "Kitware.CMake"; PathDirs = @("$env:ProgramFiles\CMake\bin") },
@{ Name = "Strawberry Perl"; Exe = "perl"; WingetId = "StrawberryPerl.StrawberryPerl"; PathDirs = @("C:\Strawberry\perl\bin") },
@{ Name = "NASM"; Exe = "nasm"; WingetId = "NASM.NASM"; PathDirs = @("$env:ProgramFiles\NASM", "$env:LOCALAPPDATA\bin\NASM") },
@{ Name = "LLVM (libclang)"; Exe = "clang"; WingetId = "LLVM.LLVM"; PathDirs = @("$env:ProgramFiles\LLVM\bin") }
)
function Main {
Step "Preflight"
Ensure-CoreTools
if (-not $SkipTools) { Ensure-BuildTools } else { Write-Log "Build-tool check skipped (-SkipTools)." }
Step "Portable MySQL $MySqlVersion"
Install-PortableMySql
Start-DevMySql
Provision-MySqlDatabase
Step "Portable Redis"
Install-PortableRedis
Start-DevRedis
Step "diesel_cli (migration runner)"
Ensure-DieselCli
Step "Database migrations (_database/sql/migrations)"
Run-Migrations
Step "Seed data (system roles + badges)"
Seed-RolesAndBadges
Step "Development secrets file"
Write-SecretsEnvIfMissing
Step "Backend build (storyteller-web)"
Build-Backend
Step "Frontend install (frontend/)"
Setup-Frontend
Print-NextSteps
}
function Ensure-CoreTools {
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
$cargoBin = Join-Path $env:USERPROFILE ".cargo\bin"
Add-SessionPath @($cargoBin)
}
if (-not (Get-Command cargo -ErrorAction SilentlyContinue)) {
Die "Rust is not installed. Install via winget (winget install Rustlang.Rustup) or https://rustup.rs, then re-run."
}
Write-Log "rust: $(rustc --version)"
if (-not (Get-Command node -ErrorAction SilentlyContinue)) {
Die "Node.js is not installed. Install Node 20+ (winget install OpenJS.NodeJS.LTS), then re-run."
}
$nodeMajor = [int]((node --version) -replace '^v(\d+).*', '$1')
if ($nodeMajor -lt 20) {
Die "Node.js 20+ is required (found $(node --version)) - Nx 21 / Vite 6 requirement."
}
Write-Log "node: $(node --version)"
}
function Ensure-BuildTools {
# Tools installed by a previous run land in dirs this shell may not have on
# PATH yet - map them in before probing, so we don't re-invoke winget.
Add-BuildToolsToSession
$missing = @($BuildTools | Where-Object { -not (Get-Command $_.Exe -ErrorAction SilentlyContinue) })
if ($missing.Count -eq 0) {
Write-Log "All native build tools present (cmake, perl, nasm, clang)."
return
}
$names = ($missing | ForEach-Object { $_.Name }) -join ", "
Write-Log "Missing build tools: $names"
if (-not (Get-Command winget -ErrorAction SilentlyContinue)) {
Die "winget is unavailable; install the tools manually: $names"
}
if (-not (Confirm-Or-Skip "Install them via winget? (installers may raise a UAC prompt)")) {
Die "Cannot build the backend without: $names (or re-run with -SkipTools if you manage them yourself)."
}
foreach ($tool in $missing) {
Write-Log "winget install $($tool.WingetId)..."
Invoke-Native "winget install --id $($tool.WingetId) -e --accept-source-agreements --accept-package-agreements --disable-interactivity"
if ($LASTEXITCODE -ne 0) {
Write-Warn2 "winget exited with $LASTEXITCODE for $($tool.Name); checking whether the tool landed anyway..."
}
Add-SessionPath $tool.PathDirs
if (-not (Get-Command $tool.Exe -ErrorAction SilentlyContinue)) {
Die "$($tool.Name) still not found after install. Open a new terminal (PATH refresh) and re-run, or install manually."
}
Write-Log "$($tool.Name) OK."
}
}
function Install-PortableMySql {
if (Test-Path (Join-Path $MySqlBaseDir "bin\mysqld.exe")) {
Write-Log "Portable MySQL already extracted."
} elseif ((Test-TcpPort 3306)) {
# Something else already serves 3306 (e.g. a system-wide MySQL install).
# Use it rather than fighting over the port; provisioning below needs root
# credentials via MYSQL_ROOT_PASSWORD if that server has a root password.
Write-Warn2 "Port 3306 is already in use by an existing MySQL server - using it instead of the portable install."
if (-not (Get-Command mysql -ErrorAction SilentlyContinue)) {
Die "A server owns port 3306 but no 'mysql' client is on PATH. Add the client or stop that server."
}
return
} else {
New-Item -ItemType Directory -Force $DownloadsDir | Out-Null
$zipPath = Join-Path $DownloadsDir "mysql-$MySqlVersion-winx64.zip"
if (-not (Test-Path $zipPath)) {
Write-Log "Downloading MySQL $MySqlVersion (~250 MB, official cdn.mysql.com)..."
Download-File $MySqlZipUrl $zipPath
}
Write-Log "Extracting..."
Expand-Archive -Path $zipPath -DestinationPath (Join-Path $DevStackDir "mysql") -Force
if (-not (Test-Path (Join-Path $MySqlBaseDir "bin\mysqld.exe"))) {
Die "Extraction finished but $MySqlBaseDir\bin\mysqld.exe is missing."
}
}
if (-not (Test-Path $MySqlIniPath)) {
# Minimal config; utf8mb4 collation comes from each migration's DDL.
# MySQL's ini parser escape-processes backslashes - use forward slashes,
# and quote the values in case the clone path contains spaces.
$baseForward = $MySqlBaseDir -replace '\\', '/'
$dataForward = $MySqlDataDir -replace '\\', '/'
$ini = @"
[mysqld]
basedir="$baseForward"
datadir="$dataForward"
port=3306
bind-address=127.0.0.1
"@
New-Item -ItemType Directory -Force $DevStackDir | Out-Null
Set-Content -Path $MySqlIniPath -Value $ini -Encoding ascii
}
if (-not (Test-Path (Join-Path $MySqlDataDir "mysql"))) {
Write-Log "Initializing MySQL data directory (root without password, local dev only)..."
$mysqld = Join-Path $MySqlBaseDir "bin\mysqld.exe"
Invoke-Native "`"$mysqld`" --defaults-file=`"$MySqlIniPath`" --initialize-insecure --console"
if ($LASTEXITCODE -ne 0) { Die "mysqld --initialize-insecure failed (exit $LASTEXITCODE). See output above." }
}
}
function Provision-MySqlDatabase {
if (Test-MySqlApp) {
Write-Log "Database '$DevMySqlDb' and user '$DevMySqlUser' already provisioned."
return
}
$mysql = Get-MySqlTool "mysql"
if (-not $mysql) { Die "mysql client not found (portable install missing and none on PATH)." }
# Portable install: root has no password. External server: honor MYSQL_ROOT_PASSWORD.
$prev = $env:MYSQL_PWD
if ($env:MYSQL_ROOT_PASSWORD) { $env:MYSQL_PWD = $env:MYSQL_ROOT_PASSWORD } else { $env:MYSQL_PWD = "" }
try {
# Same DDL as _docs/dev_setup_server.md, made idempotent. The extra
# '127.0.0.1' account covers hosts where reverse-DNS of 127.0.0.1 doesn't
# resolve to 'localhost'. Fed through a temp file + `source` so mysql's
# stderr can never become a terminating PowerShell error mid-provision.
$ddl = @"
CREATE DATABASE IF NOT EXISTS $DevMySqlDb;
CREATE USER IF NOT EXISTS '$DevMySqlUser'@'localhost' IDENTIFIED BY '$DevMySqlPassword';
CREATE USER IF NOT EXISTS '$DevMySqlUser'@'127.0.0.1' IDENTIFIED BY '$DevMySqlPassword';
GRANT ALL PRIVILEGES ON $DevMySqlDb.* TO '$DevMySqlUser'@'localhost';
GRANT ALL PRIVILEGES ON $DevMySqlDb.* TO '$DevMySqlUser'@'127.0.0.1';
FLUSH PRIVILEGES;
"@
$ddlFile = Join-Path $env:TEMP "artcraft_provision_ddl.sql"
Set-Content -Path $ddlFile -Value $ddl -Encoding ascii
$ddlForward = $ddlFile -replace '\\', '/'
$out = Invoke-Native "`"$mysql`" -u root -h 127.0.0.1 -e `"source $ddlForward`""
Remove-Item $ddlFile -Force -Confirm:$false -ErrorAction SilentlyContinue
if ($LASTEXITCODE -ne 0) {
if ($out) { Write-Host ($out -join "`n") }
Die "Provisioning failed. For an external MySQL server, set MYSQL_ROOT_PASSWORD and re-run."
}
} finally {
$env:MYSQL_PWD = $prev
}
if (-not (Test-MySqlApp)) { Die "Provisioning ran but '$DevMySqlUser' still cannot reach '$DevMySqlDb'." }
Write-Log "Provisioned."
}
function Install-PortableRedis {
if (Test-Path (Join-Path $RedisDir "redis-server.exe")) {
Write-Log "Portable Redis already extracted."
return
}
if (Test-TcpPort 6379) {
Write-Warn2 "Port 6379 is already served (existing Redis/Memurai?) - using it."
return
}
New-Item -ItemType Directory -Force $DownloadsDir | Out-Null
$zipPath = Join-Path $DownloadsDir "redis-win-x64.zip"
if (-not (Test-Path $zipPath)) {
Write-Log "Downloading Redis for Windows (~12 MB)..."
Download-File $RedisZipUrl $zipPath
}
Write-Log "Extracting..."
Expand-Archive -Path $zipPath -DestinationPath $RedisDir -Force
if (-not (Test-Path (Join-Path $RedisDir "redis-server.exe"))) {
Die "Extraction finished but $RedisDir\redis-server.exe is missing."
}
}
function Ensure-DieselCli {
Add-MySqlClientToSession
$needInstall = $true
$forceFlag = ""
if (Get-Command diesel -ErrorAction SilentlyContinue) {
# A diesel.exe built against a client lib that has since moved fails at
# load time - probe it rather than trusting its presence.
$version = Invoke-Native "diesel --version"
if ($LASTEXITCODE -eq 0) {
Write-Log "Found $version."
$needInstall = $false
} else {
Write-Warn2 "diesel is on PATH but broken (missing libmysql.dll?) - reinstalling."
$forceFlag = " --force"
}
}
if (-not $needInstall) { return }
# diesel_cli's mysql feature links against libmysqlclient. The portable MySQL
# ships its own client lib - point mysqlclient-sys at it instead of requiring
# vcpkg. sqlite is bundled so no system sqlite is needed.
if (-not (Test-Path (Join-Path $MySqlBaseDir "lib\libmysql.lib"))) {
Die "MySQL client lib not found under $MySqlBaseDir\lib (needed to compile diesel_cli)."
}
Write-Log "Installing diesel_cli (compiles from source; a few minutes)..."
$env:MYSQLCLIENT_LIB_DIR = Join-Path $MySqlBaseDir "lib"
$env:MYSQLCLIENT_VERSION = $MySqlVersion
Invoke-Native "cargo install diesel_cli --no-default-features --features mysql,sqlite-bundled$forceFlag"
if ($LASTEXITCODE -ne 0) { Die "cargo install diesel_cli failed (exit $LASTEXITCODE)." }
Write-Log "Installed $(Invoke-Native 'diesel --version')."
}
function Run-Migrations {
# diesel-cli reads DATABASE_URL (also provided by the repo-root .env, which
# diesel auto-loads from the cwd). diesel.exe needs libmysql.dll at runtime -
# Add-MySqlClientToSession put the portable lib dirs on PATH.
if (-not $env:DATABASE_URL) {
$env:DATABASE_URL = "mysql://${DevMySqlUser}:${DevMySqlPassword}@localhost/${DevMySqlDb}"
}
Push-Location $RootDir
try {
# NB: "Encountered unknown type for Mysql: enum" warnings are harmless.
Invoke-Native "diesel migration run"
if ($LASTEXITCODE -ne 0) { Die "diesel migration run failed (exit $LASTEXITCODE)." }
} finally {
Pop-Location
}
Write-Log "Migrations up to date."
}
function Seed-RolesAndBadges {
# Same data as _database/sql/seed/bootstrap_inserts_roles_etc.sh, but guarded
# so re-runs don't hit duplicate-key errors. The 'user' role is mandatory:
# account creation hardcodes user_role_slug='user'.
$roleCount = Get-MySqlCount "SELECT COUNT(*) FROM user_roles"
if ($roleCount -lt 0) { Die "Could not query user_roles - are migrations applied? (re-run bootstrap)" }
if ($roleCount -gt 0) {
Write-Log "user_roles already seeded ($roleCount rows)."
} else {
Invoke-MySqlApp -SourceFile (Join-Path $RootDir "_database\sql\seed\sql\system_roles.sql")
if ($LASTEXITCODE -ne 0) { Die "Seeding system_roles.sql failed." }
Write-Log "Inserted system roles (user, mod, admin)."
}
$badgeCount = Get-MySqlCount "SELECT COUNT(*) FROM badges"
if ($badgeCount -lt 0) { Die "Could not query badges - are migrations applied? (re-run bootstrap)" }
if ($badgeCount -gt 0) {
Write-Log "badges already seeded ($badgeCount rows)."
} else {
Invoke-MySqlApp -SourceFile (Join-Path $RootDir "_database\sql\seed\sql\user_badges.sql")
if ($LASTEXITCODE -ne 0) { Die "Seeding user_badges.sql failed." }
Write-Log "Inserted badges."
}
}
function Write-SecretsEnvIfMissing {
if (Test-Path $SecretsEnvFile) {
Write-Log "Already exists: $SecretsEnvFile (leaving it untouched)."
Add-DevStackKeysToExistingSecretsEnv
return
}
# Every var below is read with get_env_string_required at server startup but
# is NOT supplied by the checked-in config files. None is contacted at boot -
# placeholders are enough to run the server.
#
# NOTE: config loading is FIRST-VALUE-WINS (dotenv never overrides a set
# key), so values here cannot override storyteller-web.development.env.
#
# TEMP_DIR: the upload temp-dir helper defaults to /tmp, which doesn't exist
# on Windows - point it at the user temp dir (forward slashes work fine).
$tempDirForward = $env:TEMP -replace '\\', '/'
$content = @"
# Local-development secrets for storyteller-web. Generated by
# script/bootstrap/windows/bootstrap_dev_stack.ps1 - gitignored; edit freely.
#
# Placeholders below satisfy required-at-boot checks without enabling any
# external integration. Replace individual values with real dev credentials
# only if you need that specific integration locally.
# Windows: scoped upload temp dirs (code default is /tmp)
TEMP_DIR=$tempDirForward
# R2/S3 object storage (clients are constructed offline)
ACCESS_KEY=dummy-local-dev
SECRET_KEY=dummy-local-dev
REGION_NAME=us-east-1
W2L_PRIVATE_DOWNLOAD_BUCKET_NAME=dummy-local-dev-private
W2L_PUBLIC_DOWNLOAD_BUCKET_NAME=dummy-local-dev-public
# Email (Resend)
RESEND_API_KEY=dummy-local-dev
# Generation providers. NB: the omni_gen endpoints (webapp image/video
# generation) contact providers SYNCHRONOUSLY in the request handler, so with
# dummy keys a real submit would 500 - DEV_FAKE_GENERATION below avoids that.
# The legacy polling providers are only contacted by worker binaries, which
# local dev does not run.
FAL_API_KEY=dummy-local-dev
GMICLOUD_API_KEY=dummy-local-dev
GROK_API_KEY=dummy-local-dev
BEEBLE_API_KEY=dummy-local-dev
OPENAI_API_KEY=dummy-local-dev
WORLDLABS_API_KEY=dummy-local-dev
SEEDANCE2PRO_COOKIES=dummy-local-dev
SEEDANCE2PRO_WHITELIST_COOKIES=dummy-local-dev
SEEDANCE2PRO_BYTEPLUS_ULTRA_COOKIES=dummy-local-dev
# Stripe account ids (test-mode secret keys are already checked into
# storyteller-web.development.env; only the account ids are missing there)
STRIPE_FAKEYOU_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_ACCOUNT_ID=acct_dummylocaldev
STRIPE_ARTCRAFT_SECRET_KEY=sk_test_dummylocaldev
STRIPE_ARTCRAFT_SECRET_WEBHOOK_KEY=whsec_dummylocaldev
$(Get-DevStackKeysBlock)
"@
Set-Content -Path $SecretsEnvFile -Value $content -Encoding ascii
Write-Log "Wrote $SecretsEnvFile"
}
# The fully-local media + fake-generation settings (development-only; the
# server hard-ignores DEV_FAKE_GENERATION outside SERVER_ENVIRONMENT=Development).
function Get-DevStackKeysBlock {
$mediaRootForward = (Join-Path $DevStackDir "media") -replace '\\', '/'
return @"
# Fully-local dev stack (see _docs/dev_setup_local_stack.md):
# - DEV_FAKE_GENERATION: omni_gen image/video submits skip real providers and
# a resolver thread completes them a few seconds later with a placeholder
# result (or fails them when the prompt contains simulate_artcraft_failure).
# - CDN_BASE_URL: media URLs in API responses point at this server instead of
# the shared r2.dev bucket. Host only - no path suffix.
# - LOCAL_MEDIA_ROOT: directory served at /media (mirrors the bucket layout);
# seeded gallery media and fake generation results live here.
DEV_FAKE_GENERATION=true
CDN_BASE_URL=http://localhost:12345
LOCAL_MEDIA_ROOT=$mediaRootForward
"@
}
# Bootstrap re-runs on a stack whose secrets file predates the fully-local
# media/fake-generation support: append the new keys (first-value-wins never
# lets these override anything the user already set).
function Add-DevStackKeysToExistingSecretsEnv {
$existing = Get-Content $SecretsEnvFile -Raw
if ($existing -match '(?m)^\s*LOCAL_MEDIA_ROOT=') { return }
Add-Content -Path $SecretsEnvFile -Value "`r`n$(Get-DevStackKeysBlock)" -Encoding ascii
Write-Log "Appended fully-local dev stack keys (DEV_FAKE_GENERATION, CDN_BASE_URL, LOCAL_MEDIA_ROOT) to $SecretsEnvFile"
}
function Build-Backend {
if ($SkipRustBuild) {
Write-Log "Skipped (-SkipRustBuild)."
return
}
Add-BuildToolsToSession
Push-Location $RootDir
try {
# SQLX_OFFLINE only affects compile-time query verification (checked-in
# .sqlx/ cache instead of a live DB); it has no runtime effect.
$env:SQLX_OFFLINE = "true"
Write-Log "Building (first build takes several minutes)..."
Invoke-Native "cargo build -p storyteller-web --bin storyteller-web"
if ($LASTEXITCODE -ne 0) { Die "cargo build failed (exit $LASTEXITCODE)." }
} finally {
Pop-Location
}
Write-Log "Built target\debug\storyteller-web.exe."
}
function Setup-Frontend {
if ($SkipFrontend) {
Write-Log "Skipped (-SkipFrontend)."
return
}
$frontendPath = Join-Path $RootDir "frontend"
# Mirror script/common/frontend_preflight.sh: the workspace uses npm; stale
# pnpm artifacts break installs with ENOTEMPTY.
if ((Test-Path (Join-Path $frontendPath "pnpm-lock.yaml")) -or
(Test-Path (Join-Path $frontendPath "pnpm-workspace.yaml")) -or
(Test-Path (Join-Path $frontendPath "node_modules\.pnpm"))) {
Die "Stale pnpm artifacts detected in frontend\. Clean up (see frontend/README.md): remove node_modules, pnpm-lock.yaml, pnpm-workspace.yaml, then re-run."
}
Push-Location $frontendPath
try {
Invoke-Native "npm install"
if ($LASTEXITCODE -ne 0) {
Write-Warn2 "npm install failed; wiping node_modules + .nx and retrying once..."
Remove-Item -Recurse -Force -Confirm:$false node_modules, .nx -ErrorAction SilentlyContinue
Invoke-Native "npm install"
if ($LASTEXITCODE -ne 0) { Die "npm install failed again. See frontend/README.md troubleshooting." }
}
} finally {
Pop-Location
}
Write-Log "Frontend dependencies installed."
}
function Download-File([string]$Url, [string]$Destination) {
# curl.exe ships with Windows 10+ and handles redirects/resume better than
# Invoke-WebRequest (which is also painfully slow on large files in PS 5.1).
# -sS: no progress meter (it goes to stderr, which pollutes captured logs),
# but errors still print.
$out = Invoke-Native "curl.exe -fsSL --retry 3 -o `"$Destination.partial`" `"$Url`""
if ($LASTEXITCODE -ne 0) {
if ($out) { Write-Host ($out -join "`n") }
Remove-Item "$Destination.partial" -Force -Confirm:$false -ErrorAction SilentlyContinue
Die "Download failed: $Url"
}
Move-Item "$Destination.partial" $Destination -Force
}
function Print-NextSteps {
Step "Done"
Write-Host @"
The dev stack is bootstrapped. Next steps:
1. Start the backend (binds $DevBackendUrl):
.\script\bootstrap\windows\run_backend_dev.ps1
2. In another terminal, create the demo login (idempotent):
.\script\bootstrap\windows\seed_demo_user.ps1
Credentials: $DemoUsername / $DemoPassword
3. Start the webapp (binds http://localhost:4201; in dev it talks to
$DevBackendUrl automatically):
cd frontend; npx nx dev artcraft-webapp
Health checks at any time:
.\script\bootstrap\windows\dev_stack_doctor.ps1
Services (MySQL/Redis) start on demand and persist; stop them with:
.\script\bootstrap\windows\stop_dev_services.ps1
"@
}
Main
+239
View File
@@ -0,0 +1,239 @@
# Shared helpers for the Windows dev-stack bootstrap scripts.
#
# Dot-source from a sibling script:
# . (Join-Path $PSScriptRoot "common.ps1")
#
# Design: the stack is PORTABLE - MySQL and Redis live under the gitignored
# .devstack/ directory and run as plain user processes. No admin rights, no
# Windows services, no system-wide installs (build tools excepted).
# See _docs/dev_setup_local_stack.md.
$ErrorActionPreference = "Stop"
$script:RootDir = (Resolve-Path (Join-Path $PSScriptRoot "..\..\..")).Path
# Canonical local-dev database identity - must match _docs/dev_setup_server.md,
# the repo-root .env DATABASE_URL, and the server's MYSQL_URL default.
$script:DevMySqlDb = if ($env:DEV_MYSQL_DB) { $env:DEV_MYSQL_DB } else { "storyteller" }
$script:DevMySqlUser = if ($env:DEV_MYSQL_USER) { $env:DEV_MYSQL_USER } else { "storyteller" }
$script:DevMySqlPassword = if ($env:DEV_MYSQL_PASSWORD) { $env:DEV_MYSQL_PASSWORD } else { "password" }
$script:DevBackendUrl = if ($env:DEV_BACKEND_URL) { $env:DEV_BACKEND_URL } else { "http://localhost:12345" }
# Default demo account (see seed_demo_user.ps1). "demo"/"admin"/"test"/"dev"
# are on the backend's reserved-usernames list - don't use those.
$script:DemoUsername = if ($env:DEMO_USERNAME) { $env:DEMO_USERNAME } else { "localdev1" }
$script:DemoPassword = if ($env:DEMO_PASSWORD) { $env:DEMO_PASSWORD } else { "localdev1pass" }
$script:DemoEmail = if ($env:DEMO_EMAIL) { $env:DEMO_EMAIL } else { "localdev1@example.com" }
# Banked credits the demo user's artcraft wallet is seeded/topped-up to.
# Generation submits are gated on wallet balance, so 0 credits means the
# generate button silently refuses in the webapp.
$script:DemoCredits = if ($env:DEMO_CREDITS) { [uint32]$env:DEMO_CREDITS } else { 100000 }
# --- Portable service pins -------------------------------------------------
# MySQL 8.4 LTS (the 8.x series is required; 9.x is unsupported by tooling).
# cdn.mysql.com hosts only some patch versions; 8.4.11 verified available.
$script:MySqlVersion = "8.4.11"
$script:MySqlZipUrl = "https://cdn.mysql.com/Downloads/MySQL-8.4/mysql-$MySqlVersion-winx64.zip"
# Redis for Windows (tporadowski build). Old (5.0) but stable and sufficient
# for the backend's usage (keepalive/TTL/rate-limiter commands). Swap to
# Memurai or Microsoft Garnet by pointing REDIS_ZIP_URL elsewhere.
$script:RedisZipUrl = if ($env:REDIS_ZIP_URL) { $env:REDIS_ZIP_URL } else {
"https://github.com/tporadowski/redis/releases/download/v5.0.14.1/Redis-x64-5.0.14.1.zip"
}
$script:DevStackDir = Join-Path $RootDir ".devstack"
$script:DownloadsDir = Join-Path $DevStackDir "downloads"
$script:MySqlBaseDir = Join-Path $DevStackDir "mysql\mysql-$MySqlVersion-winx64"
$script:MySqlDataDir = Join-Path $DevStackDir "mysql-data"
$script:MySqlIniPath = Join-Path $DevStackDir "my.ini"
$script:RedisDir = Join-Path $DevStackDir "redis"
$script:LogsDir = Join-Path $DevStackDir "logs"
$script:PidsDir = Join-Path $DevStackDir "pids"
# Local media served by the backend at /media when LOCAL_MEDIA_ROOT points
# here (fully-local gallery + fake generation results). Mirrors the public
# bucket layout: object path /media/{...} lives at .devstack\media\media\{...}.
$script:DevMediaRoot = Join-Path $DevStackDir "media"
$script:SecretsEnvFile = Join-Path $RootDir "crates\service\web\storyteller_web\config\storyteller-web.development-secrets.env"
function Write-Log([string]$Message) { Write-Host "[bootstrap] $Message" }
function Write-Warn2([string]$Message) { Write-Host "[bootstrap] WARNING: $Message" -ForegroundColor Yellow }
function Die([string]$Message) { Write-Host "[bootstrap] ERROR: $Message" -ForegroundColor Red; exit 1 }
function Step([string]$Title) { Write-Host ""; Write-Host "=== $Title ===" -ForegroundColor Cyan }
function Confirm-Or-Skip([string]$Prompt) {
if ($script:AssumeYes) { return $true }
$reply = Read-Host "$Prompt [y/N]"
return ($reply -match '^[Yy]')
}
function Test-TcpPort([int]$Port) {
$client = New-Object System.Net.Sockets.TcpClient
try {
$async = $client.BeginConnect("127.0.0.1", $Port, $null, $null)
if (-not $async.AsyncWaitHandle.WaitOne(1500)) { return $false }
$client.EndConnect($async)
return $true
} catch {
return $false
} finally {
$client.Close()
}
}
# Resolve a client/server exe: prefer the portable install, fall back to PATH.
function Get-MySqlTool([string]$Name) {
$portable = Join-Path $MySqlBaseDir "bin\$Name.exe"
if (Test-Path $portable) { return $portable }
$cmd = Get-Command $Name -ErrorAction SilentlyContinue
if ($cmd) { return $cmd.Source }
return $null
}
function Get-RedisTool([string]$Name) {
$portable = Join-Path $RedisDir "$Name.exe"
if (Test-Path $portable) { return $portable }
$cmd = Get-Command $Name -ErrorAction SilentlyContinue
if ($cmd) { return $cmd.Source }
return $null
}
# Generate a token in the backend's format: {prefix}{crockford-lower entropy}
# padded to the same TOTAL length the Rust generators use (tokens crate,
# impl_crockford_generator!). Alphabet excludes i/l/o/u like Crockford base32.
function New-DevToken([string]$Prefix, [int]$TotalLength) {
$charset = "0123456789abcdefghjkmnpqrstvwxyz"
$count = $TotalLength - $Prefix.Length
$chars = 1..$count | ForEach-Object { $charset[(Get-Random -Maximum $charset.Length)] }
return "$Prefix$(-join $chars)"
}
# Run SQL as the app user against the dev database. Returns output lines
# (stdout and stderr merged via Invoke-Native, so a caller capturing our
# streams can never turn a mysql warning into a terminating error).
# Password goes via MYSQL_PWD so it stays off the command line.
# -Quiet discards output (for probes that are expected to fail, e.g.
# Test-MySqlApp before provisioning). Check $LASTEXITCODE for success.
function Invoke-MySqlApp([string]$Sql, [string]$SourceFile, [switch]$Quiet) {
$mysql = Get-MySqlTool "mysql"
if (-not $mysql) { Die "mysql client not found (run bootstrap_dev_stack.ps1 first)." }
$prev = $env:MYSQL_PWD
$env:MYSQL_PWD = $DevMySqlPassword
try {
if ($SourceFile) {
$forward = $SourceFile -replace '\\', '/'
$out = Invoke-Native "`"$mysql`" -u $DevMySqlUser -h 127.0.0.1 -D $DevMySqlDb -e `"source $forward`""
} else {
$out = Invoke-Native "`"$mysql`" -u $DevMySqlUser -h 127.0.0.1 -D $DevMySqlDb -N -e `"$Sql`""
}
if (-not $Quiet) { return $out }
} finally {
$env:MYSQL_PWD = $prev
}
}
# Run a query expected to return a single non-negative integer (e.g. COUNT(*)).
# Returns -1 if the query failed or produced no numeric line, so callers can
# distinguish "table is empty" (0) from "query failed" (-1).
function Get-MySqlCount([string]$Sql) {
$out = @(Invoke-MySqlApp -Sql $Sql)
if ($LASTEXITCODE -ne 0) { return -1 }
$line = $out | Where-Object { "$_" -match '^\d+$' } | Select-Object -First 1
if ($null -eq $line) { return -1 }
return [int]"$line"
}
function Test-MySqlApp {
try {
Invoke-MySqlApp -Sql "SELECT 1" -Quiet
return ($LASTEXITCODE -eq 0)
} catch {
return $false
}
}
function Test-Redis {
$cli = Get-RedisTool "redis-cli"
if (-not $cli) { return $false }
try {
$pong = Invoke-Native "`"$cli`" -h 127.0.0.1 ping"
return ("$pong" -match "PONG")
} catch {
return $false
}
}
function Test-Backend {
try {
$r = Invoke-WebRequest -Uri "$DevBackendUrl/_status" -UseBasicParsing -TimeoutSec 3
return ($r.StatusCode -eq 200)
} catch {
return $false
}
}
# Start a detached background process, remembering its PID for stop_dev_services.
function Start-DevProcess([string]$Name, [string]$Exe, [string[]]$Arguments) {
New-Item -ItemType Directory -Force $PidsDir | Out-Null
New-Item -ItemType Directory -Force $LogsDir | Out-Null
$out = Join-Path $LogsDir "$Name.out.log"
$err = Join-Path $LogsDir "$Name.err.log"
$proc = Start-Process -FilePath $Exe -ArgumentList $Arguments -WindowStyle Hidden `
-RedirectStandardOutput $out -RedirectStandardError $err -PassThru
Set-Content -Path (Join-Path $PidsDir "$Name.pid") -Value $proc.Id -Encoding ascii
return $proc
}
function Wait-ForCondition([scriptblock]$Condition, [int]$TimeoutSeconds, [string]$What) {
$deadline = (Get-Date).AddSeconds($TimeoutSeconds)
while ((Get-Date) -lt $deadline) {
if (& $Condition) { return $true }
Start-Sleep -Seconds 1
}
Die "Timed out after ${TimeoutSeconds}s waiting for: $What"
}
# Run a native command line via cmd so its stderr is merged OUTSIDE PowerShell.
# PS 5.1 wraps captured native stderr lines in ErrorRecords; combined with our
# $ErrorActionPreference = "Stop", a harmless warning (e.g. from cargo) would
# otherwise abort the script whenever a caller captures/redirects our output.
# Quote embedded paths: Invoke-Native "`"$exe`" --flag value"
function Invoke-Native([string]$CommandLine) {
cmd /s /c "$CommandLine 2>&1"
}
# Make freshly-installed tools visible in this session without a new shell.
function Add-SessionPath([string[]]$Dirs) {
foreach ($d in $Dirs) {
if ((Test-Path $d) -and (($env:Path -split ';') -notcontains $d)) {
$env:Path = "$d;$env:Path"
}
}
}
# The portable MySQL's client DLL + import lib, needed to build AND run
# diesel-cli with the mysql feature.
function Add-MySqlClientToSession {
Add-SessionPath @((Join-Path $MySqlBaseDir "bin"), (Join-Path $MySqlBaseDir "lib"))
}
# Native build tools (cmake/perl/nasm/llvm) land in these dirs when installed
# via winget; a shell opened before the install won't have them on PATH yet.
# bindgen (used by boring2 / aws-lc-sys) additionally wants LIBCLANG_PATH.
function Add-BuildToolsToSession {
Add-SessionPath @(
"$env:ProgramFiles\CMake\bin",
"C:\Strawberry\perl\bin",
"$env:ProgramFiles\NASM",
"$env:LOCALAPPDATA\bin\NASM",
"$env:ProgramFiles\LLVM\bin"
)
if (-not $env:LIBCLANG_PATH) {
$clang = Get-Command clang -ErrorAction SilentlyContinue
if ($clang) { $env:LIBCLANG_PATH = Split-Path $clang.Source }
}
}
@@ -0,0 +1,171 @@
# Diagnose the local dev stack on Windows: checks every prerequisite the
# bootstrap sets up and reports PASS / WARN / FAIL per item.
#
# FAIL = the stack cannot work until fixed (exit code 1).
# WARN = optional or run-time-only (e.g. backend not currently running).
param()
. (Join-Path $PSScriptRoot "common.ps1")
$ErrorActionPreference = "Continue"
$script:Failures = 0
$script:Warnings = 0
function Pass([string]$Message) { Write-Host " PASS $Message" }
function Fail([string]$Message) { Write-Host " FAIL $Message" -ForegroundColor Red; $script:Failures++ }
function WarnCheck([string]$Message) { Write-Host " WARN $Message" -ForegroundColor Yellow; $script:Warnings++ }
Write-Host "Dev stack doctor ($RootDir)"
Write-Host ""
Write-Host "--- Toolchains ---"
Add-SessionPath @((Join-Path $env:USERPROFILE ".cargo\bin"))
Add-MySqlClientToSession
# Freshly-installed build tools may not be on this session's PATH yet.
Add-BuildToolsToSession
if (Get-Command cargo -ErrorAction SilentlyContinue) {
Pass "rust: $(rustc --version)"
} else {
Fail "rust: cargo not on PATH (winget install Rustlang.Rustup)"
}
if (Get-Command diesel -ErrorAction SilentlyContinue) {
$dv = Invoke-Native "diesel --version"
if ($LASTEXITCODE -eq 0) { Pass "diesel_cli: $dv" }
else { Fail "diesel_cli: present but broken (libmysql.dll missing from PATH? re-run bootstrap)" }
} else {
Fail "diesel_cli: not installed (bootstrap installs it against the portable MySQL client lib)"
}
if (Get-Command node -ErrorAction SilentlyContinue) {
$nodeMajor = [int]((node --version) -replace '^v(\d+).*', '$1')
if ($nodeMajor -ge 20) { Pass "node: $(node --version) (>= 20 required by Nx 21 / Vite 6)" }
else { Fail "node: $(node --version) is too old - Node 20+ required" }
} else {
Fail "node: not installed (Node 20+ required for the frontend)"
}
foreach ($tool in @("cmake", "perl", "nasm", "clang")) {
if (Get-Command $tool -ErrorAction SilentlyContinue) {
Pass "${tool}: found"
} else {
WarnCheck "${tool}: not on PATH (needed to COMPILE the backend; bootstrap offers winget installs)"
}
}
Write-Host ""
Write-Host "--- Services (portable, under .devstack\) ---"
if (Test-Path (Join-Path $MySqlBaseDir "bin\mysqld.exe")) {
Pass "mysql: portable install present ($MySqlVersion)"
} elseif (Test-TcpPort 3306) {
WarnCheck "mysql: no portable install, but something serves port 3306 (external server assumed)"
} else {
Fail "mysql: not installed (run bootstrap_dev_stack.ps1)"
}
if (Test-TcpPort 3306) {
if (Test-MySqlApp) {
Pass "mysql: reachable as '$DevMySqlUser' on database '$DevMySqlDb'"
$migrationDirs = (Get-ChildItem -Directory (Join-Path $RootDir "_database\sql\migrations")).Count
$applied = Get-MySqlCount "SELECT COUNT(*) FROM __diesel_schema_migrations"
if (($applied -ge $migrationDirs) -and ($applied -gt 0)) {
Pass "migrations: $applied applied ($migrationDirs in _database\sql\migrations)"
} elseif ($applied -gt 0) {
Fail "migrations: only $applied/$migrationDirs applied - run 'diesel migration run'"
} else {
Fail "migrations: none applied - run 'diesel migration run' (or re-run bootstrap)"
}
$roleCount = Get-MySqlCount "SELECT COUNT(*) FROM user_roles"
if ($roleCount -ge 3) {
Pass "seed: user_roles has $roleCount rows (user/mod/admin present)"
} else {
Fail "seed: user_roles has $roleCount rows - account creation needs the 'user' role (re-run bootstrap)"
}
$badgeCount = Get-MySqlCount "SELECT COUNT(*) FROM badges"
if ($badgeCount -gt 0) { Pass "seed: badges has $badgeCount rows" }
else { WarnCheck "seed: badges table is empty (cosmetic; re-run bootstrap to fill)" }
$demoCount = Get-MySqlCount "SELECT COUNT(*) FROM users WHERE username='$DemoUsername'"
if ($demoCount -gt 0) { Pass "demo user: '$DemoUsername' exists" }
else { WarnCheck "demo user: '$DemoUsername' not created yet (seed_demo_user.ps1, backend must be running)" }
if ($demoCount -gt 0) {
$credits = Get-MySqlCount "SELECT COALESCE(MAX(banked_credits + monthly_credits), 0) FROM wallets WHERE wallet_namespace='artcraft' AND owner_user_token=(SELECT token FROM users WHERE username='$DemoUsername')"
if ($credits -gt 0) { Pass "demo wallet: '$DemoUsername' has $credits credits" }
else { WarnCheck "demo wallet: '$DemoUsername' has no credits - generation flows are blocked (re-run seed_demo_user.ps1)" }
$mediaCount = Get-MySqlCount "SELECT COUNT(*) FROM media_files WHERE maybe_creator_user_token=(SELECT token FROM users WHERE username='$DemoUsername')"
if ($mediaCount -gt 0) { Pass "demo media: '$DemoUsername' has $mediaCount media rows" }
else { WarnCheck "demo media: gallery is empty (re-run seed_demo_user.ps1)" }
}
} else {
Fail "mysql: port 3306 is up but '$DevMySqlUser' cannot access '$DevMySqlDb' (re-run bootstrap provisioning)"
}
} else {
Fail "mysql: nothing listening on 3306 (start_dev_services.ps1)"
}
if (Test-Redis) {
Pass "redis: PONG"
} elseif (Test-TcpPort 6379) {
WarnCheck "redis: port 6379 open but redis-cli not found to verify"
} else {
Fail "redis: not reachable on 6379 (the backend's r2d2 pool connects eagerly at boot; start_dev_services.ps1)"
}
Write-Host ""
Write-Host "--- Backend ---"
if (Test-Path $SecretsEnvFile) {
Pass "secrets env: storyteller-web.development-secrets.env exists"
$secretsContent = Get-Content $SecretsEnvFile -Raw
if ($secretsContent -match '(?m)^\s*LOCAL_MEDIA_ROOT=') {
Pass "secrets env: fully-local dev stack keys present (DEV_FAKE_GENERATION, CDN_BASE_URL, LOCAL_MEDIA_ROOT)"
} else {
WarnCheck "secrets env: fully-local dev stack keys missing - generation/gallery use remote CDN and real providers (re-run bootstrap to append)"
}
} else {
Fail "secrets env: missing - the server aborts at boot on ~19 required vars (re-run bootstrap)"
}
if ((Test-Path (Join-Path $RootDir "target\debug\storyteller-web.exe")) -or
(Test-Path (Join-Path $RootDir "target\release\storyteller-web.exe"))) {
Pass "binary: storyteller-web.exe is built"
} else {
WarnCheck "binary: storyteller-web not built yet (bootstrap builds it; cargo run builds on demand)"
}
if (Test-Backend) {
Pass "server: $DevBackendUrl/_status responds"
} else {
WarnCheck "server: not running (run_backend_dev.ps1)"
}
Write-Host ""
Write-Host "--- Frontend ---"
if (Test-Path (Join-Path $RootDir "frontend\node_modules")) {
Pass "frontend: node_modules present"
} else {
WarnCheck "frontend: node_modules missing (cd frontend; npm install)"
}
if ((Test-Path (Join-Path $RootDir "frontend\pnpm-lock.yaml")) -or
(Test-Path (Join-Path $RootDir "frontend\node_modules\.pnpm"))) {
Fail "frontend: stale pnpm artifacts detected (see frontend/README.md - the workspace uses npm)"
} else {
Pass "frontend: no stale pnpm artifacts"
}
Write-Host ""
Write-Host "Summary: $Failures failure(s), $Warnings warning(s)."
if ($Failures -gt 0) {
Write-Host "Fix failures with: .\script\bootstrap\windows\bootstrap_dev_stack.ps1 (safe to re-run)"
exit 1
}
@@ -0,0 +1,33 @@
# Run the storyteller-web backend for local development (binds 0.0.0.0:12345).
#
# Requires a bootstrapped environment (bootstrap_dev_stack.ps1). Starts the
# portable MySQL/Redis if they aren't running. Always runs from the repo root:
# the server's config search path (crates/service/web/storyteller_web/config)
# and its includes/ defaults are cwd-relative.
param()
. (Join-Path $PSScriptRoot "common.ps1")
. (Join-Path $PSScriptRoot "start_dev_services.ps1")
Start-DevMySql
Start-DevRedis
if (-not (Test-MySqlApp)) { Die "MySQL is up but '$DevMySqlUser' cannot reach '$DevMySqlDb'. Run bootstrap_dev_stack.ps1." }
if (-not (Test-Path $SecretsEnvFile)) { Die "Missing $SecretsEnvFile. Run bootstrap_dev_stack.ps1." }
Add-SessionPath @((Join-Path $env:USERPROFILE ".cargo\bin"))
# cargo run may need to (re)compile, which needs the native build toolchain.
Add-BuildToolsToSession
Push-Location $RootDir
try {
# SERVER_ENVIRONMENT defaults to Development when unset; dev config files are
# picked up from the search path automatically. SQLX_OFFLINE only affects
# compile-time query checking, never runtime. TEMP_DIR belt-and-suspenders in
# case an older generated secrets file predates the Windows TEMP_DIR line.
$env:SQLX_OFFLINE = "true"
if (-not $env:TEMP_DIR) { $env:TEMP_DIR = $env:TEMP -replace '\\', '/' }
Write-Log "Starting storyteller-web on $DevBackendUrl (Ctrl-C to stop)..."
Invoke-Native "cargo run -p storyteller-web --bin storyteller-web"
} finally {
Pop-Location
}
+166
View File
@@ -0,0 +1,166 @@
# Create (or verify) the local demo user via the running backend, then verify
# that login works. Idempotent: an already-existing user is a success.
#
# Requires the backend to be up (run_backend_dev.ps1) - waits up to
# -WaitSeconds (default 60) for it.
#
# The account is created through POST /v1/create_account rather than raw SQL so
# it exercises the real signup path (bcrypt hash, session, firehose row) and
# stays correct if the users schema evolves.
#
# Override credentials via DEMO_USERNAME / DEMO_PASSWORD / DEMO_EMAIL env vars.
# Username rules: 3-16 chars of [A-Za-z0-9_-], not on the reserved list
# (includes/binary_includes/usernames/reserved_usernames.txt); password >= 6.
param(
[int]$WaitSeconds = 60
)
. (Join-Path $PSScriptRoot "common.ps1")
Write-Log "Waiting for backend at $DevBackendUrl (up to ${WaitSeconds}s)..."
Wait-ForCondition { Test-Backend } $WaitSeconds "backend /_status to respond (start it with run_backend_dev.ps1)" | Out-Null
Write-Log "Backend is up."
# curl.exe + ConvertFrom-Json: Invoke-RestMethod in PS 5.1 throws on non-2xx
# and hides the response body, which we need for the already-exists case.
# Invoke-Native keeps curl's stderr from becoming a terminating PS error.
function Invoke-JsonPost([string]$Url, [hashtable]$Body) {
$json = $Body | ConvertTo-Json -Compress
$tmp = Join-Path $env:TEMP "bootstrap_post_body.json"
Set-Content -Path $tmp -Value $json -Encoding ascii
$response = Invoke-Native "curl.exe -sS -X POST `"$Url`" -H `"Content-Type: application/json`" --data `"@$tmp`""
$curlExit = $LASTEXITCODE
Remove-Item $tmp -Force -Confirm:$false -ErrorAction SilentlyContinue
$joined = ($response -join "`n")
if ($curlExit -ne 0) { Die "Request to $Url failed (curl exit $curlExit): $joined" }
try {
return $joined | ConvertFrom-Json
} catch {
Die "Non-JSON response from ${Url}: $joined"
}
}
Write-Log "Creating account '$DemoUsername'..."
$create = Invoke-JsonPost "$DevBackendUrl/v1/create_account" @{
username = $DemoUsername
password = $DemoPassword
password_confirmation = $DemoPassword
email_address = $DemoEmail
}
if ($create.success -eq $true) {
Write-Log "Account created."
} else {
$errorCode = "$($create.error_type)$($create.error_code)$($create.error_code_str)"
if ($errorCode -match "(?i)username_?taken|email_?taken") {
Write-Log "Account already exists ($errorCode) - continuing."
} else {
Die "Account creation failed: $($create | ConvertTo-Json -Compress)"
}
}
Write-Log "Verifying login..."
$login = Invoke-JsonPost "$DevBackendUrl/v1/login" @{
username_or_email = $DemoUsername
password = $DemoPassword
}
if ($login.success -ne $true) { Die "Login verification failed: $($login | ConvertTo-Json -Compress)" }
if (-not $login.signed_session) { Die "Login succeeded but returned no signed_session." }
# --- Credits wallet ---------------------------------------------------------
# Generation submits deduct from the user's artcraft wallet and the webapp
# gates the generate button on the balance, so a user without credits cannot
# exercise any generation flow. Signup does not create a wallet; seed one.
# Raw SQL (not an endpoint): the only credit-granting paths are Stripe
# webhooks and moderator tooling, neither of which works locally.
Write-Log "Seeding credits wallet ($DemoCredits banked credits)..."
$userToken = @(Invoke-MySqlApp -Sql "SELECT token FROM users WHERE username = '$DemoUsername' LIMIT 1") |
Where-Object { "$_" -match '^user_' } | Select-Object -First 1
if (-not $userToken) { Die "Could not find users row for '$DemoUsername'." }
$walletRow = @(Invoke-MySqlApp -Sql "SELECT token, banked_credits FROM wallets WHERE wallet_namespace = 'artcraft' AND owner_user_token = '$userToken' LIMIT 1") |
Where-Object { "$_" -match '^wallet_' } | Select-Object -First 1
if (-not $walletRow) {
$walletToken = New-DevToken "wallet_" 32
$createLedgerToken = New-DevToken "wle_" 32
$creditLedgerToken = New-DevToken "wle_" 32
Invoke-MySqlApp -Sql (
"INSERT INTO wallets SET token='$walletToken', wallet_namespace='artcraft', owner_user_token='$userToken', banked_credits=$DemoCredits, monthly_credits=0; " +
"INSERT INTO wallet_ledger_entries SET token='$createLedgerToken', wallet_token='$walletToken', entry_type='create', credits_delta=0, banked_credits_before=0, banked_credits_after=0, monthly_credits_before=0, monthly_credits_after=0; " +
"INSERT INTO wallet_ledger_entries SET token='$creditLedgerToken', wallet_token='$walletToken', entry_type='credit_banked', maybe_entity_ref='dev_bootstrap_seed', credits_delta=$DemoCredits, banked_credits_before=0, banked_credits_after=$DemoCredits, monthly_credits_before=0, monthly_credits_after=0;"
) | Out-Null
if ($LASTEXITCODE -ne 0) { Die "Failed to insert the demo wallet." }
Write-Log "Wallet created with $DemoCredits banked credits."
} else {
$parts = "$walletRow" -split "`t"
$walletToken = $parts[0]
$balance = [uint64]$parts[1]
if ($balance -lt $DemoCredits) {
$creditLedgerToken = New-DevToken "wle_" 32
$delta = [uint64]$DemoCredits - $balance
Invoke-MySqlApp -Sql (
"UPDATE wallets SET banked_credits=$DemoCredits, version=version+1 WHERE token='$walletToken' LIMIT 1; " +
"INSERT INTO wallet_ledger_entries SET token='$creditLedgerToken', wallet_token='$walletToken', entry_type='credit_banked', maybe_entity_ref='dev_bootstrap_topup', credits_delta=$delta, banked_credits_before=$balance, banked_credits_after=$DemoCredits, monthly_credits_before=0, monthly_credits_after=0;"
) | Out-Null
if ($LASTEXITCODE -ne 0) { Die "Failed to top up the demo wallet." }
Write-Log "Wallet topped up: $balance -> $DemoCredits banked credits."
} else {
Write-Log "Wallet already has $balance banked credits - leaving as-is."
}
}
# --- Gallery media ----------------------------------------------------------
# Seed a few images so the library/gallery isn't empty on a fresh stack. The
# files live under .devstack\media (served by the backend at /media when
# LOCAL_MEDIA_ROOT + CDN_BASE_URL are set - bootstrap writes both into the
# generated secrets env). Rows mirror what a real image upload would write.
Write-Log "Seeding demo gallery media..."
$seedImages = @(
@{ Source = "test_data\image\juno.jpg"; Title = "Dev seed - Juno 1" },
@{ Source = "test_data\image\mochi.jpg"; Title = "Dev seed - Mochi 1" },
@{ Source = "test_data\image\juno.jpg"; Title = "Dev seed - Juno 2" },
@{ Source = "test_data\image\mochi.jpg"; Title = "Dev seed - Mochi 2" },
@{ Source = "test_data\image\juno.jpg"; Title = "Dev seed - Juno 3" },
@{ Source = "test_data\image\mochi.jpg"; Title = "Dev seed - Mochi 3" }
)
$existingMedia = Get-MySqlCount "SELECT COUNT(*) FROM media_files WHERE maybe_creator_user_token='$userToken' AND maybe_title LIKE 'Dev seed %'"
if ($existingMedia -ge $seedImages.Count) {
Write-Log "Gallery already has $existingMedia seeded media rows - leaving as-is."
} else {
foreach ($img in $seedImages) {
$source = Join-Path $RootDir $img.Source
if (-not (Test-Path $source)) { Die "Seed image missing: $source" }
# Mirror MediaFileBucketPath: /media/{h0..h4}/{hash}/image_{hash}.jpg
$hash = New-DevToken "" 32
$fanout = ($hash.Substring(0, 5).ToCharArray() -join "\")
$objectDir = Join-Path $DevMediaRoot "media\$fanout\$hash"
New-Item -ItemType Directory -Force $objectDir | Out-Null
Copy-Item $source (Join-Path $objectDir "image_$hash.jpg") -Force
$sha = (Get-FileHash -Algorithm SHA256 $source).Hash.ToLower()
$size = (Get-Item $source).Length
$mediaToken = New-DevToken "m_" 32
Invoke-MySqlApp -Sql (
"INSERT INTO media_files SET token='$mediaToken', maybe_creator_user_token='$userToken', " +
"origin_category='upload', origin_product_category='unknown', media_class='image', media_type='jpg', " +
"maybe_mime_type='image/jpeg', file_size_bytes=$size, checksum_sha2='$sha', " +
"public_bucket_directory_hash='$hash', maybe_public_bucket_prefix='image_', maybe_public_bucket_extension='.jpg', " +
"creator_ip_address='127.0.0.1', creator_set_visibility='public', is_user_upload=1, is_intermediate_system_file=0, " +
"maybe_title='$($img.Title)'"
) | Out-Null
if ($LASTEXITCODE -ne 0) { Die "Failed to insert seeded media row for $($img.Title)." }
}
Write-Log "Seeded $($seedImages.Count) gallery images under $DevMediaRoot."
}
Write-Log "Demo user is ready."
Write-Log " Username: $DemoUsername"
Write-Log " Password: $DemoPassword"
Write-Log "Log in at the webapp /login page (http://localhost:4201/login once 'nx dev artcraft-webapp' is running)."
@@ -0,0 +1,52 @@
# Start (or verify) the portable MySQL and Redis processes under .devstack/.
# Idempotent: already-running services are left alone. Safe to call from other
# scripts via dot-sourcing (defines Start-DevMySql / Start-DevRedis) or run
# directly.
param()
. (Join-Path $PSScriptRoot "common.ps1")
function Start-DevMySql {
if (Test-TcpPort 3306) {
Write-Log "MySQL already listening on 3306."
return
}
$mysqld = Join-Path $MySqlBaseDir "bin\mysqld.exe"
if (-not (Test-Path $mysqld)) {
Die "Portable MySQL not installed at $MySqlBaseDir. Run bootstrap_dev_stack.ps1 first."
}
if (-not (Test-Path (Join-Path $MySqlDataDir "mysql"))) {
Die "MySQL data directory not initialized at $MySqlDataDir. Run bootstrap_dev_stack.ps1 first."
}
Write-Log "Starting mysqld (portable, data in .devstack\mysql-data)..."
Start-DevProcess -Name "mysqld" -Exe $mysqld -Arguments @("--defaults-file=`"$MySqlIniPath`"") | Out-Null
Wait-ForCondition { Test-TcpPort 3306 } 60 "MySQL to listen on 3306 (see $LogsDir\mysqld.err.log and the .err file in $MySqlDataDir)" | Out-Null
Write-Log "MySQL is up."
}
function Start-DevRedis {
if (Test-TcpPort 6379) {
Write-Log "Redis already listening on 6379."
return
}
$redis = Join-Path $RedisDir "redis-server.exe"
if (-not (Test-Path $redis)) {
Die "Portable Redis not installed at $RedisDir. Run bootstrap_dev_stack.ps1 first."
}
Write-Log "Starting redis-server (portable)..."
# Keep the working dir (dump.rdb etc.) inside .devstack. NB: Start-Process
# rejects empty-string ArgumentList elements, so pass `--save ""` as one
# token to disable snapshotting.
Start-DevProcess -Name "redis" -Exe $redis -Arguments @(
"--port", "6379",
"--save", "`"`"",
"--dir", "`"$RedisDir`""
) | Out-Null
Wait-ForCondition { Test-TcpPort 6379 } 30 "Redis to listen on 6379 (see $LogsDir\redis.err.log)" | Out-Null
Write-Log "Redis is up."
}
if ($MyInvocation.InvocationName -ne '.') {
Start-DevMySql
Start-DevRedis
}
@@ -0,0 +1,41 @@
# Stop the portable MySQL and Redis processes started by start_dev_services.ps1.
param()
. (Join-Path $PSScriptRoot "common.ps1")
# Graceful MySQL shutdown (flushes InnoDB) - root has no password in the
# portable install (initialized with --initialize-insecure).
if (Test-TcpPort 3306) {
$mysqladmin = Get-MySqlTool "mysqladmin"
if ($mysqladmin) {
Write-Log "Shutting down MySQL..."
try { & $mysqladmin -u root -h 127.0.0.1 shutdown 2>$null } catch {}
$deadline = (Get-Date).AddSeconds(30)
while ((Test-TcpPort 3306) -and ((Get-Date) -lt $deadline)) { Start-Sleep -Seconds 1 }
}
}
if (Test-TcpPort 6379) {
$cli = Get-RedisTool "redis-cli"
if ($cli) {
Write-Log "Shutting down Redis..."
try { & $cli -h 127.0.0.1 shutdown nosave 2>$null } catch {}
}
}
# Last resort: kill anything left via recorded PIDs.
if (Test-Path $PidsDir) {
foreach ($pidFile in Get-ChildItem $PidsDir -Filter "*.pid" -ErrorAction SilentlyContinue) {
$procId = Get-Content $pidFile.FullName -ErrorAction SilentlyContinue
if ($procId) {
$proc = Get-Process -Id $procId -ErrorAction SilentlyContinue
if ($proc -and -not $proc.HasExited) {
Write-Log "Force-stopping $($pidFile.BaseName) (pid $procId)..."
try { Stop-Process -Id $procId -Force -Confirm:$false -ErrorAction Stop } catch {}
}
}
Remove-Item $pidFile.FullName -Force -Confirm:$false -ErrorAction SilentlyContinue
}
}
Write-Log "Dev services stopped."