mirror of
https://github.com/storytold/artcraft.git
synced 2026-10-20 17:02:53 -04:00
infra/dev-env-bootstrap
5
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Tanish Manku
|
f5b80ea982 |
Dev infra: local filesystem uploads + fully-local pagescene media
Closes the two remaining fully-local-stack seams found after the last round. Uploads (previously every upload endpoint returned 500 with dummy S3 creds - the bucket put happens before the DB insert and rust-s3 is fail-on-err): - LegacyBucketClient gains an internal backend enum (S3 unchanged | LocalDir) and a create_local(root, bucket_name, root_prefix) constructor. All upload/ download methods route LocalDir to the filesystem; bucket_name() reports the configured name. Objects map disk = root + rooted object path via the new pub local_object_disk_path (rejects ".." and backslash segments - PMX zips feed user-controlled names), unit-tested. The fake-generation resolver now uses the same function instead of its private copy so the layouts can't drift. - build_dependencies: when SERVER_ENVIRONMENT=Development and LOCAL_MEDIA_ROOT is set, all three bucket clients are constructed in local mode (private/gc have zero callers in storyteller-web, so this is purely uniform); the server_environment read moved above client construction. Uploads land under .devstack/media and are immediately viewable through the existing /media mount + CDN_BASE_URL - no S3/R2/MinIO. Frontend (pagescene was the only surface actually broken against a local backend; moodboard/library/create-* already consume media_links correctly): - The vite /__cdn dev proxy assumed https upstreams, so localhost media (http) died on a TLS handshake with a 502 - breaking every pagescene GLB/splat/animation fetch. fetchAsset no longer proxies localhost at all (the dev backend sends CORS for any localhost origin), and the proxy itself now picks http for localhost hosts as a backstop. - GetCdnOrigin() returns the API host when it is localhost (the local backend serves media itself; CDN origin == API origin), production CDN otherwise. The pagescene adapter's getCdnUrl skips the Cloudflare /cdn-cgi/image resize segment on localhost - the local backend has no resizer, and dev serves originals anyway. - vite /v1 fallback proxy now targets http://localhost:12345 instead of production, so nothing can silently fall back to prod traffic in dev; fixed the stale main.tsx comment claiming setDevelopment() is disabled. Docs: seams table updated (uploads + vite proxy + GetCdnOrigin closed; API_TARGETS/SegmentationApi noted as dead code; the earlier moodboard claim was wrong - it never had a hardcoded host), LOCAL_MEDIA_ROOT now documents the storage mode, verification checklist gains the upload check. Verified E2E on this machine: POST /v1/media_files/upload/image (multipart, real endpoint) returns success, the object lands under .devstack/media at the canonical fan-out path, lists in the library with media_class=image, and serves 200 image/jpeg from http://localhost:12345/media/...; cloud_storage unit tests pass; storyteller-web + webapp dev server boot clean with "Local filesystem storage mode is ON". |
||
|
Tanish Manku
|
335015d9e2 |
Dev infra: fully-local generation and gallery for the dev stack
Closes the two gaps found smoke-testing the local stack: an empty gallery (zero media_files rows, media URLs hardcoded to the remote r2.dev bucket) and generation flows that could not complete (omni_gen calls providers synchronously in the handler, so dummy keys meant a 500 after billing; the webapp blocked submits earlier anyway because signup grants no wallet). Backend (all development-only): - CDN_BASE_URL env override in cdn_link.rs (host-only, validated loudly; precedence over the compiled-in hosts for every domain/env) and a dev-gated actix-files mount serving LOCAL_MEDIA_ROOT/media at /media, mirroring the public bucket layout - media URLs in API responses resolve against the backend itself - DEV_FAKE_GENERATION flag on ServerState (easyenv bool, forced off outside Development): the omni_gen image/video pipelines skip the external provider call and return a synthetic Fal enqueue (external id fake_<uuid>), so billing, prompt insert, and the pending generic_inference_jobs row all stay real - dev_fake_generation_resolver_thread: polls pending fake_ jobs (new runtime-query list_pending_fake_generation_jobs in mysql_queries) and after DEV_FAKE_GENERATION_RESOLVE_SECS (default 6s) marks them complete_success with a media_files row backed by a test_data asset copied under LOCAL_MEDIA_ROOT (mp4 results get -thumb.jpg/-thumb.gif siblings since the thumbnail job never runs locally), or complete_failure via the existing simulate_artcraft_failure prompt triggers (category word supported, e.g. "simulate_artcraft_failure face_not_detected") Bootstrap/seeds (PS + bash kept in lockstep): - seed_demo_user seeds an artcraft wallet (DEMO_CREDITS, default 100k banked + ledger rows; idempotent top-up) - signup creates no wallet and the webapp gates generate on balance, which is why submits went nowhere - seed_demo_user also seeds 6 gallery images (test_data jpgs) with objects under .devstack/media in the exact bucket fan-out layout - fixed the already-exists detection in seed_demo_user: create_account returns error_type, not error_code, so re-runs died on UsernameTaken - generated secrets env gains DEV_FAKE_GENERATION/CDN_BASE_URL/ LOCAL_MEDIA_ROOT (appended to existing files on bootstrap re-runs); stale "providers are only contacted by workers" comments corrected - doctor checks wallet balance, media rows, and the new secrets keys Docs: dev_setup_local_stack.md documents the new flags, corrects the sync-provider-call and upload-500 claims, updates the seams table and verification checklist; storyteller_web CLAUDE.md video model examples fixed (seedance_2p0_fast_g -> seedance_2p0_fast). Verified E2E on this machine: gallery lists 8 items on localhost URLs; image and video submits go pending ~6s then complete with locally-served results (mp4 + gif preview 200); failure trigger yields complete_failure with category face_not_detected; credits genuinely deducted (100000 -> 99932); doctor 0 failures / 0 warnings. |
||
|
Tanish Manku
|
f23a6df80b |
Dev infra: harden Windows bootstrap after two adversarial debug passes
Pass 1 (correctness sweep): - All native invocations (mysql, mysqld init, winget, curl downloads, cargo run, seed curl) now go through Invoke-Native, so a caller capturing the scripts' output can never turn ordinary stderr (warnings, progress) into a terminating NativeCommandError under Stop preference - Provisioning DDL feeds through a temp file + `source` and prints the mysql output when it fails (with the MYSQL_ROOT_PASSWORD hint) - Get-MySqlCount distinguishes "table empty" (0) from "query failed" (-1); bootstrap and doctor no longer misread a failed COUNT as unseeded - diesel_cli that is present but broken (moved libmysql.dll) is detected via an exit-code probe and reinstalled with --force - Shared Add-BuildToolsToSession maps winget install dirs + LIBCLANG_PATH in bootstrap, doctor, and run_backend_dev (cargo run may recompile); the tools check maps PATH before probing so re-runs skip winget entirely - my.ini values quoted (clone paths with spaces); service-start timeouts point at the .devstack log files Pass 2 (live retest matrix, all green on this machine): - Full re-run with output captured in-PowerShell: exit 0, no NativeCommandError - Wiped-state rebuild (mysql-data/my.ini/redis/secrets deleted): re-init, re-provision, 281 migrations replayed, seeds + secrets regenerated, backend booted, demo user recreated via /v1/create_account, doctor 0 fail / 0 warn - Failure paths: seed without backend and doctor without services both die cleanly with actionable messages and exit 1; stop_dev_services frees both ports gracefully |
||
|
Tanish Manku
|
345cbe766a |
Dev infra: Windows-first bootstrap - portable MySQL/Redis stack, no WSL required
Adds script/bootstrap/windows/*.ps1 (PowerShell 5.1-compatible) as the primary bootstrap path, validated end-to-end on a real Windows 10 machine: - Portable stack under gitignored .devstack/: MySQL 8.4.11 (official cdn.mysql.com zip, --initialize-insecure, bound to 127.0.0.1) and Redis 5.0.14.1 (tporadowski build, REDIS_ZIP_URL-overridable) run as plain user processes - no admin, no services, no WSL, no Docker - bootstrap_dev_stack.ps1: winget-installs missing build tools (Strawberry Perl, NASM, LLVM; wreq's BoringSSL + aws-lc-sys need them), compiles diesel_cli against the portable MySQL's own client lib (MYSQLCLIENT_LIB_DIR), runs all 281 migrations, seeds roles/badges idempotently, generates the dummy secrets env (plus TEMP_DIR for Windows), builds storyteller-web, installs frontend deps - start/stop_dev_services.ps1, run_backend_dev.ps1, seed_demo_user.ps1, dev_stack_doctor.ps1 mirror the bash variants - Invoke-Native (cmd /c ... 2>&1) wraps long-running native calls so PS 5.1 cannot turn harmless stderr warnings into terminating NativeCommandErrors when output is captured; my.ini uses forward slashes (MySQL escape rules) Verified on this machine: 281/281 migrations applied, backend built (3m05s) and booted on :12345 with dummy secrets, POST /v1/create_account + /v1/login round-trip succeeded, webapp dev server served on :4201, doctor reports 0 failures. The bash scripts remain the Linux/WSL2/CI path. |
||
|
Tanish Manku
|
537409eb3e |
Dev infra: fresh-clone bootstrap for the fully-local dev stack (backend + webapp)
Adds script/bootstrap/ — idempotent Ubuntu 22.04+/WSL2/CI automation that takes a fresh clone to a running local stack with zero production traffic: - bootstrap_dev_stack.sh: apt packages, MySQL 8 + Redis up, storyteller DB/user, 281 diesel migrations, idempotent role/badge seeds, generated dummy development-secrets.env (19 required-at-boot vars no checked-in config supplies), SQLX_OFFLINE backend build, frontend npm install - run_backend_dev.sh / seed_demo_user.sh: run the API from the repo root and create+verify a demo login through the real POST /v1/create_account path - dev_stack_doctor.sh: PASS/WARN/FAIL diagnosis of every prerequisite _docs/dev_setup_local_stack.md records the requirements the scripts encode and corrects stale claims (server reads MYSQL_URL, not DATABASE_URL; repo-root .env is not loaded by the server; config precedence is first-value-wins; SQLX_OFFLINE is compile-time only). .gitattributes now forces LF for *.sh/*.env so Windows clones run under WSL2. |