mirror of
https://github.com/storytold/artcraft.git
synced 2026-11-01 15:25:55 -05:00
Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
5074052094 |
@@ -0,0 +1,47 @@
|
||||
# API Conventions
|
||||
|
||||
## HTTP Handlers (Actix-Web)
|
||||
|
||||
Handlers in `storyteller_web` follow this structure:
|
||||
|
||||
1. **Request/Response types** with `#[derive(Deserialize/Serialize, ToSchema)]`
|
||||
2. **Error enum** implementing `ResponseError`, `Display`, `Serialize`, `ToSchema`
|
||||
3. **Handler function** annotated with `#[utoipa::path(...)]` for OpenAPI docs
|
||||
4. Handler returns `Result<Json<Response>, ErrorType>`. Only use `Result<HttpResponse, ErrorType>` when you need to set headers/cookies.
|
||||
|
||||
### Error Types
|
||||
|
||||
- Use `CommonWebError` for simple handlers that only need standard HTTP error codes
|
||||
- Use a custom error enum when you need endpoint-specific error variants (e.g. `LoginErrorType::InvalidCredentials`)
|
||||
- Custom error enums MUST implement `ResponseError` (actix), `Display`, and `Serialize`
|
||||
- When adding `From<RequireUserSessionError>` or similar conversions, implement them on `CommonWebError` in its own module (orphan rule)
|
||||
- Error response bodies include: `success: false`, `error_code`, `error_code_str`
|
||||
|
||||
### Session/Auth
|
||||
|
||||
- `require_user_session_using_connection()` reuses an open DB connection (preferred)
|
||||
- `require_user_session()` acquires its own connection (deprecated)
|
||||
- `require_moderator()` for admin-only endpoints
|
||||
|
||||
### API Type Definitions
|
||||
|
||||
- Shared request/response types go in `artcraft_api_defs` crate (under `crates/api_clients/artcraft/artcraft_api_defs/`)
|
||||
- Types that need `ResponseError` impl must stay in `storyteller_web` (orphan rule)
|
||||
- New types must be added to `api_doc.rs` schemas (in storyteller-web) for Swagger generation
|
||||
|
||||
## Enums (Database-backed)
|
||||
|
||||
- Located in `crates/schema/public/enums/src/by_table/{table_name}/`
|
||||
- Require manual `to_str()` and `from_str()` implementations
|
||||
- Use `impl_enum_display_and_debug_using_to_str!` and `impl_mysql_enum_coders!` macros
|
||||
- Add comprehensive tests: serialization, round-trip, variant count
|
||||
- YOU CAN ADD NEW VALUES but DO NOT CHANGE EXISTING VALUES without a migration strategy
|
||||
- Note that adding new values can break older versions of the Tauri desktop app that read
|
||||
responses over Rust - make sure it consumes variants with an "Unknown(String)" variant
|
||||
catch-all to future proof the APIs.
|
||||
|
||||
## Tokens (Primary Keys)
|
||||
|
||||
- Located in `crates/schema/public/tokens/src/tokens/`
|
||||
- Opaque `String` wrappers with Stripe-like prefixes (e.g. `user_`, `mf_`)
|
||||
- Use `impl_string_token!`, `impl_mysql_token_from_row!`, `impl_crockford_generator!` macros
|
||||
@@ -0,0 +1,32 @@
|
||||
# Code Style
|
||||
|
||||
## Rust
|
||||
|
||||
- Two spaces for indentation (not four)
|
||||
- No minimum supported Rust version; use latest stable features freely
|
||||
- `#[macro_use] extern crate serde_derive;` is used in binary crates; library crates use `use serde_derive::{Serialize, Deserialize};`
|
||||
- Prefer `log` crate macros (`info!`, `warn!`, `error!`) over `println!`
|
||||
- Use `anyhow::Result` / `AnyhowResult` for fallible functions in application code
|
||||
- Enum Display/Debug: use `impl_enum_display_and_debug_using_to_str!` macro, not `derive_more::Display` (CLion doesn't understand it)
|
||||
- Fields that are optional use the `maybe_` prefix: `maybe_creator_user_token`, `maybe_prompt_token`
|
||||
- When implementing `fmt::Display` for error types, use the pattern: `write!(f, "{:?}", self)`
|
||||
|
||||
## Naming
|
||||
|
||||
- Handler functions: `{verb}_{noun}_handler` (e.g. `get_health_check_handler`, `login_handler`)
|
||||
- Error enums: `{Feature}Error` (e.g. `FalWebhookError`, `HealthCheckError`)
|
||||
- Request/response structs: `{Action}Request`, `{Action}Response`, `{Action}SuccessResponse`
|
||||
- Path params: `{Action}PathInfo`
|
||||
- Builder structs: `{Thing}Builder` with `set_` prefixed setters and a `build()` method
|
||||
|
||||
## Imports
|
||||
|
||||
- Group imports: std, external crates, internal workspace crates, `crate::` imports
|
||||
- Use fully qualified paths for one-off references; `use` for repeated references
|
||||
- Prefer specific imports over wildcards, except in `api_doc.rs` and test modules
|
||||
|
||||
## TypeScript / Frontend
|
||||
|
||||
- TypeScript with React, Vite, Zustand, Three.js
|
||||
- Nx monorepo under `crates/frontend/`
|
||||
- Two spaces for indentation
|
||||
@@ -0,0 +1,27 @@
|
||||
# Testing
|
||||
|
||||
## Running Tests
|
||||
|
||||
- Most crates: `cargo test -p {crate_name}`
|
||||
- Crates using SQLx offline mode: `SQLX_OFFLINE=true cargo test -p {crate_name}`
|
||||
- The `storyteller-web` and `mysql_queries` crates require `SQLX_OFFLINE=true` for compilation
|
||||
- Use `cargo check -p {crate_name}` for fast compilation checks without running tests
|
||||
|
||||
## Test Patterns
|
||||
|
||||
- Tests live in `#[cfg(test)] mod tests { ... }` at the bottom of each file
|
||||
- Group related tests in sub-modules: `mod explicit_checks`, `mod mechanical_checks`
|
||||
- Enum tests should cover: serialization, deserialization, `to_str`/`from_str` round-trip, variant count
|
||||
- Password/crypto tests: be mindful of computation cost, don't add too many bcrypt test cases
|
||||
|
||||
## What to Test
|
||||
|
||||
- New error types and their mappings
|
||||
- Enum variant serialization stability (these are stored in the database)
|
||||
- Hash/deduplication functions with known inputs
|
||||
- Edge cases in parsers (empty input, missing fields, malformed data)
|
||||
|
||||
## What Not to Test
|
||||
|
||||
- Don't add integration tests that require a running database unless explicitly asked
|
||||
- Don't add tests for simple struct construction or trivial getters
|
||||
@@ -0,0 +1,22 @@
|
||||
# artcraft_api_defs
|
||||
|
||||
Shared HTTP API type definitions used by both the server (`storyteller_web`) and clients.
|
||||
|
||||
## Structure
|
||||
|
||||
Organized by feature: `users/`, `media_file/`, `jobs/`, `moderation/`, etc.
|
||||
|
||||
## Conventions
|
||||
|
||||
- Request types: `#[derive(Deserialize, ToSchema)]`
|
||||
- Response types: `#[derive(Serialize, ToSchema)]`
|
||||
- Path params: `#[derive(Deserialize, ToSchema)]` with `PathInfo` suffix
|
||||
- Query params: `#[derive(Deserialize, IntoParams)]`
|
||||
|
||||
## Orphan Rule
|
||||
|
||||
Types that need `ResponseError` impl (actix-web) CANNOT live here because of Rust's orphan rule. Keep error response types in `storyteller_web` and only put data types here.
|
||||
|
||||
## Dependencies
|
||||
|
||||
Only `serde`, `serde_derive`, `utoipa`, `chrono`, `url`, and internal schema crates (`enums`, `tokens`). No actix-web dependency.
|
||||
@@ -0,0 +1,40 @@
|
||||
# Pager Crate
|
||||
|
||||
Alerting/paging system that sends notifications via Rootly.
|
||||
|
||||
## Key Types
|
||||
|
||||
- `Pager` - Main client, cloneable. Has `enqueue_page()` and `shutdown_worker()`
|
||||
- `PagerBuilder` - Constructs a Pager with optional background worker
|
||||
- `NotificationDetails` - Alert payload with title, description, urgency, HTTP context, entity tokens
|
||||
- `NotificationDetailsBuilder` - Fluent builder for constructing notifications
|
||||
- `NotificationUrgency` - High, Medium, Low
|
||||
|
||||
## Building Notifications
|
||||
|
||||
```rust
|
||||
let notification = NotificationDetailsBuilder::from_title("Something happened".to_string())
|
||||
.set_description(Some("Details...".to_string()))
|
||||
.set_urgency(Some(NotificationUrgency::High))
|
||||
.set_http_method(Some("POST".to_string()))
|
||||
.set_http_path(Some("/v1/endpoint".to_string()))
|
||||
.set_inference_job_token(Some(job_token.to_string()))
|
||||
.build();
|
||||
```
|
||||
|
||||
For errors, use `from_error()` which auto-populates title and description from the error:
|
||||
|
||||
```rust
|
||||
NotificationDetailsBuilder::from_error(&err)
|
||||
.set_title("Custom title".to_string())
|
||||
.set_urgency(Some(NotificationUrgency::Medium))
|
||||
.build()
|
||||
```
|
||||
|
||||
## Deduplication
|
||||
|
||||
`NotificationDetails::to_deduplication_key()` generates a SHA-256 hash for dedup. Same title + HTTP context + hour = same key.
|
||||
|
||||
## Worker Thread
|
||||
|
||||
The pager worker uses `Condvar::wait()` (blocking). It MUST run on a dedicated OS thread, not a tokio task.
|
||||
@@ -0,0 +1,26 @@
|
||||
# Enums Crate
|
||||
|
||||
Database-backed enums stored as VARCHAR fields in MySQL.
|
||||
|
||||
## Adding a New Enum
|
||||
|
||||
1. Create file: `src/by_table/{table_name}/{field_name}.rs`
|
||||
2. Define the enum with required derives:
|
||||
```rust
|
||||
#[cfg_attr(test, derive(EnumIter, EnumCount))]
|
||||
#[derive(Clone, Copy, Eq, PartialEq, Hash, Ord, PartialOrd, Deserialize, Serialize, ToSchema)]
|
||||
pub enum MyEnum { ... }
|
||||
```
|
||||
3. Implement `to_str()` and `from_str()` manually
|
||||
4. Add macros: `impl_enum_display_and_debug_using_to_str!(MyEnum);` and `impl_mysql_enum_coders!(MyEnum);`
|
||||
5. Add to parent `mod.rs`
|
||||
6. Write tests: serialization, round-trip, `all_variants()` count
|
||||
|
||||
## Rules
|
||||
|
||||
- **NEVER change existing variant string values** - they are stored in the database
|
||||
- New variants CAN be added freely - but be very careful about the Tauri API callers
|
||||
ingesting new values. Make sure to pair the client with variants with an "Unknown(String)"
|
||||
catch-all / future-proof bucket.
|
||||
- All string values must be snake_case
|
||||
- Use `#[serde(rename = "snake_case_value")]` on each variant
|
||||
@@ -0,0 +1,22 @@
|
||||
# Tokens Crate
|
||||
|
||||
Strongly-typed primary key identifiers with Stripe-like prefixes (e.g. `user_abc123`, `prompt_xyz789`, etc.)
|
||||
|
||||
## Adding a New Token
|
||||
|
||||
1. Create file in `src/tokens/{entity_name}.rs`
|
||||
2. Define as a newtype wrapper:
|
||||
```rust
|
||||
#[derive(Clone, PartialEq, Eq, PartialOrd, Ord, Hash, Debug, Serialize, Deserialize, ToSchema)]
|
||||
#[cfg_attr(feature = "database", derive(sqlx::Type))]
|
||||
#[cfg_attr(feature = "database", sqlx(transparent))]
|
||||
pub struct MyToken(pub String);
|
||||
```
|
||||
3. Apply macros:
|
||||
```rust
|
||||
impl_string_token!(MyToken);
|
||||
impl_mysql_token_from_row!(MyToken);
|
||||
impl_crockford_generator!(MyToken, 32usize, TokenPrefix::MyEntity, CrockfordLower);
|
||||
```
|
||||
4. Add the prefix to `TokenPrefix` enum in `src/token_prefix.rs`
|
||||
5. Add to parent `mod.rs`
|
||||
@@ -0,0 +1,42 @@
|
||||
# Background Jobs
|
||||
|
||||
Background job crates that run as standalone processes alongside storyteller-web.
|
||||
|
||||
## Common Pattern
|
||||
|
||||
All jobs follow the same structure:
|
||||
|
||||
1. `main.rs` - Bootstrap, DB pool, bucket client, pager setup, thread spawning
|
||||
2. `job_dependencies.rs` - Shared state struct passed to the main loop
|
||||
3. `startup/build_pager.rs` - Pager initialization from env vars
|
||||
4. `http_server/` - Health check endpoint for k8s liveness probes
|
||||
5. `job/` or `jobs/` - Main processing logic
|
||||
|
||||
## Threading Model
|
||||
|
||||
- **Main thread** (`#[tokio::main]`): runs the job's main loop
|
||||
- **HTTP server thread** (`std::thread::spawn`): dedicated OS thread with its own `actix_web::rt::System`
|
||||
- **Pager worker thread** (`std::thread::spawn`): dedicated OS thread (uses blocking `Condvar::wait`)
|
||||
- **Signal handler** (`tokio::spawn`): listens for SIGTERM/SIGINT to set `application_shutdown` flag
|
||||
|
||||
Important: Do NOT use `#[actix_web::main]` - it registers signal handlers that steal SIGINT. Use `#[tokio::main]` instead.
|
||||
|
||||
Important: Do NOT spawn the pager worker via `tokio::spawn` - its `Condvar::wait` blocks the runtime. Use a dedicated OS thread.
|
||||
|
||||
## Pager Integration
|
||||
|
||||
Use the `alert_pager_and_return_err()` helper pattern for error reporting:
|
||||
|
||||
```rust
|
||||
if let Err(err) = some_operation().await {
|
||||
error!("Descriptive message: {:?}", err);
|
||||
return alert_pager_and_return_err(&deps.pager, "Alert title", err, optional_job_ref);
|
||||
}
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Shared across jobs (via `shared_env_var_config`):
|
||||
- `ENABLE_PAGING` - Master switch for pager alerts
|
||||
- `ROOTLY_API_KEY` - Rootly API key for sending alerts
|
||||
- `ROOTLY_NOTIFICATION_TARGET_TYPE` / `ROOTLY_NOTIFICATION_TARGET_ID`
|
||||
@@ -0,0 +1,44 @@
|
||||
# storyteller-web
|
||||
|
||||
The main HTTP API monolith. This is the backend for the ArtCraft application.
|
||||
|
||||
## Building
|
||||
|
||||
```
|
||||
cargo check -p storyteller-web
|
||||
```
|
||||
|
||||
There are ~370 pre-existing warnings. Check the last 8-10 lines of output for actual errors.
|
||||
|
||||
## Structure
|
||||
|
||||
- `src/http_server/endpoints/` - HTTP handlers organized by feature
|
||||
- `src/http_server/middleware/` - Actix middleware (error alerting, etc.)
|
||||
- `src/http_server/common_responses/` - Shared error types (`CommonWebError`)
|
||||
- `src/http_server/web_utils/` - Session checking, auth helpers
|
||||
- `src/state/` - Server state, feature flags
|
||||
- `src/startup/` - Initialization (pager, etc.)
|
||||
- `src/threads/` - Background threads (health checker, etc.)
|
||||
- `src/docs/` - OpenAPI/Swagger documentation (`api_doc.rs`)
|
||||
|
||||
## Handler Pattern
|
||||
|
||||
Handlers return `Result<Json<Response>, ErrorType>`. The `Result<HttpResponse, ErrorType>` pattern is
|
||||
deprecated or only used when setting other HTTP headers, like cookies.
|
||||
|
||||
Prefer `CommonWebError` for new handlers unless you need custom error variants.
|
||||
|
||||
When a handler uses `require_user_session`, prefer `require_user_session_using_connection`
|
||||
which reuses an existing DB connection.
|
||||
|
||||
## api_doc.rs
|
||||
|
||||
When adding new request/response/error types, add them to `src/docs/api_doc.rs` in the
|
||||
schemas section (alphabetically sorted). Types from `artcraft_api_defs` need explicit imports.
|
||||
|
||||
## Pager Integration
|
||||
|
||||
- The `Pager` is available on `ServerState` as `server_state.pager`
|
||||
- Health check handler sends alerts when unhealthy
|
||||
- Error alerting middleware sends alerts on HTTP 500s (when enabled)
|
||||
- Use `NotificationDetailsBuilder` for constructing alerts
|
||||
Reference in New Issue
Block a user