docs: add architecture guide

2026-07-28 17:09:09 +00:00
1 parent 1cae60ab46
commit fbf0f81df1
@@ -0,0 +1,457 @@
 # MinerOps Architecture
 ## Purpose
 This document is the primary engineering architecture guide for MinerOps. It
 explains the codebase's current structure and the rules contributors must follow
 when adding or changing features.
 MinerOps is self-hosted software with operational data and workflows that must
 remain understandable as the system grows. Maintainability, testability, and
 clear boundaries are primary architectural goals. A feature is not complete
 merely because it works; its responsibilities and dependencies must also be in
 the correct layer.
 **Current implementation:** The repository is transitional. Most existing HTTP
 routes, persistence, and server-rendered assets are concentrated in
 `backend/app/main.py`. Identity has been separated into a FastAPI router.
 Inventory is the first domain with explicit models, exceptions, a service, a
 repository, and focused tests.
 **Engineering rule:** New domain code follows the layered architecture in this
 document. Existing integrated code is compatibility context, not a pattern to
 copy.
 ## Core Principles
 ### Separation of Concerns
 Transport, business rules, persistence, and presentation are different
 responsibilities. Keep them separate so each can change and be tested without
 requiring unrelated changes.
 ### Single Responsibility
 Each module, class, and function should have one clear reason to change. A
 router handles HTTP. A service handles domain behavior. A repository handles
 persistence. Combining these responsibilities makes ownership ambiguous and
 tests expensive.
 ### Thin HTTP Layer
 Routers accept and validate transport input, invoke services, and map results to
 HTTP responses. They do not implement business workflows and do not access the
 database directly.
 ### Business Logic Belongs in Services
 Services own normalization, domain validation, decisions, and orchestration.
 They must not depend on FastAPI request or response objects, route decorators,
 or HTTP status codes.
 ### Persistence Belongs in Repositories
 Repositories own queries, writes, database-facing filters, transaction
 mechanics, and persistence mapping. They contain no HTTP behavior, permission
 rules, or product decisions.
 ### Explicit Dependencies
 Dependencies are visible at construction or call boundaries. The current
 `InventoryService` receives its repository through its constructor, allowing a
 substitute repository in service tests and a PostgreSQL repository in
 integration tests.
 ### Incremental Evolution
 MinerOps has working behavior that must remain usable while responsibilities
 are separated. Improve boundaries in small, verified changes. Do not require a
 large rewrite before a domain can become clearer.
 ### Testability
 Architecture should make the correct test easy to write. Business rules should
 be testable without FastAPI or PostgreSQL. Repository behavior should be
 testable against a disposable PostgreSQL database. Cross-application behavior
 should be testable through the disposable development stack.
 ## High-Level Architecture
 The required dependency flow for domain-backed application features is:
 ```text
 Browser
  |
  v
 FastAPI Routers
  |
  v
 Services
  |
  v
 Repositories
  |
  v
 PostgreSQL
 ```
 - **Browser:** Presents the interface and sends authenticated HTTP requests. The
  current interface is served by FastAPI using HTML, CSS, and JavaScript.
 - **FastAPI routers:** Define the HTTP contract, enforce transport-level
  validation and authorization dependencies, call services, and serialize
  responses.
 - **Services:** Implement use cases and domain rules without knowing how the
  request arrived.
 - **Repositories:** Perform database operations and expose persistence behavior
  needed by services.
 - **PostgreSQL:** Stores authoritative operational records and enforces durable
  constraints.
 Redis and Nginx are current infrastructure components, but they do not change
 this domain dependency flow. Redis is configured as an application service and
 included in health checks. Nginx is the gateway in the development Compose
 stack.
 ## Repository Organization
 The repository is organized by runtime responsibility and supporting material:
 - **`backend/`:** Python application code, FastAPI composition, domain packages,
  Alembic migrations, backend dependencies, tests, and the backend container
  definition.
 - **`frontend/` (planned):** A future standalone frontend may live here after a
  frontend technology and migration plan are selected. No `frontend/` directory
  or standalone frontend application exists today.
 - **`docs/`:** Engineering and product documentation that applies across
  domains.
 - **`branding/`:** Product identity guidelines and source vector assets. It has
  no runtime application responsibility.
 - **`scripts/`:** Development setup, disposable test runners, schema checks,
  retention helpers, and operational utilities.
 Root Compose files define containerized services for the backend, PostgreSQL,
 and Redis. The development configuration also defines an Nginx gateway and runs
 Alembic before starting Uvicorn.
 ## Backend Layers
 ### Routers
 Routers are the HTTP boundary. A domain router should be an `APIRouter` included
 by the application entry point.
 Router responsibilities:
 - Declare paths, HTTP methods, parameters, and response status codes.
 - Parse path, query, header, and body values.
 - Apply authentication and permission dependencies.
 - Construct domain input models from transport input.
 - Call services.
 - Translate domain exceptions into consistent HTTP errors.
 - Serialize service results into documented response models.
 Routers must never contain:
 - SQL or database connection management.
 - Business rules, normalization policy, or lifecycle decisions.
 - Multi-step persistence workflows or transaction control.
 - Device protocol implementations.
 - Copies of logic already owned by a service or repository.
 A router may reject structurally invalid HTTP input. It must delegate questions
 such as whether an Asset ID is valid for a use case, whether a lifecycle change
 is allowed, or whether a record conflicts with existing state to the service.
 **Current implementation:** `backend/app/main.py` creates the `FastAPI`
 application and includes the router from `backend/app/identity.py`. Identity is
 the current example of router extraction. Most other routes remain registered
 directly on `app` and often contain SQL or business logic. Those routes predate
 the current layering rules and must not be used as templates for new domains.
 The Inventory package currently has no router and registers no HTTP endpoints.
 ### Services
 Services implement application use cases for a domain. A service coordinates
 repositories and applies rules that must remain consistent regardless of the
 transport calling it.
 Service responsibilities:
 - Normalize values according to domain semantics.
 - Validate business rules and allowed state transitions.
 - Detect conflicts meaningful to the domain.
 - Coordinate multiple repository operations as one use case.
 - Establish required transaction scope through repository abstractions.
 - Return domain models or domain-specific results.
 - Raise domain exceptions rather than HTTP exceptions.
 The current `InventoryService` demonstrates these responsibilities. It:
 - Normalizes Asset IDs and optional text.
 - Normalizes and deduplicates tags.
 - Validates `active`, `spare`, and `retired` lifecycle states.
 - Detects duplicate Asset IDs.
 - Defines never-seen timestamps for manually created miners.
 - Coordinates miner creation or update with tag associations.
 - Maps database-shaped records to Inventory summary and detail models.
 The service imports no FastAPI components. Its extension hooks are domain-level
 hooks, not route callbacks. A service must not inspect cookies, headers, request
 objects, or response objects.
 The current Inventory service catches a PostgreSQL uniqueness exception to
 translate a race into `DuplicateAssetID`. This is a known persistence coupling.
 New code should prevent database-specific exceptions from spreading further
 through the service layer, preferably by translating them at an explicit
 repository boundary.
 ### Repositories
 Repositories are the persistence boundary for a domain. They expose operations
 in terms useful to services while keeping SQL and database mechanics out of
 higher layers.
 Repository responsibilities:
 - Execute parameterized reads and writes.
 - Implement database-facing search, filtering, ordering, and pagination.
 - Manage connections, commits, rollbacks, and transaction-bound instances.
 - Maintain associations owned by the domain.
 - Map database values into persistence records.
 - Translate low-level persistence failures when an owned abstraction exists.
 Repositories contain database access only. They must not:
 - Import FastAPI or router modules.
 - Choose HTTP responses or status codes.
 - Check the current user's permissions.
 - Normalize values according to product policy.
 - Decide whether a lifecycle transition or operational action is allowed.
 `InventoryRepository` is the current reference implementation. It uses
 `psycopg`, accepts an optional connection factory, supports explicit transaction
 scope, and owns miner and tag CRUD, search, and associations. Its methods return
 database-shaped dictionaries; `InventoryService` maps those records to domain
 models.
 Dynamic SQL must be restricted to trusted identifiers or fixed fragments.
 User-provided values always use database parameters. Repositories must not accept
 raw SQL from routers or services.
 ### Database
 PostgreSQL is the authoritative persistent store for MinerOps. Application rules
 that must hold for every writer are also represented as database constraints
 where practical, including uniqueness, allowed values, and referential
 integrity.
 Alembic is the authority for new persistent schema changes. Migrations must
 state how existing data is handled and provide a downgrade when it can be done
 without ambiguity or data loss.
 **Current implementation:** Alembic migrations cover the foundational,
 scanning, audit, identity, role, and Inventory schemas. Some legacy code in
 `main.py` still creates or alters tables at runtime for compatibility. Do not add
 new runtime DDL. Move persistent schema changes through reviewed Alembic
 migrations.
 The repository also contains a SQLAlchemy engine and session factory, while
 most current operational persistence and the Inventory repository use direct
 `psycopg`; some existing agent paths use `asyncpg`. A domain must choose and
 encapsulate its persistence mechanism inside its repository rather than expose
 driver details to routers.
 ## Domains
 A domain package groups one operational capability behind an explicit public
 interface. Inventory is the current reference domain:
 ```text
 backend/app/inventory/
    __init__.py
    exceptions.py
    models.py
    repository.py
    service.py
 ```
 - `models.py` defines typed request, update, search, summary, detail, and tag
  contracts.
 - `exceptions.py` defines errors meaningful to Inventory callers.
 - `service.py` defines Inventory rules and use cases.
 - `repository.py` defines Inventory persistence.
 - `__init__.py` exposes the package's intended public surface.
 Inventory's internal asset identity is the miner UUID. Product-facing Asset IDs
 are normalized and checked independently of mutable network fields. The domain
 maps product terminology such as `asset_id` and `psu_serial` to existing database
 columns at its service and persistence boundary.
 Future domain packages should follow the same separation when it fits their use
 cases. Identity, Repairs, Monitoring, Operations, Scheduling, and Locations are
 examples of possible domain boundaries, not a fixed package list. A domain
 boundary must be based on ownership of rules and data, not on menu pages or
 route prefixes alone.
 A domain may call another domain only through an explicit service contract. It
 must not import another domain's repository to update tables it does not own.
 Shared read requirements should use a clearly owned query service or read model
 rather than duplicating cross-domain SQL.
 ## Dependency Rules
 Dependencies flow downward:
 ```text
 Routers
  |
  v
 Services
  |
  v
 Repositories
  |
  v
 Database
 ```
 Allowed dependencies:
 - The application entry point may include routers and install application-wide
  middleware.
 - Routers may depend on domain models, services, and exceptions.
 - Services may depend on their domain's models, exceptions, and explicit
  repository contracts.
 - Repositories may depend on configuration, database drivers, and
  persistence-specific utilities.
 - Tests may replace repositories or connection factories at defined dependency
  boundaries.
 Examples of allowed flow:
 - An Inventory router validates a UUID, calls `InventoryService.get_miner`, and
  translates `MinerNotFound` into an HTTP 404 response.
 - `InventoryService.create_miner` normalizes an Asset ID and coordinates miner
  and tag writes through an `InventoryRepository` transaction.
 - `InventoryRepository.search_miners` builds parameterized filters and executes
  them against PostgreSQL.
 Prohibited dependencies:
 - Repositories importing services, routers, or `app.main`.
 - Services importing FastAPI request or response classes.
 - Routers opening database connections or issuing SQL.
 - Domain code importing globals from `app.main`.
 - A lower layer choosing permissions or HTTP status codes.
 - A domain bypassing another domain's service to write its tables.
 - Circular imports between domains or layers.
 Examples of prohibited flow:
 - A route updating `miners` directly after checking request JSON.
 - A service returning `JSONResponse` or raising `HTTPException`.
 - A repository checking whether the current user is an administrator.
 - A Repairs service importing `InventoryRepository` to mutate miner ownership
  without an Inventory service contract.
 Dependency direction is an architectural constraint, not a naming convention.
 Putting SQL in a file named `service.py` does not make it service logic.
 ## Database Philosophy
 - **Alembic-managed schema:** New persistent tables, columns, constraints, and
  indexes are introduced through versioned migrations.
 - **Explicit migrations:** Every migration explains existing-data behavior,
  upgrade ordering, and downgrade behavior. Migrations do not silently merge,
  rename, or discard operational records.
 - **Stable identifiers:** Internal identity uses immutable identifiers rather
  than mutable IP addresses, MAC addresses, names, or locations.
 - **Repository ownership:** Each domain has one clear persistence boundary for
  its owned records. Other layers do not duplicate its SQL.
 - **Database enforcement:** PostgreSQL constraints protect invariants that must
  hold independently of application code.
 - **Transactional consistency:** Operations that must succeed or fail together
  execute in one explicit transaction.
 - **Data preservation:** Schema evolution preserves existing records unless a
  separately reviewed product decision explicitly requires otherwise.
 Migration tests should verify upgrade, downgrade, constraints, and existing-data
 behavior. Runtime schema changes are legacy compatibility behavior and are not
 permitted for new work.
 ## API Philosophy
 MinerOps APIs use resource-oriented HTTP semantics. New or replaced domain APIs
 follow these conventions:
 - Use nouns for resources and standard HTTP methods for operations.
 - Validate path, query, and body structure at the router boundary.
 - Apply business validation in the service.
 - Use Pydantic models for request and response contracts.
 - Return predictable status codes and response shapes.
 - Translate domain exceptions into consistent, non-leaking API errors.
 - Bound collection endpoints with pagination.
 - Make supported filtering, searching, and ordering explicit.
 - Preserve stable identifiers in URLs and response contracts.
 - Enforce authentication and permissions on the server.
 - Avoid duplicate endpoints for the same use case.
 **Current implementation:** Many routes return the common `data`, `meta`, and
 `error` envelope defined in `main.py`. Pydantic validation, pagination, and
 error shapes are not uniform across all existing routes. Identity supplies
 current authentication and role-aware dependencies. New endpoints should follow
 the conventions above while preserving compatibility for any route they
 replace.
 Pagination metadata should be sufficient for callers to understand the returned
 window. Filters must be parameterized in repositories, not interpolated in
 routers. Database exceptions and internal details must not be exposed to API
 clients.
 ## Testing Philosophy
 Tests follow the same boundaries as production code.
 - **Unit tests:** Exercise service rules with substituted dependencies. They
  should not require FastAPI or a database when the behavior is purely domain
  logic.
 - **Repository tests:** Exercise real queries, constraints, associations, and
  transactions against a disposable PostgreSQL database.
 - **Migration tests:** Verify schema upgrade, downgrade, constraint behavior,
  and existing-data preservation.
 - **Integration tests:** Exercise application startup, migrations,
  authentication, authorization, safety behavior, and cross-layer workflows in
  a disposable Compose stack.
 - **Architecture tests:** Protect dependency rules that can be checked
  mechanically, such as the Inventory package containing no FastAPI imports or
  router registration.
 Current Inventory tests use a fake repository for service rules and PostgreSQL
 for repository behavior. The focused runner migrates a disposable database
 through Alembic before executing the suite. The development baseline starts an
 isolated Compose project and checks schema, login gating, roles, and safety
 interlocks.
 Before committing, run validation appropriate to the change. This commonly
 includes Python compilation, shell syntax checks, focused unit or repository
 tests, migration tests for schema work, the disposable development baseline for
 cross-application changes, and `git diff --check`.
 Keep commits small and reviewable. A commit should express one coherent change,
 include focused tests when behavior changes, and avoid unrelated formatting or
 refactoring. Broad integration coverage does not replace a focused test for a
 new rule.
 ## Future Evolution
 MinerOps will evolve incrementally as existing responsibilities are moved into
 clear domain boundaries. New infrastructure or domain packages must preserve
 the router-to-service-to-repository dependency direction, explicit ownership,
 and testability rules in this document. Architectural changes should clarify
 those rules rather than bypass them.
 ## When In Doubt
 When a feature does not fit naturally into an existing module, refine the
 architecture instead of weakening architectural boundaries. Clear boundaries
 are easier to maintain than clever shortcuts.