docs: add architecture guide

docs/ARCHITECTURE.md

@@ -0,0 +1,457 @@
+# MinerOps Architecture
+
+## Purpose
+
+This document is the primary engineering architecture guide for MinerOps. It
+explains the codebase's current structure and the rules contributors must follow
+when adding or changing features.
+
+MinerOps is self-hosted software with operational data and workflows that must
+remain understandable as the system grows. Maintainability, testability, and
+clear boundaries are primary architectural goals. A feature is not complete
+merely because it works; its responsibilities and dependencies must also be in
+the correct layer.
+
+**Current implementation:** The repository is transitional. Most existing HTTP
+routes, persistence, and server-rendered assets are concentrated in
+`backend/app/main.py`. Identity has been separated into a FastAPI router.
+Inventory is the first domain with explicit models, exceptions, a service, a
+repository, and focused tests.
+
+**Engineering rule:** New domain code follows the layered architecture in this
+document. Existing integrated code is compatibility context, not a pattern to
+copy.
+
+## Core Principles
+
+### Separation of Concerns
+
+Transport, business rules, persistence, and presentation are different
+responsibilities. Keep them separate so each can change and be tested without
+requiring unrelated changes.
+
+### Single Responsibility
+
+Each module, class, and function should have one clear reason to change. A
+router handles HTTP. A service handles domain behavior. A repository handles
+persistence. Combining these responsibilities makes ownership ambiguous and
+tests expensive.
+
+### Thin HTTP Layer
+
+Routers accept and validate transport input, invoke services, and map results to
+HTTP responses. They do not implement business workflows and do not access the
+database directly.
+
+### Business Logic Belongs in Services
+
+Services own normalization, domain validation, decisions, and orchestration.
+They must not depend on FastAPI request or response objects, route decorators,
+or HTTP status codes.
+
+### Persistence Belongs in Repositories
+
+Repositories own queries, writes, database-facing filters, transaction
+mechanics, and persistence mapping. They contain no HTTP behavior, permission
+rules, or product decisions.
+
+### Explicit Dependencies
+
+Dependencies are visible at construction or call boundaries. The current
+`InventoryService` receives its repository through its constructor, allowing a
+substitute repository in service tests and a PostgreSQL repository in
+integration tests.
+
+### Incremental Evolution
+
+MinerOps has working behavior that must remain usable while responsibilities
+are separated. Improve boundaries in small, verified changes. Do not require a
+large rewrite before a domain can become clearer.
+
+### Testability
+
+Architecture should make the correct test easy to write. Business rules should
+be testable without FastAPI or PostgreSQL. Repository behavior should be
+testable against a disposable PostgreSQL database. Cross-application behavior
+should be testable through the disposable development stack.
+
+## High-Level Architecture
+
+The required dependency flow for domain-backed application features is:
+
+```text
+Browser
+  |
+  v
+FastAPI Routers
+  |
+  v
+Services
+  |
+  v
+Repositories
+  |
+  v
+PostgreSQL
+```
+
+- **Browser:** Presents the interface and sends authenticated HTTP requests. The
+  current interface is served by FastAPI using HTML, CSS, and JavaScript.
+- **FastAPI routers:** Define the HTTP contract, enforce transport-level
+  validation and authorization dependencies, call services, and serialize
+  responses.
+- **Services:** Implement use cases and domain rules without knowing how the
+  request arrived.
+- **Repositories:** Perform database operations and expose persistence behavior
+  needed by services.
+- **PostgreSQL:** Stores authoritative operational records and enforces durable
+  constraints.
+
+Redis and Nginx are current infrastructure components, but they do not change
+this domain dependency flow. Redis is configured as an application service and
+included in health checks. Nginx is the gateway in the development Compose
+stack.
+
+## Repository Organization
+
+The repository is organized by runtime responsibility and supporting material:
+
+- **`backend/`:** Python application code, FastAPI composition, domain packages,
+  Alembic migrations, backend dependencies, tests, and the backend container
+  definition.
+- **`frontend/` (planned):** A future standalone frontend may live here after a
+  frontend technology and migration plan are selected. No `frontend/` directory
+  or standalone frontend application exists today.
+- **`docs/`:** Engineering and product documentation that applies across
+  domains.
+- **`branding/`:** Product identity guidelines and source vector assets. It has
+  no runtime application responsibility.
+- **`scripts/`:** Development setup, disposable test runners, schema checks,
+  retention helpers, and operational utilities.
+
+Root Compose files define containerized services for the backend, PostgreSQL,
+and Redis. The development configuration also defines an Nginx gateway and runs
+Alembic before starting Uvicorn.
+
+## Backend Layers
+
+### Routers
+
+Routers are the HTTP boundary. A domain router should be an `APIRouter` included
+by the application entry point.
+
+Router responsibilities:
+
+- Declare paths, HTTP methods, parameters, and response status codes.
+- Parse path, query, header, and body values.
+- Apply authentication and permission dependencies.
+- Construct domain input models from transport input.
+- Call services.
+- Translate domain exceptions into consistent HTTP errors.
+- Serialize service results into documented response models.
+
+Routers must never contain:
+
+- SQL or database connection management.
+- Business rules, normalization policy, or lifecycle decisions.
+- Multi-step persistence workflows or transaction control.
+- Device protocol implementations.
+- Copies of logic already owned by a service or repository.
+
+A router may reject structurally invalid HTTP input. It must delegate questions
+such as whether an Asset ID is valid for a use case, whether a lifecycle change
+is allowed, or whether a record conflicts with existing state to the service.
+
+**Current implementation:** `backend/app/main.py` creates the `FastAPI`
+application and includes the router from `backend/app/identity.py`. Identity is
+the current example of router extraction. Most other routes remain registered
+directly on `app` and often contain SQL or business logic. Those routes predate
+the current layering rules and must not be used as templates for new domains.
+The Inventory package currently has no router and registers no HTTP endpoints.
+
+### Services
+
+Services implement application use cases for a domain. A service coordinates
+repositories and applies rules that must remain consistent regardless of the
+transport calling it.
+
+Service responsibilities:
+
+- Normalize values according to domain semantics.
+- Validate business rules and allowed state transitions.
+- Detect conflicts meaningful to the domain.
+- Coordinate multiple repository operations as one use case.
+- Establish required transaction scope through repository abstractions.
+- Return domain models or domain-specific results.
+- Raise domain exceptions rather than HTTP exceptions.
+
+The current `InventoryService` demonstrates these responsibilities. It:
+
+- Normalizes Asset IDs and optional text.
+- Normalizes and deduplicates tags.
+- Validates `active`, `spare`, and `retired` lifecycle states.
+- Detects duplicate Asset IDs.
+- Defines never-seen timestamps for manually created miners.
+- Coordinates miner creation or update with tag associations.
+- Maps database-shaped records to Inventory summary and detail models.
+
+The service imports no FastAPI components. Its extension hooks are domain-level
+hooks, not route callbacks. A service must not inspect cookies, headers, request
+objects, or response objects.
+
+The current Inventory service catches a PostgreSQL uniqueness exception to
+translate a race into `DuplicateAssetID`. This is a known persistence coupling.
+New code should prevent database-specific exceptions from spreading further
+through the service layer, preferably by translating them at an explicit
+repository boundary.
+
+### Repositories
+
+Repositories are the persistence boundary for a domain. They expose operations
+in terms useful to services while keeping SQL and database mechanics out of
+higher layers.
+
+Repository responsibilities:
+
+- Execute parameterized reads and writes.
+- Implement database-facing search, filtering, ordering, and pagination.
+- Manage connections, commits, rollbacks, and transaction-bound instances.
+- Maintain associations owned by the domain.
+- Map database values into persistence records.
+- Translate low-level persistence failures when an owned abstraction exists.
+
+Repositories contain database access only. They must not:
+
+- Import FastAPI or router modules.
+- Choose HTTP responses or status codes.
+- Check the current user's permissions.
+- Normalize values according to product policy.
+- Decide whether a lifecycle transition or operational action is allowed.
+
+`InventoryRepository` is the current reference implementation. It uses
+`psycopg`, accepts an optional connection factory, supports explicit transaction
+scope, and owns miner and tag CRUD, search, and associations. Its methods return
+database-shaped dictionaries; `InventoryService` maps those records to domain
+models.
+
+Dynamic SQL must be restricted to trusted identifiers or fixed fragments.
+User-provided values always use database parameters. Repositories must not accept
+raw SQL from routers or services.
+
+### Database
+
+PostgreSQL is the authoritative persistent store for MinerOps. Application rules
+that must hold for every writer are also represented as database constraints
+where practical, including uniqueness, allowed values, and referential
+integrity.
+
+Alembic is the authority for new persistent schema changes. Migrations must
+state how existing data is handled and provide a downgrade when it can be done
+without ambiguity or data loss.
+
+**Current implementation:** Alembic migrations cover the foundational,
+scanning, audit, identity, role, and Inventory schemas. Some legacy code in
+`main.py` still creates or alters tables at runtime for compatibility. Do not add
+new runtime DDL. Move persistent schema changes through reviewed Alembic
+migrations.
+
+The repository also contains a SQLAlchemy engine and session factory, while
+most current operational persistence and the Inventory repository use direct
+`psycopg`; some existing agent paths use `asyncpg`. A domain must choose and
+encapsulate its persistence mechanism inside its repository rather than expose
+driver details to routers.
+
+## Domains
+
+A domain package groups one operational capability behind an explicit public
+interface. Inventory is the current reference domain:
+
+```text
+backend/app/inventory/
+    __init__.py
+    exceptions.py
+    models.py
+    repository.py
+    service.py
+```
+
+- `models.py` defines typed request, update, search, summary, detail, and tag
+  contracts.
+- `exceptions.py` defines errors meaningful to Inventory callers.
+- `service.py` defines Inventory rules and use cases.
+- `repository.py` defines Inventory persistence.
+- `__init__.py` exposes the package's intended public surface.
+
+Inventory's internal asset identity is the miner UUID. Product-facing Asset IDs
+are normalized and checked independently of mutable network fields. The domain
+maps product terminology such as `asset_id` and `psu_serial` to existing database
+columns at its service and persistence boundary.
+
+Future domain packages should follow the same separation when it fits their use
+cases. Identity, Repairs, Monitoring, Operations, Scheduling, and Locations are
+examples of possible domain boundaries, not a fixed package list. A domain
+boundary must be based on ownership of rules and data, not on menu pages or
+route prefixes alone.
+
+A domain may call another domain only through an explicit service contract. It
+must not import another domain's repository to update tables it does not own.
+Shared read requirements should use a clearly owned query service or read model
+rather than duplicating cross-domain SQL.
+
+## Dependency Rules
+
+Dependencies flow downward:
+
+```text
+Routers
+  |
+  v
+Services
+  |
+  v
+Repositories
+  |
+  v
+Database
+```
+
+Allowed dependencies:
+
+- The application entry point may include routers and install application-wide
+  middleware.
+- Routers may depend on domain models, services, and exceptions.
+- Services may depend on their domain's models, exceptions, and explicit
+  repository contracts.
+- Repositories may depend on configuration, database drivers, and
+  persistence-specific utilities.
+- Tests may replace repositories or connection factories at defined dependency
+  boundaries.
+
+Examples of allowed flow:
+
+- An Inventory router validates a UUID, calls `InventoryService.get_miner`, and
+  translates `MinerNotFound` into an HTTP 404 response.
+- `InventoryService.create_miner` normalizes an Asset ID and coordinates miner
+  and tag writes through an `InventoryRepository` transaction.
+- `InventoryRepository.search_miners` builds parameterized filters and executes
+  them against PostgreSQL.
+
+Prohibited dependencies:
+
+- Repositories importing services, routers, or `app.main`.
+- Services importing FastAPI request or response classes.
+- Routers opening database connections or issuing SQL.
+- Domain code importing globals from `app.main`.
+- A lower layer choosing permissions or HTTP status codes.
+- A domain bypassing another domain's service to write its tables.
+- Circular imports between domains or layers.
+
+Examples of prohibited flow:
+
+- A route updating `miners` directly after checking request JSON.
+- A service returning `JSONResponse` or raising `HTTPException`.
+- A repository checking whether the current user is an administrator.
+- A Repairs service importing `InventoryRepository` to mutate miner ownership
+  without an Inventory service contract.
+
+Dependency direction is an architectural constraint, not a naming convention.
+Putting SQL in a file named `service.py` does not make it service logic.
+
+## Database Philosophy
+
+- **Alembic-managed schema:** New persistent tables, columns, constraints, and
+  indexes are introduced through versioned migrations.
+- **Explicit migrations:** Every migration explains existing-data behavior,
+  upgrade ordering, and downgrade behavior. Migrations do not silently merge,
+  rename, or discard operational records.
+- **Stable identifiers:** Internal identity uses immutable identifiers rather
+  than mutable IP addresses, MAC addresses, names, or locations.
+- **Repository ownership:** Each domain has one clear persistence boundary for
+  its owned records. Other layers do not duplicate its SQL.
+- **Database enforcement:** PostgreSQL constraints protect invariants that must
+  hold independently of application code.
+- **Transactional consistency:** Operations that must succeed or fail together
+  execute in one explicit transaction.
+- **Data preservation:** Schema evolution preserves existing records unless a
+  separately reviewed product decision explicitly requires otherwise.
+
+Migration tests should verify upgrade, downgrade, constraints, and existing-data
+behavior. Runtime schema changes are legacy compatibility behavior and are not
+permitted for new work.
+
+## API Philosophy
+
+MinerOps APIs use resource-oriented HTTP semantics. New or replaced domain APIs
+follow these conventions:
+
+- Use nouns for resources and standard HTTP methods for operations.
+- Validate path, query, and body structure at the router boundary.
+- Apply business validation in the service.
+- Use Pydantic models for request and response contracts.
+- Return predictable status codes and response shapes.
+- Translate domain exceptions into consistent, non-leaking API errors.
+- Bound collection endpoints with pagination.
+- Make supported filtering, searching, and ordering explicit.
+- Preserve stable identifiers in URLs and response contracts.
+- Enforce authentication and permissions on the server.
+- Avoid duplicate endpoints for the same use case.
+
+**Current implementation:** Many routes return the common `data`, `meta`, and
+`error` envelope defined in `main.py`. Pydantic validation, pagination, and
+error shapes are not uniform across all existing routes. Identity supplies
+current authentication and role-aware dependencies. New endpoints should follow
+the conventions above while preserving compatibility for any route they
+replace.
+
+Pagination metadata should be sufficient for callers to understand the returned
+window. Filters must be parameterized in repositories, not interpolated in
+routers. Database exceptions and internal details must not be exposed to API
+clients.
+
+## Testing Philosophy
+
+Tests follow the same boundaries as production code.
+
+- **Unit tests:** Exercise service rules with substituted dependencies. They
+  should not require FastAPI or a database when the behavior is purely domain
+  logic.
+- **Repository tests:** Exercise real queries, constraints, associations, and
+  transactions against a disposable PostgreSQL database.
+- **Migration tests:** Verify schema upgrade, downgrade, constraint behavior,
+  and existing-data preservation.
+- **Integration tests:** Exercise application startup, migrations,
+  authentication, authorization, safety behavior, and cross-layer workflows in
+  a disposable Compose stack.
+- **Architecture tests:** Protect dependency rules that can be checked
+  mechanically, such as the Inventory package containing no FastAPI imports or
+  router registration.
+
+Current Inventory tests use a fake repository for service rules and PostgreSQL
+for repository behavior. The focused runner migrates a disposable database
+through Alembic before executing the suite. The development baseline starts an
+isolated Compose project and checks schema, login gating, roles, and safety
+interlocks.
+
+Before committing, run validation appropriate to the change. This commonly
+includes Python compilation, shell syntax checks, focused unit or repository
+tests, migration tests for schema work, the disposable development baseline for
+cross-application changes, and `git diff --check`.
+
+Keep commits small and reviewable. A commit should express one coherent change,
+include focused tests when behavior changes, and avoid unrelated formatting or
+refactoring. Broad integration coverage does not replace a focused test for a
+new rule.
+
+## Future Evolution
+
+MinerOps will evolve incrementally as existing responsibilities are moved into
+clear domain boundaries. New infrastructure or domain packages must preserve
+the router-to-service-to-repository dependency direction, explicit ownership,
+and testability rules in this document. Architectural changes should clarify
+those rules rather than bypass them.
+
+## When In Doubt
+
+When a feature does not fit naturally into an existing module, refine the
+architecture instead of weakening architectural boundaries. Clear boundaries
+are easier to maintain than clever shortcuts.