a3b300b76a
- AGENTS.md: canonical project entry point (Quick Nav, pipeline, constraints) - docs/: complete docs tree — architecture, API spec, DB schema, infra, runbook, requirements, tech stack, principles, reference ADRs, guides, tasks backlog, testing strategy - tests/README.md: test commands, structure, known gaps - README.md / CLAUDE.md / DEPLOYMENT.md: updated with canonical doc links - .archive/: backup of pre-documentation-pipeline originals - backend/uv.lock: uv dependency lockfile - Delete committed __pycache__ .pyc files (should have been gitignored) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
113 lines
4.7 KiB
Markdown
113 lines
4.7 KiB
Markdown
# Testing Strategy Guide
|
||
|
||
<!-- SCOPE: testing-strategy | owner: ln-140 | generated: 2026-04-29 -->
|
||
|
||
## Philosophy
|
||
|
||
Risk-Based Testing: **Priority = Business Impact (1–5) × Probability of failure (1–5)**.
|
||
|
||
| Priority | Decision | Example |
|
||
|----------|----------|---------|
|
||
| ≥ 15 | MUST test | RBAC logic, job state machine, audit logger |
|
||
| 9–14 | SHOULD test | Translation pipeline, TTS routing |
|
||
| ≤ 8 | SKIP (manual sufficient) | Email template rendering, UI cosmetics |
|
||
|
||
Write tests for business logic, not for framework behaviour. Never test that FastAPI routes a request — test that YOUR business logic in the handler produces the correct outcome.
|
||
|
||
---
|
||
|
||
## Current Coverage (as of 2026-04-29 audit)
|
||
|
||
| Layer | Files | Files with tests | Risk-weighted coverage |
|
||
|-------|-------|-----------------|----------------------|
|
||
| Backend | 118 | 8 (7%) | ~3% |
|
||
| Frontend | 98 | ~12 (12%) | — |
|
||
| E2E | — | 3 spec files | Effectively 0 (most tests skipped) |
|
||
|
||
**Critical gap:** All Celery tasks (10 files) and 19 service files have zero test coverage. See full audit at `/tmp/audit/test-audit.md`.
|
||
|
||
---
|
||
|
||
## Test Pyramid
|
||
|
||
| Level | Framework | Location | Current count |
|
||
|-------|-----------|----------|--------------|
|
||
| Unit (backend) | pytest + AsyncMock | `backend/tests/unit/` | 8 files, ~338 assertions |
|
||
| Unit (frontend) | Vitest + RTL | `frontend/src/**/__tests__/` | 9 files, ~218 assertions |
|
||
| Integration (backend) | pytest + FastAPI TestClient | `backend/tests/integration/` | Does not exist yet |
|
||
| E2E | Playwright | `frontend/tests/e2e/` | 3 files, mostly skipped |
|
||
|
||
---
|
||
|
||
## Test Commands
|
||
|
||
| Command | What it runs |
|
||
|---------|-------------|
|
||
| `cd backend && poetry run pytest` | All backend unit tests |
|
||
| `cd backend && poetry run pytest -v tests/unit/test_security.py` | Single test file |
|
||
| `cd frontend && npm run test` | All frontend unit tests (Vitest) |
|
||
| `cd frontend && npm run test:e2e` | Playwright E2E tests |
|
||
| `cd frontend && npm run test:coverage` | Unit tests with coverage report |
|
||
| `docker compose exec backend python -m pytest` | Tests inside Docker (for integration tests) |
|
||
|
||
---
|
||
|
||
## What Exists and Is High-Value
|
||
|
||
| Test file | Value | What it tests |
|
||
|-----------|-------|--------------|
|
||
| `backend/tests/unit/test_security.py` | HIGH | JWT encode/decode, expiry, type fields, password hashing |
|
||
| `backend/tests/unit/test_vtt.py` | HIGH | VTT parsing (26 tests) |
|
||
| `backend/tests/unit/test_vtt_retimer.py` | HIGH | VTT timing logic (27 tests) |
|
||
| `frontend/src/lib/__tests__/auth.test.ts` | HIGH | JWT in-memory store, refresh flow |
|
||
| `frontend/src/components/Auth/__tests__/RequireAuth.test.tsx` | HIGH | Auth guard redirect |
|
||
|
||
---
|
||
|
||
## Priority Gaps to Fill
|
||
|
||
The following are MUST-fill based on Priority ≥15:
|
||
|
||
| Priority | Module | Gap |
|
||
|----------|--------|-----|
|
||
| 25 | `tasks/ingest_and_ai.py` | Job state machine — zero tests |
|
||
| 20 | `core/authz.py` | RBAC permission checks — zero tests |
|
||
| 20 | `services/audit_logger.py` | Audit trail correctness — zero tests |
|
||
| 20 | `services/glossary_service.py` | Hybrid retrieval — zero tests |
|
||
| 16 | `services/language_qc.py` | QC state transitions — zero tests |
|
||
| 16 | `tasks/translate_and_synthesize.py` | Translation pipeline — zero tests |
|
||
|
||
Full test plan at `/tmp/audit/test-plan.md`.
|
||
|
||
---
|
||
|
||
## Anti-Patterns to Avoid
|
||
|
||
| Anti-pattern | Why | Fix |
|
||
|-------------|-----|-----|
|
||
| Hardcoded job IDs like `test-job-123` | Non-existent in test DB | Use factories to create real test data |
|
||
| `with patch(...) as mock:` in every test method | Setup duplication | Move to `@pytest.fixture(autouse=True)` |
|
||
| `MagicMock()` on async functions | Silently returns a mock, not a coroutine | Use `AsyncMock()` |
|
||
| Testing that a library function was called | Tests library, not our logic | Test the business outcome |
|
||
| E2E tests that are `.skip` | They provide no coverage | Implement auth fixture and un-skip |
|
||
|
||
---
|
||
|
||
## Infrastructure Required Before Writing Integration/E2E Tests
|
||
|
||
| Blocker | What's needed |
|
||
|---------|--------------|
|
||
| Backend `conftest.py` | Shared `MockSettings`, `mock_db`, `test_user_factory`, `test_job_factory` |
|
||
| Celery test mode | `task_always_eager=True` fixture for synchronous task execution |
|
||
| Playwright auth fixture | Wire `tests/helpers/auth.ts` into `beforeEach` in all spec files |
|
||
| Playwright seed fixture | `tests/fixtures/seed.ts` to create test jobs, glossary, linguists |
|
||
| Mock AI responses | `tests/mocks/gemini-responses/*.json` fixtures |
|
||
|
||
---
|
||
|
||
## Maintenance
|
||
|
||
**Update triggers:** New test file added, coverage target changes, new testing tool added.
|
||
**Verification:** All commands in the Commands table execute without error. Priority gap table matches the current test-audit report.
|
||
|
||
<!-- END SCOPE: testing-strategy -->
|