Oliver/amazon-transcreation
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
amazon-transcreation/README.md
DJP 5e0a148b96 feat: add token usage tracking, feedback highlighting, cost on cards, help page 
- Wire token usage from LLM agents through pipeline context to DB and frontend
- Agents 2 and 4 accumulate input/output tokens and cost into PipelineContext
- job_tasks.py saves token totals to locale instance after pipeline completion
- Monitoring cards show total tokens and estimated cost instead of broken 0/0
- Make feedback highlighting bolder: colored card borders, stronger button states
- Add estimated cost display to dashboard job cards
- Add Help page with full documentation and link in sidebar navigation
- Comprehensive README with ASCII architecture diagrams

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-04-10 16:47:36 -04:00

678 lines
35 KiB
Markdown
Raw Blame History

# Amazon AI Transcreation Platform
An AI-powered transcreation platform that adapts Amazon marketing copy across 12 European locales using Claude LLM agents. Replaces a manual LibreChat workflow with structured, one-click multi-locale processing, real-time monitoring, in-app review, and proper job/file management.
---
## Table of Contents
- [Architecture Overview](#architecture-overview)
- [How It Works](#how-it-works)
- [The Agent Pipeline](#the-agent-pipeline)
- [Tech Stack](#tech-stack)
- [Getting Started](#getting-started)
- [Configuration](#configuration)
- [Storage Layout](#storage-layout)
- [Supported Locales & Channels](#supported-locales--channels)
- [API Reference](#api-reference)
- [Database Schema](#database-schema)
- [User Guide](#user-guide)
- [Development](#development)
- [Deployment](#deployment)
---
## Architecture Overview
```
┌─────────────────────────────────────────────────────────────────────────────────┐
│ AMAZON TRANSCREATION PLATFORM │
└─────────────────────────────────────────────────────────────────────────────────┘
┌──────────────────┐ ┌──────────────────────────────────────────────────┐
│ │ HTTP │ FastAPI Backend │
│ Next.js 14 │ ◄─────►│ │
│ Frontend │ REST │ ┌────────────┐ ┌──────────┐ ┌─────────────┐ │
│ │ │ │ Auth │ │ Jobs │ │ Output │ │
│ ┌────────────┐ │ Poll │ │ Service │ │ API │ │ API │ │
│ │ Dashboard │ │ (3sec) │ └────────────┘ └────┬─────┘ └─────────────┘ │
│ │ Job Wizard │ │ │ │ │
│ │ Monitor │ │ │ ┌────────────────────▼───────────────────────┐ │
│ │ Review │ │ │ │ Celery Task Queue │ │
│ │ Admin │ │ │ │ (4 concurrent workers) │ │
│ └────────────┘ │ │ └────────────────────┬───────────────────────┘ │
└──────────────────┘ │ │ │
│ ┌────────────────────▼───────────────────────┐ │
│ │ Pipeline Orchestrator │ │
│ │ │ │
│ │ VALIDATE ► TM_RETRIEVE ► RANK │ │
│ │ │ │ │ │ │
│ │ ▼ ▼ ▼ │ │
│ │ TRANSCREATE ◄── COMPLY ──► FORMAT │ │
│ │ │ (retry x3) │ │ │
│ │ ▼ ▼ │ │
│ │ DONE │ │
│ └─────────────────────────────────────────────┘ │
└──────────┬──────────────────────────┬─────────────┘
│ │
┌──────────▼──────┐ ┌──────────▼──────────┐
│ PostgreSQL 16 │ │ Redis 7 │
│ │ │ │
│ 11 tables │ │ Celery broker │
│ Jobs, Output, │ │ Task results │
│ Users, Audit │ │ WebSocket pub/sub │
└─────────────────┘ └─────────────────────┘
┌─────────────────┐ ┌─────────────────────┐
│ Claude API │ │ File Storage │
│ (Anthropic) │ │ │
│ │ │ /storage/amazon/ │
│ Agents 2 & 4 │ │ tm/ (JSONL) │
│ make LLM calls │ │ ref/ (JSON) │
└─────────────────┘ └─────────────────────┘
```
---
## How It Works
### The Workflow (End to End)
```
USER PLATFORM CLAUDE API
│ │ │
│ 1. Create Job │ │
│ (campaign, locale, │ │
│ channel, programme) │ │
│ ──────────────────────────►│ │
│ │ │
│ 2. Upload Source xlsx │ │
│ (EN_GB lines, char │ │
│ limits, copy types) │ │
│ ──────────────────────────►│ │
│ │ │
│ 3. Launch │ │
│ ──────────────────────────►│ │
│ │ Celery dispatches per-locale │
│ │ tasks in PARALLEL (up to 4) │
│ │ ─────────┐ │
│ │ │ │
│ 4. Monitor Progress │ ┌───────▼────────┐ │
│ (polls every 3 sec) │ │ Agent Pipeline │ │
│ ◄─── 25% Matching TM ─────│ │ │──── LLM ────►│
│ ◄─── 50% Translating ─────│ │ 6 stages per │◄── matches ──│
│ ◄─── 65% Batch 2/4 ───────│ │ locale │──── LLM ────►│
│ ◄─── 80% Batch 4/4 ───────│ │ │◄── drafts ───│
│ ◄─── 90% Formatting ──────│ └───────┬─────────┘ │
│ ◄── 100% Complete ────────│ │ │
│ │ ┌───────▼────────┐ │
│ 5. Review Output │ │ Output saved │ │
│ (per-locale, per-line │ │ to DB + xlsx │ │
│ with confidence tiers) │ └────────────────┘ │
│ ──────────────────────────►│ │
│ │ │
│ 6. Approve / Revise │ │
│ ──────────────────────────►│ │
│ │ │
│ 7. Download xlsx │ │
│ ◄──────────────────────────│ │
```
### What Happens When You Launch a Job
1. **Job created** with campaign name, programme (Retail/Prime/Brand), channel (Value/Mass/Onsite/Outbound), and target locales
2. **Source file uploaded** - an xlsx with English (en_GB) marketing copy, character limits, copy types, and creative guidance
3. **Launch** dispatches one Celery task per locale - up to 4 run in parallel
4. Each locale runs through the **6-agent pipeline** (see below)
5. Real-time **progress updates** are stored in the database and polled by the frontend every 3 seconds
6. On completion, output is viewable in the **review interface** with confidence badges, backtranslations, and rationale
7. **Export** downloads a formatted xlsx file
---
## The Agent Pipeline
Each locale goes through 6 sequential agents. The pipeline includes a compliance retry loop (max 3 attempts).
```
┌─────────────────────────────────────────────────────────────────────────┐
│ PER-LOCALE PIPELINE │
│ │
│ ┌──────────────┐ Deterministic. Parses xlsx, loads glossary, │
│ │ AGENT 1 │ blacklist, TOV, locale considerations, and │
│ │ Validator │ date/percent format files. Builds PipelineContext. │
│ │ [no LLM] │ ~0.1 seconds │
│ └──────┬───────┘ │
│ │ 10% │
│ ┌──────▼───────┐ Loads TM file (~128 entries for de-DE Value). │
│ │ AGENT 2 │ Sends ALL source lines + ALL TM entries to │
│ │ TM Retrieval│ Claude in one call. LLM finds semantic matches │
│ │ [LLM call] │ (exact / high / medium similarity). │
│ └──────┬───────┘ ~45 seconds, ~$0.13 │
│ │ 25% │
│ ┌──────▼───────┐ Deterministic. Scores each match by channel │
│ │ AGENT 3 │ fit, sub-channel fit, and recency (year). │
│ │ Ranker │ Assigns confidence: HIGH (1 opt), MODERATE │
│ │ [no LLM] │ (2 opts), LOW (3 opts). │
│ └──────┬───────┘ ~0.01 seconds │
│ │ 40% │
│ ┌──────▼───────┐ Core creative agent. Processes source lines in │
│ │ AGENT 4 │ batches of 15. System prompt includes voice │
│ │ Transcreator│ profile, glossary, blacklist, TOV guidelines. │
│ │ [LLM call] │ Generates options with backtranslations and │
│ │ x4 batches │ rationale. Cites TM entries where applicable. │
│ └──────┬───────┘ ~4 min (4 batches), ~$0.36 │
│ │ 50-80% │
│ ┌──────▼───────┐ Deterministic. 3 checks: │
│ │ AGENT 5 │ 1. Character count vs char_limit │
│ │ Compliance │ 2. Blacklist term scanning │
│ │ [no LLM] │ 3. Domain check (Amazon.co.uk in non-en_GB) │
│ └──────┬───────┘ If errors: retry from Agent 4 (max 3x) │
│ │ 82% ~0.01 seconds │
│ ┌──────▼───────┐ │
│ │ AGENT 6 │ Deterministic. Generates output xlsx │
│ │ Formatter │ (Tab 1: output table, Tab 2: summary). │
│ │ [no LLM] │ Saves output rows to database. │
│ └──────┬───────┘ ~0.1 seconds │
│ │ 100% │
│ ▼ │
│ DONE (~5.5 min total, ~$0.49 per locale) │
└─────────────────────────────────────────────────────────────────────────┘
```
### Confidence Tiers and Option Counts
```
TM Match Quality Confidence Options Generated
───────────────────────── ──────────── ──────────────────
Same channel + recent year HIGH 1 option (anchored to TM)
Cross-channel or older MODERATE 2 options
No TM match found LOW 3 creative options
```
### Voice Profiles (per Programme)
| Programme | Voice Attributes |
|-----------|-----------------|
| **Retail** | Real, Clear, Playful, Witty |
| **Prime** | Optimistic, Honest, Self-aware, Witty, Relatable |
| **Brand** | Authentic, Customer-obsessed, Intelligent, Warm, Understated |
### Deterministic Modules
The pipeline uses 9 pure-Python modules (no LLM) for specific tasks:
| Module | Purpose |
|--------|---------|
| `source_file_parser` | Parse xlsx, validate columns, detect display format |
| `tm_file_loader` | Parse JSONL TM files (compact + multi-field formats) |
| `ref_file_loader` | Load glossary, blacklist, TOV, locale considerations |
| `character_counter` | Unicode grapheme cluster counting (not `len()`) |
| `blacklist_scanner` | Exact + root-based forbidden term matching |
| `date_format_validator` | Validate date/percent formats per locale |
| `domain_substitutor` | Amazon.co.uk to locale-specific domain mapping |
| `line_break_normaliser` | Handle `\n` for TM queries vs Excel output |
| `excel_writer` | Generate formatted xlsx with output + summary tabs |
---
## Tech Stack
```
┌───────────────────────────────────────────────────────────────┐
│ FRONTEND │ BACKEND │ INFRASTRUCTURE │
├───────────────────────┼──────────────────────┼────────────────┤
│ Next.js 14 (App Rtr) │ Python 3.12 │ Docker Compose │
│ React 18 │ FastAPI │ PostgreSQL 16 │
│ TypeScript 5.4 │ SQLAlchemy 2 (async) │ Redis 7 │
│ Tailwind CSS 3.4 │ Alembic (migrations) │ Nginx (prod) │
│ Radix UI primitives │ Celery 5.4 │ │
│ Recharts (charts) │ Pydantic v2 │ │
│ Axios │ Anthropic SDK │ │
│ Lucide (icons) │ openpyxl │ │
│ │ bcrypt + JWT │ │
└───────────────────────┴──────────────────────┴────────────────┘
```
---
## Getting Started
### Prerequisites
- Docker and Docker Compose v2
- An Anthropic API key (for Claude)
- Node.js 18+ (for frontend builds)
- Git
### Quick Start
```bash
# 1. Clone the repository
git clone git@bitbucket.org:zlalani/amazon-transcreation.git
cd amazon-transcreation
# 2. Copy environment file and set your API key
cp .env.example .env
# Edit .env and set:
# ANTHROPIC_API_KEY=sk-ant-your-key-here
# JWT_SECRET_KEY=a-random-secret-string
# 3. Start all services
make up
# or: docker compose up -d
# 4. Run database migrations
make migrate
# 5. Seed default data (Amazon client + test users)
make seed
# 6. Build the frontend
cd frontend && npm install && npm run build && cd ..
# 7. Access the application
# Backend API: http://localhost:8040/api/v1
# Frontend: http://localhost:3000
```
### Default Users (after seeding)
| Email | Role | Password |
|-------|------|----------|
| admin@amazon.com | Admin | admin123 |
| manager@amazon.com | TM Manager | admin123 |
| reviewer@amazon.com | Reviewer | admin123 |
### Makefile Commands
| Command | Description |
|---------|-------------|
| `make up` | Start all Docker services |
| `make down` | Stop all services |
| `make build` | Rebuild Docker images |
| `make migrate` | Run database migrations |
| `make seed` | Seed default client and test users |
| `make test` | Run backend test suite |
| `make shell` | Open a bash shell in the backend container |
| `make logs` | Stream all container logs |
| `make restart` | Restart backend + Celery worker |
| `make db-shell` | Open PostgreSQL interactive shell |
| `make redis-cli` | Open Redis CLI |
---
## Configuration
All configuration is via environment variables in `.env`:
| Variable | Default | Description |
|----------|---------|-------------|
| `DATABASE_URL` | `postgresql+asyncpg://...` | PostgreSQL async connection string |
| `REDIS_URL` | `redis://redis:6379/0` | Redis connection for Celery + pub/sub |
| `ANTHROPIC_API_KEY` | *(required)* | Your Anthropic API key for Claude |
| `JWT_SECRET_KEY` | *(required)* | Secret key for JWT token signing |
| `JWT_ALGORITHM` | `HS256` | JWT signing algorithm |
| `JWT_EXPIRY_HOURS` | `8` | Access token expiry in hours |
| `STORAGE_ROOT` | `/storage` | Root path for file storage |
| `LLM_MODEL` | `claude-sonnet-4-6` | Claude model to use for agents |
---
## Storage Layout
```
storage/amazon/
├── tm/ # Translation Memory files (JSONL)
│ ├── de-DE/
│ │ ├── flat_value_de-de.json # Value channel TM (~128 entries)
│ │ ├── flat_MASS_de-de.json # Mass channel TM
│ │ ├── flat_Onsite_de-de.json # Onsite channel TM
│ │ └── flat_Outbound_de-de.json # Outbound channel TM
│ ├── fr-FR/
│ │ └── ...
│ └── ... (12 locale directories)
│
└── ref/ # Reference files (JSON)
├── glossary/ # Locale-specific term glossaries
│ ├── de_DE_glossary.json
│ └── ...
├── blacklist/ # Forbidden terms per locale
│ ├── de_DE_blacklist.json
│ └── ...
├── tov_global/ # Global Tone of Voice guidelines
│ └── Amazon_TOV_Guidelines_for_Transcreation_290224.json
├── tov_supplement/ # Supplementary TOV (de-DE, de-AT)
│ └── DE_AT_TOV_Guidelines.json
├── locale_considerations/ # Locale-specific rules and notes
│ └── ...
└── date_pct_formats/ # Approved date/percentage formats
└── ...
```
### TM File Format (JSONL)
Each line is a JSON object. Two formats are supported:
**Compact format** (existing files):
```json
{"t": "Value Q1 24 Radio 001 VO de-de As Sophie opened the door... Als Sophie die Tuer oeffnete..."}
```
**Multi-field format**:
```json
{"seg_key": "Value Q1 24 Radio 001", "en": "As Sophie opened...", "lc": "de-de", "tx": "Als Sophie...", "nt": "VO", "channel": "value"}
```
---
## Supported Locales & Channels
### Locales (12)
| Code | Language | Notes |
|------|----------|-------|
| de-DE | German (Germany) | Shares TM/TOV supplement with de-AT |
| de-AT | German (Austria) | Shares TM/TOV supplement with de-DE |
| fr-FR | French (France) | Shares TM with fr-BE |
| fr-BE | French (Belgium) | Shares TM with fr-FR |
| es-ES | Spanish (Spain) | Shares TM with ca-ES |
| ca-ES | Catalan (Spain) | Enforced as Catalan, not Spanish |
| it-IT | Italian (Italy) | - |
| nl-NL | Dutch (Netherlands) | Independent from nl-BE |
| nl-BE | Dutch (Belgium) | Independent from nl-NL |
| pl-PL | Polish (Poland) | - |
| pt-PT | Portuguese (Portugal) | - |
| sv-SE | Swedish (Sweden) | - |
### Channels & TM Files
| Channel | TM File Pattern |
|---------|----------------|
| Value | `flat_value_{lc}.json` |
| Mass | `flat_MASS_{lc}.json` |
| Onsite | `flat_Onsite_{lc}.json` |
| Outbound | `flat_Outbound_{lc}.json` |
### Programmes & Voice Profiles
| Programme | Voice | Description |
|-----------|-------|-------------|
| Retail | Real, Clear, Playful, Witty | Everyday value messaging |
| Prime | Optimistic, Honest, Self-aware, Witty, Relatable | Prime membership benefits |
| Brand | Authentic, Customer-obsessed, Intelligent, Warm, Understated | Brand-level communications |
---
## API Reference
### Authentication
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/v1/auth/login` | Login (email + password) |
| POST | `/api/v1/auth/refresh` | Refresh access token |
| GET | `/api/v1/auth/me` | Get current user claims |
### Jobs
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/v1/jobs` | Create job |
| GET | `/api/v1/jobs` | List jobs (filterable) |
| GET | `/api/v1/jobs/{id}` | Get job detail + locale instances |
| DELETE | `/api/v1/jobs/{id}` | Delete job (admin only) |
| PUT | `/api/v1/jobs/{id}/source` | Upload source xlsx |
| POST | `/api/v1/jobs/{id}/supplementary` | Upload supplementary file |
| POST | `/api/v1/jobs/{id}/launch` | Launch processing |
| POST | `/api/v1/jobs/{id}/cancel` | Cancel job |
| POST | `/api/v1/jobs/{id}/locales/{code}/rerun` | Re-run single locale |
### Output & Feedback
| Method | Endpoint | Description |
|--------|----------|-------------|
| GET | `/api/v1/output/jobs/{id}/locales/{code}/preview` | Output preview |
| GET | `/api/v1/output/jobs/{id}/locales/{code}/export` | Download xlsx |
| POST | `/api/v1/output/feedback` | Submit feedback |
| GET | `/api/v1/output/feedback/{output_id}` | Get feedback |
### File Management
| Method | Endpoint | Description |
|--------|----------|-------------|
| POST | `/api/v1/files/tm` | Upload TM file |
| GET | `/api/v1/files/tm` | List TM files |
| DELETE | `/api/v1/files/tm/{id}` | Delete TM file |
| POST | `/api/v1/files/reference` | Upload reference file |
| GET | `/api/v1/files/reference` | List reference files |
| DELETE | `/api/v1/files/reference/{id}` | Delete reference file |
### Admin & Reports
| Method | Endpoint | Description |
|--------|----------|-------------|
| CRUD | `/api/v1/users` | User management (admin) |
| CRUD | `/api/v1/clients` | Client management (admin) |
| GET | `/api/v1/audit/logs` | Audit trail |
| GET | `/api/v1/reports/usage` | Usage statistics |
| GET | `/api/v1/reports/tokens` | Token cost breakdown |
| GET | `/api/v1/reports/quality` | Quality metrics |
---
## Database Schema
```
┌──────────────┐ ┌──────────────┐ ┌──────────────────┐
│ clients │ │ users │ │ user_clients │
│──────────────│ │──────────────│ │──────────────────│
│ id (PK) │◄────│ id (PK) │ │ user_id (FK) │
│ name │ │ email │ │ client_id (FK) │
│ settings │ │ name │ │ role_override │
└──────┬───────┘ │ password_hash│ └──────────────────┘
│ │ role (enum) │
│ │ status │
│ └──────┬───────┘
│ │
┌──────▼───────┐ │
│ jobs │ │
│──────────────│◄───────────┘ (created_by)
│ id (PK) │
│ client_id │ ┌──────────────────┐
│ campaign_name│ │ source_lines │
│ programme │ │──────────────────│
│ channel │ │ id (PK) │
│ status │◄────│ job_id (FK) │
│ job_type │ │ en_gb │
└──────┬───────┘ │ copy_type │
│ │ char_limit │
│ │ is_display_format│
┌──────▼───────────┐ └──────────────────┘
│ locale_instances │
│──────────────────│ ┌──────────────────┐
│ id (PK) │ │ output_rows │
│ job_id (FK) │ │──────────────────│
│ locale_code │ │ id (PK) │
│ status │◄─│ instance_id (FK) │
│ progress │ │ line_id (FK) │
│ current_stage │ │ confidence_tier │
│ token_usage │ │ option_1,2,3 │ ┌──────────────┐
│ started_at │ │ backtranslation │ │ feedback │
│ completed_at │ │ rationale │ │──────────────│
└──────────────────┘ │ char_counts │◄──│ output_id │
└──────────────────┘ │ user_id │
│ flag_type │
┌──────────────────┐ ┌──────────────────┐ │ comment │
│ tm_file_registry │ │ reference_files │ └──────────────┘
│──────────────────│ │──────────────────│
│ client_id │ │ client_id │ ┌──────────────┐
│ locale_code │ │ file_type │ │ audit_logs │
│ channel │ │ locale_scope │ │──────────────│
│ filename │ │ filename │ │ user_id │
│ segment_count │ │ file_path │ │ action │
└──────────────────┘ └──────────────────┘ │ entity_type │
│ details │
┌──────────────────┐ └──────────────┘
│ token_usage_logs │
│──────────────────│
│ instance_id │
│ agent_name │
│ input_tokens │
│ output_tokens │
│ estimated_cost │
└──────────────────┘
```
11 tables total. All primary keys are UUIDs. Cascading deletes from jobs down through locale_instances, output_rows, and source_lines.
---
## User Guide
### Creating a Job
1. Navigate to **Jobs > New Job**
2. Fill in the job details:
- **Client** - Select the client (e.g. Amazon)
- **Campaign Name** - Name of the campaign (e.g. "DDA 26 BFW")
- **Programme** - Retail, Prime, or Brand (determines voice profile)
- **Channel** - Value, Mass, Onsite, or Outbound (determines TM file)
- **Locales** - Select one or more target locales
3. Upload the **source xlsx** file with columns:
- `EN_GB` (required) - English source copy
- `Copy Type` - Type of copy (headline, body, CTA, script, etc.)
- `Creative Guidance` - Context or instructions for the transcreator
- `Visual Ref` - Reference to visual assets
- `Char Limit` - Maximum character count for the translation
4. Optionally add a **context/override prompt** with special instructions
5. Review the summary and click **Launch**
### Monitoring Progress
Once launched, the job monitoring page shows real-time updates:
- Per-locale progress bars (0-100%)
- Current stage: Loading Files > Matching TM > Ranking > Translating (batch X/Y) > Reviewing > Formatting > Complete
- Token usage and elapsed time
- Error details if any locale fails
Multiple locales process in **parallel** (up to 4 at once).
### Reviewing Output
Click **Preview** on a completed locale to open the review interface:
- Each source line shows its **confidence tier** (High / Moderate / Low)
- **High confidence**: 1 option anchored to a TM match
- **Moderate confidence**: 2 creative options
- **Low confidence**: 3 creative options
- Every option includes a **backtranslation** and **character count**
- Expandable **rationale** explains the translation choices and TM citations
- Feedback buttons: **Approve**, **Needs Revision**, or add a **Comment**
- **Export** button downloads the formatted xlsx
### Admin Features
Admins have access to additional pages:
- **User Management** - Create, edit, and deactivate users
- **Client Management** - Manage client configurations
- **TM Files** - Upload and manage Translation Memory files
- **Reference Files** - Manage glossaries, blacklists, TOV guidelines
- **Reports** - Usage statistics, token costs, quality metrics
- **Audit Logs** - Complete trail of all system actions
- **Delete Jobs** - Remove old jobs (with confirmation)
---
## Development
### Project Structure
```
amazon-transcreation/
├── backend/
│ ├── app/
│ │ ├── main.py # FastAPI app factory
│ │ ├── config.py # pydantic-settings env loader
│ │ ├── dependencies.py # DI: get_db, get_current_user
│ │ ├── auth/ # JWT auth (SSO-ready provider pattern)
│ │ ├── api/v1/ # REST endpoint routers
│ │ ├── models/ # SQLAlchemy models (11 tables)
│ │ ├── schemas/ # Pydantic request/response models
│ │ ├── services/ # Business logic layer
│ │ ├── pipeline/
│ │ │ ├── orchestrator.py # State machine (6 stages + retry)
│ │ │ ├── contracts.py # Inter-agent Pydantic models
│ │ │ ├── agents/ # 6 pipeline agents
│ │ │ └── modules/ # 9 deterministic modules
│ │ ├── tasks/ # Celery task definitions
│ │ ├── llm/ # Anthropic SDK wrapper + retry
│ │ └── ws/ # WebSocket handler + manager
│ ├── alembic/ # Database migrations
│ └── tests/
├── frontend/
│ └── src/
│ ├── app/ # Next.js App Router pages
│ ├── components/ # React UI components
│ ├── hooks/ # Custom React hooks
│ └── lib/ # API client, types, utilities
├── storage/ # Runtime file storage (mounted volume)
├── docker-compose.yml # Development services
├── docker-compose.prod.yml # Production services
├── deploy.sh # Server deployment script
├── Makefile # Dev convenience commands
└── .env.example # Environment variable template
```
### Running Tests
```bash
make test
# or
docker compose exec backend python -m pytest tests/ -v
```
### Adding a New Locale
1. Create TM files in `storage/amazon/tm/{locale_code}/`
2. Create reference files in the appropriate `storage/amazon/ref/` subdirectories
3. Add the locale to the frontend locale selector
4. Add the locale display name to `LOCALE_NAMES` in `agent_4_transcreator.py`
---
## Deployment
### Using deploy.sh
```bash
# First time setup (clones repo, builds, migrates, seeds)
./deploy.sh --init
# Regular updates (pulls code, rebuilds changed services, migrates)
./deploy.sh
# Full rebuild (recreates all containers from scratch)
./deploy.sh --rebuild
```
### Docker Services
| Service | Internal Port | External Port | Description |
|---------|--------------|---------------|-------------|
| PostgreSQL | 5432 | 5492 | Database |
| Redis | 6379 | 6389 | Task broker |
| Backend (FastAPI) | 8000 | 8040 | API server |
| Celery Worker | - | - | 4 concurrent task workers |
| Frontend (Next.js) | 3000 | 3000 | SSR app |
| Nginx (prod only) | 80/443 | 80/443 | Reverse proxy + SSL |
### Cost Estimation
For a typical 53-line source brief:
| | Per Locale | 12 Locales |
|---|-----------|------------|
| Agent 2 (TM Retrieval) | ~$0.13 | ~$1.56 |
| Agent 4 (Transcreation) | ~$0.36 | ~$4.32 |
| **Total** | **~$0.49** | **~$5.88** |
| Processing time | ~5.5 min | ~5.5 min (parallel) |
Reference in a new issue View git blame Copy permalink