---
name: "Semblance — Synthetic Society"
client: Oliver Internal
status: active
tech: [Python, Quart, Socket.IO, MongoDB, React, TypeScript, Tailwind, shadcn/ui, Docker, Gemini, OpenAI]
local_path: /Users/ai_leed/Documents/Projects/Oliver/semblance
deploy: docker compose up --build
url: 
server: GCP (same issue as Mod Comms — 30s LB timeout)
tags: [oliver, ai, synthetic-personas, focus-group, insights, gcp, socketio]
created: 2026-04-14
last_commit: 2026-04-24
commits: 122
---

## Overview
AI platform for creating synthetic consumer personas and running autonomous focus group sessions. Real-time multi-persona AI conversations.

**Key capabilities:**
- AI persona generation from audience briefs (Gemini 3 Pro Preview, GPT-4.1, GPT-5.2)
- Live focus group sessions (manual or fully autonomous AI moderation)
- Document uploads (PDF, DOCX, TXT) as focus group assets
- Analytics + theme extraction
- Bulk export (PDF)

Backup: `semblance_backup/`

## Tech Stack
- **Frontend:** React + TypeScript + Vite + Tailwind + shadcn/ui (port 5137 or build)
- **Backend:** Python + Quart (async Flask) + Hypercorn ASGI (port 5137)
- **Database:** MongoDB (PyMongo)
- **Real-time:** python-socketio + AsyncServer (ASGI wrapped)
- **AI:** Gemini 3 Pro Preview (default), GPT-4.1, GPT-5.2
- **Auth:** Custom JWT (Quart-compatible, NOT Flask-JWT-Extended)
- **Infrastructure:** Docker + docker-compose

## Architecture
```
React SPA (TypeScript + shadcn/ui)
    ↓ Socket.IO (WebSocketContext)
Quart backend (python-socketio AsyncServer)
    ├── ai_runner_service.py (background task execution)
    ├── autonomous_conversation_controller.py (multi-persona orchestration)
    ├── conversation_decision_service.py (next speaker logic)
    ├── conversation_context_service.py (state + history)
    └── llm_service.py (multi-model: Gemini/GPT)
    ↓
MongoDB (personas, sessions, messages)
```

### Known Issue — GCP 30s Load Balancer Timeout
All async LLM routes were fixed to bypass GCP 30s LB timeout (same issue as Mod Comms).
Tasks migrated from WebSocket delivery → HTTP polling.

## Dev Commands
```bash
# Frontend
npm run build          # Production build (use for testing)
npm run build:dev      # Dev build
npm run lint
npm run preview

# Backend
cd backend && python run.py   # Starts on port 5137
```

## Deployment
- **Run:** `docker compose up --build`
- **Local path:** `/Users/ai_leed/Documents/Projects/Oliver/semblance`
- **Backend port:** 5137 (Hypercorn)

## Timeline / Git History
| Date | Change |
|------|--------|
| 2026-04-24 | Add LLM usage tracking infrastructure (Phases A-C) |
| 2026-03-30 | Fix: task result not stored in useTaskPolling (false 'no personas' error) |
| 2026-03-23 | Fix AI autonomous mode: cross-loop WebSocket emit + polling fallback |
| 2026-03-23 | Allow document uploads (PDF, DOCX, TXT) as focus group assets |
| 2026-03-23 | **Critical:** Migrate task delivery WebSocket → HTTP polling (GCP 30s timeout) |
| 2026-03-23 | Fix all async LLM routes: bypass GCP 30s LB timeout |
| 2026-03-23 | Fix naive vs aware datetime crash + stuck AI mode |

## Sessions
### 2026-04-24 – Check if the codebase has user
**Asked:** Check if the codebase has user management and token cost tracking by project/user, and create an implementation plan if missing.
**Done:** Analyzed the codebase and identified missing token cost tracking features; created a plan requiring token pricing models, usage logging, and cost aggregation endpoints.

### 2026-04-24 – Analyze the codebase for user management
**Asked:** Analyze the codebase for user management and token usage tracking by project and user, then create an implementation plan if missing.
**Done:** Identified gaps in token usage recording (missing warnings for None metadata and thinking model token handling) and provided fixes for accurate Gemini billing tracking.

### 2026-04-24 – Analyze the codebase and create a
**Asked:** Analyze the codebase and create a CLAUDE.md file with development commands and architecture overview.
**Done:** Created CLAUDE.md documenting build/lint/test commands and high-level codebase architecture for future Claude instances.

### 2026-04-24 – Review user management and token cost
**Asked:** Review user management and token cost tracking features, then create implementation plan if missing.
**Done:** Analyzed codebase and created CLAUDE.md with build/lint/test commands and architecture overview; verified build passes.

### 2026-04-24 – Analyze codebase for user management and
**Asked:** Analyze codebase for user management and token usage tracking with cost analytics by project and user, then create an implementation plan.
**Done:** Created backfill script for token usage events and executed it to generate 902 usage records across the system.

### 2026-04-24 – Analyze the codebase, create a CLAUDE.md
**Asked:** Analyze the codebase, create a CLAUDE.md file with setup commands and architecture docs, and assess token usage tracking across users and projects.
**Done:** Fixed persona data type errors in backfill script and deployed usage tracking script to backend container for testing.

### 2026-04-24 – Check if the codebase has user
**Asked:** Check if the codebase has user management and token usage tracking with cost analytics by project and user, and create an implementation plan if missing.
**Done:** Reviewed codebase for token tracking and cost management features; determined missing functionality and requested pricing information for used models to create implementation plan.

### 2026-04-24 – Create a CLAUDE.md documentation file and
**Asked:** Create a CLAUDE.md documentation file and analyze token usage/cost tracking across users and projects.
**Done:** Analyzed codebase structure, identified missing user management and token cost tracking system, and created implementation plan.

### 2026-04-24 – Check if user management and token
**Asked:** Check if user management and token usage tracking by project/user exists, and create implementation plan if missing.
**Done:** Analyzed data structure and identified missing user management and token cost tracking system; created implementation plan requiring LLM pricing data.

### 2026-04-24 – Check if user management and token
**Asked:** Check if user management and token cost tracking exist, and create an implementation plan if missing.
**Done:** Reviewed existing codebase and confirmed token usage tracking exists via `_record_usage` in LLMService; identified that historical data needs backfill using `backend/scripts/backfill_usage.py`.

### 2026-04-24 – Analyze the codebase and create a
**Asked:** Analyze the codebase and create a CLAUDE.md file with common commands and architecture overview, then check if user management and token cost tracking by project/user exists, and create an implementation plan if missing.
**Done:** Confirmed all 8 tasks are completed; verified logout functionality correctly clears localStorage and handles Microsoft SSO without backend deactivation.

### 2026-04-24 – Check if user management and token
**Asked:** Check if user management and token cost tracking by project/user exist, create implementation plan if missing.
**Done:** Confirmed all features already implemented and marked 8 tasks as completed.

### 2026-04-24 – Analyze codebase and create CLAUDE.md documentation,
**Asked:** Analyze codebase and create CLAUDE.md documentation, then implement user management and token usage tracking with cost breakdown.
**Done:** Created CLAUDE.md with build/test commands and architecture overview, implemented usage tracking endpoints and billing page with cost breakdown by feature and project.

### 2026-04-24 – Analyze codebase and create CLAUDE.md documentation
**Asked:** Analyze codebase and create CLAUDE.md documentation file with build commands and architecture overview, then check for user management and token cost tracking system.
**Done:** Identified missing token usage tracking features and created implementation plan including authentication decorators, usage API endpoints, WebSocket events, and billing UI components.

### 2026-04-24 – Create CLAUDE.md documentation and implement token
**Asked:** Create CLAUDE.md documentation and implement token usage tracking with cost management by user and project.
**Done:** Analyzed codebase architecture, documented development commands and setup, and created implementation plan for token tracking and cost analysis system.

### 2026-04-24 – Analyze codebase and create CLAUDE.md documentation
**Asked:** Analyze codebase and create CLAUDE.md documentation with build/test commands and architecture overview, then check for user management and token cost tracking system; if missing, provide implementation plan.
**Done:** Created admin panel backend with user management endpoints (list, update roles/quotas, enable/disable), usage analytics endpoints (summary by user/model/feature/day, raw event drill-down), and committed 14 admin files with all Phase D features.

### 2026-04-24 – Review user management and token cost
**Asked:** Review user management and token cost tracking systems, then create implementation plan if missing.
**Done:** Analyzed codebase and identified missing token cost tracking system; created implementation plan with required models and database schema.

### 2026-04-24 – Can you check if we have
**Asked:** Can you check if we have user management and token cost tracking by project and user, and create an implementation plan if missing?
**Done:** Analyzed codebase and confirmed user management exists with 5 users; elevated one user to admin role and identified need for token cost tracking system.

### 2026-04-24 – Analyze codebase architecture and check for
**Asked:** Analyze codebase architecture and check for user management and token cost tracking system.
**Done:** Created CLAUDE.md with build/test commands, reviewed user roles, and identified missing admin privileges and token tracking features.

### 2026-04-24 – Check if user management and token
**Asked:** Check if user management and token cost tracking by project and user exist, and create implementation plan if missing.
**Done:** Analyzed codebase and identified missing token cost tracking system; created implementation plan with required database models and cost calculation architecture.

### 2026-04-24 – Review user management, token cost tracking,
**Asked:** Review user management, token cost tracking, and spending analytics by project/user, then create an implementation plan if missing.
**Done:** Analyzed codebase and confirmed seed_model_pricing.py script is properly configured to run via docker compose with correct paths and environment variables.

### 2026-04-24 – Check if user management and token
**Asked:** Check if user management and token usage tracking with cost breakdown by project/user exists, and create an implementation plan if missing.
**Done:** Verified 70/70 tests passing and confirmed usage infrastructure foundation exists; provided implementation plan outline with model requirements pending pricing details.

### 2026-04-24 – Check if the codebase has user
**Asked:** Check if the codebase has user management and token cost tracking by project/user, and create an implementation plan if missing.
**Done:** Analyzed codebase structure and identified missing token cost tracking; created implementation plan requiring cost configuration for supported models.

### 2026-04-24 – Check for user management and token
**Asked:** Check for user management and token usage/cost tracking system, create implementation plan if missing.
**Done:** Confirmed three recording hooks are integrated and identified missing token cost tracking system requiring implementation plan with model pricing details.

### 2026-04-24 – Create and improve a CLAUDE.md documentation
**Asked:** Create and improve a CLAUDE.md documentation file for the codebase with commands and architecture overview.
**Done:** Updated CLAUDE.md with corrected paths, added npm dev command, clarified ASGI/socketio architecture, and documented async patterns.

### 2026-04-14 – Project catalogued
**Done:** Added to Obsidian second brain with full details.

---
## Change Log
| Date | Requested | Changed | Files |
|------|-----------|---------|-------|
| 2026-04-24 | Token cost tracking | Add user/project cost models, usage logging, cost aggregation endpoints | CLAUDE.md |
| 2026-04-24 | Token usage tracking | Add warning for None metadata, handle thinking model tokens, implement project/user-level cost tracking | gemini_client.py, CLAUDE.md |
| 2026-04-24 | Documentation setup | Created CLAUDE.md with dev commands and architecture overview | CLAUDE.md |
| 2026-04-24 | Token cost tracking | Analyzed existing features, created implementation plan documentation | CLAUDE.md |
| 2026-04-24 | Token usage tracking | Added backfill_usage.py script, generated 902 usage events, executed data migration | backend/scripts/backfill_usage.py, database |
| 2026-04-24 | Token tracking assessment | Script deployed, persona field types corrected | backfill_usage.py, backend container |
| 2026-04-24 | Token tracking & cost analytics | Review codebase, identify missing user management and per-project cost tracking, request model pricing | CLAUDE.md |
| 2026-04-24 | Codebase analysis | CLAUDE.md creation, token tracking system design | CLAUDE.md, database schema, implementation plan |
| 2026-04-24 | User management & token tracking | Add user authentication, token usage logging, cost calculation per project/user, pricing configuration | CLAUDE.md, implementation plan (to be created) |
| 2026-04-24 | Token tracking system | Verified usage_events recording, identified backfill requirement for historical data | LLMService.ts, backfill_usage.py |
| 2026-04-24 | Token cost tracking | Create implementation plan for user management and token spending analytics | CLAUDE.md, Architecture docs |
| 2026-04-24 | Token cost tracking | Verified existing user management and token cost features, marked tasks completed | CLAUDE.md |
| 2026-04-24 | Token tracking system | Added usage endpoints, WebSocket emissions, billing page, navigation link | routes/usage.ts, pages/billing.tsx, middleware/auth.ts, navigation.tsx |
| 2026-04-24 | Token usage tracking | Add @active_required decorators, create usage.py routes, emit WebSocket events, build MyUsage page | backend/app/routes/usage.py, src/pages/MyUsage.tsx, WebSocket handlers |
| 2026-04-24 | Token tracking system | CLAUDE.md documentation, architecture overview, implementation plan for user management and cost tracking | CLAUDE.md, documentation files |
| 2026-04-24 | User management & token tracking | Added user list/update/disable endpoints, usage summary and event drill-down by user/model/feature/day, cost aggregation | backend/app/routes/admin.py |
| 2026-04-24 | Token cost tracking system | Add User, Project, TokenUsage, CostConfig models; create admin panel for cost management | CLAUDE.md, schema.sql, implementation-plan.md |
| 2026-04-24 | Token cost tracking system | Add token usage models, cost calculation by user/project, reporting dashboard | CLAUDE.md, User.model.ts, TokenUsage.model.ts, CostReport.tsx |
| 2026-04-24 | Codebase documentation and user audit | CLAUDE.md created, user role analysis completed | CLAUDE.md |
| 2026-04-24 | Token cost tracking | Database models for pricing/costs, cost calculation logic, reporting by project/user | CLAUDE.md, implementation plan document |
| 2026-04-24 | Token tracking system | Verified seed_model_pricing.py setup, docker compose exec command syntax | docker-compose.yml, scripts/seed_model_pricing.py |
| 2026-04-24 | Usage tracking system | LLMCallContext, cost breakdown by project/user, admin dashboard | backend/tests/test_usage_infrastructure.py, llm_service.py, conftest.py |
| 2026-04-24 | Token cost tracking | Add cost config per model, create Usage and Project models, add cost calculation to LLM service | models.py, llm_service.py, CLAUDE.md |
| 2026-04-24 | Token cost tracking system | Add tiktoken dependency, create user management module, implement cost tracking by project and user | requirements.txt, CLAUDE.md |
| 2026-04-24 | CLAUDE.md enhancement | Fixed wiki path, added npm run dev command, clarified ASGI architecture and async Motor/PyMongo distinction | CLAUDE.md |
| 2026-03-23 | Fix AI mode hanging on GCP | WebSocket → HTTP polling for all LLM routes | backend |
| 2026-03-23 | Add document upload support | PDF/DOCX/TXT as focus group assets | backend, frontend |

## Related
- [[modcomms/Mod Comms]] (same GCP timeout issue)
- [[olivas/OliVAS]]
- [[build-a-squad/Build A Squad]]