obsidian/01 Projects/semblance/Semblance.md
2026-04-27 18:04:42 +01:00

16 KiB
Raw Blame History

name client status tech local_path deploy url server tags created last_commit commits
Semblance — Synthetic Society Oliver Internal active
Python
Quart
Socket.IO
MongoDB
React
TypeScript
Tailwind
shadcn/ui
Docker
Gemini
OpenAI
/Users/ai_leed/Documents/Projects/Oliver/semblance docker compose up --build optical-web-1
oliver
ai
synthetic-personas
focus-group
insights
gcp
socketio
2026-04-14 2026-04-24 122

Overview

AI platform for creating synthetic consumer personas and running autonomous focus group sessions. Real-time multi-persona AI conversations.

Key capabilities:

  • AI persona generation from audience briefs (Gemini 3 Pro Preview, GPT-4.1, GPT-5.2)
  • Live focus group sessions (manual or fully autonomous AI moderation)
  • Document uploads (PDF, DOCX, TXT) as focus group assets
  • Analytics + theme extraction
  • Bulk export (PDF)

Backup: semblance_backup/

Tech Stack

  • Frontend: React + TypeScript + Vite + Tailwind + shadcn/ui (port 5137 or build)
  • Backend: Python + Quart (async Flask) + Hypercorn ASGI (port 5137)
  • Database: MongoDB (PyMongo)
  • Real-time: python-socketio + AsyncServer (ASGI wrapped)
  • AI: Gemini 3 Pro Preview (default), GPT-4.1, GPT-5.2
  • Auth: Custom JWT (Quart-compatible, NOT Flask-JWT-Extended)
  • Infrastructure: Docker + docker-compose

Architecture

React SPA (TypeScript + shadcn/ui)
    ↓ Socket.IO (WebSocketContext)
Quart backend (python-socketio AsyncServer)
    ├── ai_runner_service.py (background task execution)
    ├── autonomous_conversation_controller.py (multi-persona orchestration)
    ├── conversation_decision_service.py (next speaker logic)
    ├── conversation_context_service.py (state + history)
    └── llm_service.py (multi-model: Gemini/GPT)
    ↓
MongoDB (personas, sessions, messages)

Known Issue — GCP 30s Load Balancer Timeout

All async LLM routes were fixed to bypass GCP 30s LB timeout (same issue as Mod Comms). Tasks migrated from WebSocket delivery → HTTP polling.

Dev Commands

# Frontend
npm run build          # Production build (use for testing)
npm run build:dev      # Dev build
npm run lint
npm run preview

# Backend
cd backend && python run.py   # Starts on port 5137

Deployment

  • Run: docker compose up --build
  • Local path: /Users/ai_leed/Documents/Projects/Oliver/semblance
  • Backend port: 5137 (Hypercorn)

Timeline / Git History

Date Change
2026-04-24 Add LLM usage tracking infrastructure (Phases A-C)
2026-03-30 Fix: task result not stored in useTaskPolling (false 'no personas' error)
2026-03-23 Fix AI autonomous mode: cross-loop WebSocket emit + polling fallback
2026-03-23 Allow document uploads (PDF, DOCX, TXT) as focus group assets
2026-03-23 Critical: Migrate task delivery WebSocket → HTTP polling (GCP 30s timeout)
2026-03-23 Fix all async LLM routes: bypass GCP 30s LB timeout
2026-03-23 Fix naive vs aware datetime crash + stuck AI mode

Sessions

2026-04-24 Check if the codebase has user

Asked: Check if the codebase has user management and token cost tracking by project/user, and create an implementation plan if missing. Done: Analyzed the codebase and identified missing token cost tracking features; created a plan requiring token pricing models, usage logging, and cost aggregation endpoints.

2026-04-24 Analyze the codebase for user management

Asked: Analyze the codebase for user management and token usage tracking by project and user, then create an implementation plan if missing. Done: Identified gaps in token usage recording (missing warnings for None metadata and thinking model token handling) and provided fixes for accurate Gemini billing tracking.

2026-04-24 Analyze the codebase and create a

Asked: Analyze the codebase and create a CLAUDE.md file with development commands and architecture overview. Done: Created CLAUDE.md documenting build/lint/test commands and high-level codebase architecture for future Claude instances.

2026-04-24 Review user management and token cost

Asked: Review user management and token cost tracking features, then create implementation plan if missing. Done: Analyzed codebase and created CLAUDE.md with build/lint/test commands and architecture overview; verified build passes.

2026-04-24 Analyze codebase for user management and

Asked: Analyze codebase for user management and token usage tracking with cost analytics by project and user, then create an implementation plan. Done: Created backfill script for token usage events and executed it to generate 902 usage records across the system.

2026-04-24 Analyze the codebase, create a CLAUDE.md

Asked: Analyze the codebase, create a CLAUDE.md file with setup commands and architecture docs, and assess token usage tracking across users and projects. Done: Fixed persona data type errors in backfill script and deployed usage tracking script to backend container for testing.

2026-04-24 Check if the codebase has user

Asked: Check if the codebase has user management and token usage tracking with cost analytics by project and user, and create an implementation plan if missing. Done: Reviewed codebase for token tracking and cost management features; determined missing functionality and requested pricing information for used models to create implementation plan.

2026-04-24 Create a CLAUDE.md documentation file and

Asked: Create a CLAUDE.md documentation file and analyze token usage/cost tracking across users and projects. Done: Analyzed codebase structure, identified missing user management and token cost tracking system, and created implementation plan.

2026-04-24 Check if user management and token

Asked: Check if user management and token usage tracking by project/user exists, and create implementation plan if missing. Done: Analyzed data structure and identified missing user management and token cost tracking system; created implementation plan requiring LLM pricing data.

2026-04-24 Check if user management and token

Asked: Check if user management and token cost tracking exist, and create an implementation plan if missing. Done: Reviewed existing codebase and confirmed token usage tracking exists via _record_usage in LLMService; identified that historical data needs backfill using backend/scripts/backfill_usage.py.

2026-04-24 Analyze the codebase and create a

Asked: Analyze the codebase and create a CLAUDE.md file with common commands and architecture overview, then check if user management and token cost tracking by project/user exists, and create an implementation plan if missing. Done: Confirmed all 8 tasks are completed; verified logout functionality correctly clears localStorage and handles Microsoft SSO without backend deactivation.

2026-04-24 Check if user management and token

Asked: Check if user management and token cost tracking by project/user exist, create implementation plan if missing. Done: Confirmed all features already implemented and marked 8 tasks as completed.

2026-04-24 Analyze codebase and create CLAUDE.md documentation,

Asked: Analyze codebase and create CLAUDE.md documentation, then implement user management and token usage tracking with cost breakdown. Done: Created CLAUDE.md with build/test commands and architecture overview, implemented usage tracking endpoints and billing page with cost breakdown by feature and project.

2026-04-24 Analyze codebase and create CLAUDE.md documentation

Asked: Analyze codebase and create CLAUDE.md documentation file with build commands and architecture overview, then check for user management and token cost tracking system. Done: Identified missing token usage tracking features and created implementation plan including authentication decorators, usage API endpoints, WebSocket events, and billing UI components.

2026-04-24 Create CLAUDE.md documentation and implement token

Asked: Create CLAUDE.md documentation and implement token usage tracking with cost management by user and project. Done: Analyzed codebase architecture, documented development commands and setup, and created implementation plan for token tracking and cost analysis system.

2026-04-24 Analyze codebase and create CLAUDE.md documentation

Asked: Analyze codebase and create CLAUDE.md documentation with build/test commands and architecture overview, then check for user management and token cost tracking system; if missing, provide implementation plan. Done: Created admin panel backend with user management endpoints (list, update roles/quotas, enable/disable), usage analytics endpoints (summary by user/model/feature/day, raw event drill-down), and committed 14 admin files with all Phase D features.

2026-04-24 Review user management and token cost

Asked: Review user management and token cost tracking systems, then create implementation plan if missing. Done: Analyzed codebase and identified missing token cost tracking system; created implementation plan with required models and database schema.

2026-04-24 Can you check if we have

Asked: Can you check if we have user management and token cost tracking by project and user, and create an implementation plan if missing? Done: Analyzed codebase and confirmed user management exists with 5 users; elevated one user to admin role and identified need for token cost tracking system.

2026-04-24 Analyze codebase architecture and check for

Asked: Analyze codebase architecture and check for user management and token cost tracking system. Done: Created CLAUDE.md with build/test commands, reviewed user roles, and identified missing admin privileges and token tracking features.

2026-04-24 Check if user management and token

Asked: Check if user management and token cost tracking by project and user exist, and create implementation plan if missing. Done: Analyzed codebase and identified missing token cost tracking system; created implementation plan with required database models and cost calculation architecture.

2026-04-24 Review user management, token cost tracking,

Asked: Review user management, token cost tracking, and spending analytics by project/user, then create an implementation plan if missing. Done: Analyzed codebase and confirmed seed_model_pricing.py script is properly configured to run via docker compose with correct paths and environment variables.

2026-04-24 Check if user management and token

Asked: Check if user management and token usage tracking with cost breakdown by project/user exists, and create an implementation plan if missing. Done: Verified 70/70 tests passing and confirmed usage infrastructure foundation exists; provided implementation plan outline with model requirements pending pricing details.

2026-04-24 Check if the codebase has user

Asked: Check if the codebase has user management and token cost tracking by project/user, and create an implementation plan if missing. Done: Analyzed codebase structure and identified missing token cost tracking; created implementation plan requiring cost configuration for supported models.

2026-04-24 Check for user management and token

Asked: Check for user management and token usage/cost tracking system, create implementation plan if missing. Done: Confirmed three recording hooks are integrated and identified missing token cost tracking system requiring implementation plan with model pricing details.

2026-04-24 Create and improve a CLAUDE.md documentation

Asked: Create and improve a CLAUDE.md documentation file for the codebase with commands and architecture overview. Done: Updated CLAUDE.md with corrected paths, added npm dev command, clarified ASGI/socketio architecture, and documented async patterns.

2026-04-14 Project catalogued

Done: Added to Obsidian second brain with full details.


Change Log

Date Requested Changed Files
2026-04-24 Token cost tracking Add user/project cost models, usage logging, cost aggregation endpoints CLAUDE.md
2026-04-24 Token usage tracking Add warning for None metadata, handle thinking model tokens, implement project/user-level cost tracking gemini_client.py, CLAUDE.md
2026-04-24 Documentation setup Created CLAUDE.md with dev commands and architecture overview CLAUDE.md
2026-04-24 Token cost tracking Analyzed existing features, created implementation plan documentation CLAUDE.md
2026-04-24 Token usage tracking Added backfill_usage.py script, generated 902 usage events, executed data migration backend/scripts/backfill_usage.py, database
2026-04-24 Token tracking assessment Script deployed, persona field types corrected backfill_usage.py, backend container
2026-04-24 Token tracking & cost analytics Review codebase, identify missing user management and per-project cost tracking, request model pricing CLAUDE.md
2026-04-24 Codebase analysis CLAUDE.md creation, token tracking system design CLAUDE.md, database schema, implementation plan
2026-04-24 User management & token tracking Add user authentication, token usage logging, cost calculation per project/user, pricing configuration CLAUDE.md, implementation plan (to be created)
2026-04-24 Token tracking system Verified usage_events recording, identified backfill requirement for historical data LLMService.ts, backfill_usage.py
2026-04-24 Token cost tracking Create implementation plan for user management and token spending analytics CLAUDE.md, Architecture docs
2026-04-24 Token cost tracking Verified existing user management and token cost features, marked tasks completed CLAUDE.md
2026-04-24 Token tracking system Added usage endpoints, WebSocket emissions, billing page, navigation link routes/usage.ts, pages/billing.tsx, middleware/auth.ts, navigation.tsx
2026-04-24 Token usage tracking Add @active_required decorators, create usage.py routes, emit WebSocket events, build MyUsage page backend/app/routes/usage.py, src/pages/MyUsage.tsx, WebSocket handlers
2026-04-24 Token tracking system CLAUDE.md documentation, architecture overview, implementation plan for user management and cost tracking CLAUDE.md, documentation files
2026-04-24 User management & token tracking Added user list/update/disable endpoints, usage summary and event drill-down by user/model/feature/day, cost aggregation backend/app/routes/admin.py
2026-04-24 Token cost tracking system Add User, Project, TokenUsage, CostConfig models; create admin panel for cost management CLAUDE.md, schema.sql, implementation-plan.md
2026-04-24 Token cost tracking system Add token usage models, cost calculation by user/project, reporting dashboard CLAUDE.md, User.model.ts, TokenUsage.model.ts, CostReport.tsx
2026-04-24 Codebase documentation and user audit CLAUDE.md created, user role analysis completed CLAUDE.md
2026-04-24 Token cost tracking Database models for pricing/costs, cost calculation logic, reporting by project/user CLAUDE.md, implementation plan document
2026-04-24 Token tracking system Verified seed_model_pricing.py setup, docker compose exec command syntax docker-compose.yml, scripts/seed_model_pricing.py
2026-04-24 Usage tracking system LLMCallContext, cost breakdown by project/user, admin dashboard backend/tests/test_usage_infrastructure.py, llm_service.py, conftest.py
2026-04-24 Token cost tracking Add cost config per model, create Usage and Project models, add cost calculation to LLM service models.py, llm_service.py, CLAUDE.md
2026-04-24 Token cost tracking system Add tiktoken dependency, create user management module, implement cost tracking by project and user requirements.txt, CLAUDE.md
2026-04-24 CLAUDE.md enhancement Fixed wiki path, added npm run dev command, clarified ASGI architecture and async Motor/PyMongo distinction CLAUDE.md
2026-03-23 Fix AI mode hanging on GCP WebSocket → HTTP polling for all LLM routes backend
2026-03-23 Add document upload support PDF/DOCX/TXT as focus group assets backend, frontend