Commit graph

44 commits

Author SHA1 Message Date
Vadym Samoilenko
ad619d45fc Improve live token extraction: warn on missing usage_metadata, capture thinking tokens
- Add WARNING log when usage_metadata/usage is None so zero-cost events
  are visible in logs instead of silently disappearing
- Capture thoughts_token_count from Gemini thinking models into reasoning field
  (already included in candidates_token_count for billing, now also tracked separately)
- Add same warning for OpenAI missing usage object

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 19:13:39 +01:00
Vadym Samoilenko
bc4138f332 Final pieces: decorators on LLM routes, usage self-service, billing page, WS events
Backend:
- @active_required + @with_user_context applied to all LLM-invoking routes
  in personas.py, focus_group_ai.py, ai_personas.py
- backend/app/routes/usage.py: GET /api/usage/me (MTD summary by feature),
  GET /api/usage/focus-groups/<id> (owner or admin)
- Registered usage_bp in app/__init__.py
- llm_service._record_usage now emits usage_update WS event to focus group room

Frontend:
- useMyUsage + useFocusGroupUsage hooks
- MyUsage.tsx: personal billing dashboard (cost cards + per-feature table)
- /billing route (ProtectedRoute) + Billing nav link
- FocusGroupSession: quota_warning amber banner with Progress bar,
  quota_exceeded + quota_warning WS events wired via websocketServiceNew

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:43:13 +01:00
Vadym Samoilenko
915c81b8f1 Complete phases D–G: quota enforcement, token invalidation, admin writes, backfill
Backend:
- token_version in JWT (bump_token_version, get_token_version on User model);
  jwt_required checks tv claim → 401 on mismatch; login routes embed version
- Quota pre-flight in all 3 LLM public methods (QuotaExceededError bubbles up)
- AI runner catches QuotaExceededError → sets status paused_quota + emits WS event
- Admin routes: POST /users (create), POST /users/<id>/reset-password,
  POST /pricing, GET /focus-groups with aggregated cost; PUT /users/<id>
  now bumps token_version on disable or role change
- backfill_usage.py: idempotent estimated-event generator for historical data,
  tiktoken for GPT models, char/3.8 for Gemini, --dry-run flag

Frontend:
- 402 interceptor dispatches quota_exceeded CustomEvent
- adminApi: createUser, resetPassword, createPricing, listFocusGroups
- UsersTab: New User dialog + Reset Password in edit dialog
- PricingTab: New Price dialog (model, provider, input/output/cached prices)
- FocusGroupsTab: focus groups table sorted by total cost
- Admin.tsx: 4th tab (Focus Groups)
- FocusGroupSession: admin-only cost badge + dismissable quota exceeded banner

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:34:48 +01:00
Vadym Samoilenko
3e9ccafad2 Add LLM usage tracking infrastructure (Phases A-C)
- Model renames: gpt-5.2 → gpt-5.4-2026-03-05, gemini-3-pro-preview → gemini-3.1-pro-preview; retire gpt-4.1 via alias fallback
- New: llm_usage_context.py (ContextVar-based attribution), model_pricing.py (tiered pricing + 60s cache), usage_event.py (append-only telemetry), quota.py (user/FG quota enforcement with 80% warning)
- Wire _record_usage into all 3 LLM methods; set_llm_context at every service entry point
- Fix admin_required decorator (was sync, never awaited User.find_by_id); add active_required and with_user_context decorators
- Inject user_id into ContextVar from JWT on every authenticated request
- Add DB indexes for usage_events, model_pricing, users collections
- Seed script for model pricing (gpt-5.4 single-tier, gemini-3.1 two-tier 200k threshold)
- Fix parse_json_response NameError (logger undefined at module level)
- 70 passing tests: conftest.py with sys.modules stubs, test_usage_infrastructure.py (52 tests), rewrite stale test_llm_service.py (18 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:08:27 +01:00
Vadym Samoilenko
2e85fc1acc Fix root cause: naive vs aware datetime crash + stuck AI mode indicator
The autonomous loop was crashing on every decision with:
  TypeError: can't subtract offset-naive and offset-aware datetimes
because MongoDB stores created_at without timezone info but the code
compared it against datetime.now(timezone.utc).

- conversation_context_service: make created_at timezone-aware before
  subtraction (replace tzinfo=utc when naive)
- DiscussionPanel: fix sync effect — when server reports AI mode is
  inactive, always clear localAiModeActive regardless of its value,
  so the "AI is generating..." spinner doesn't get stuck when the
  backend fails/stops before the frontend has confirmed AI mode started

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 19:30:04 +00:00
Vadym Samoilenko
7e72d07329 Fix AI loop hanging: add asyncio.wait_for timeouts on LLM calls
The autonomous conversation loop could hang indefinitely because
self.response_timeout=30 was defined but never used in wait_for().

- autonomous_conversation_controller: wrap generate_persona_response()
  with asyncio.wait_for(timeout=120s); 30s was too short for production
  LLMs, raised to 120s; TimeoutError returns an error dict so the loop
  can continue or count toward consecutive_silence limit
- conversation_decision_service: add asyncio.wait_for(timeout=60s)
  around LLMService.generate_content() for the decision call; add
  asyncio import and explicit TimeoutError handling

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 19:17:36 +00:00
Vadym Samoilenko
33272cc677 Allow document uploads (PDF, DOCX, TXT, etc.) as focus group assets
- Expand allowed file types from images-only to also include:
  PDF, DOCX, DOC, TXT, MD, CSV, XLSX, XLS, PPTX, PPT, RTF
- validate_asset_file: skip PIL validation for non-image files; 50MB limit for docs / 10MB for images
- Correct MIME type detection for document extensions
- Store asset_type: "document"|"image" in metadata
- ImageDescriptionService: text files → LLM summary; binary docs → label; images → existing multimodal flow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 17:08:30 +00:00
Vadym Samoilenko
1b387daacf Migrate task result delivery from WebSocket to HTTP polling
Backend:
- task_manager.py: add result/error/completed_at storage, TTL sweeper (5min), store_task_result() helper
- tasks.py: add GET /<task_id> endpoint returning stored result; cancel route stores 'cancelled' status
- __init__.py: start TTL sweeper on app startup
- All 8 bg functions: store result before emitting lightweight WS hint (no payload data)

Frontend:
- src/lib/taskPolling.ts: waitForTaskResult() — polls GET /tasks/{id} every 2s, WS hint triggers immediate poll, 5min timeout
- src/hooks/useTaskPolling.ts: drop-in replacement for useCancellableGeneration using polling
- Migrate 6 Promise-based WS listeners → waitForTaskResult() in DiscussionPanel, FocusGroupSession (×2), PersonaProfile, PersonaModificationModal, useDiscussionGuideGeneration
- Migrate 3 hook-based consumers → useTaskPolling in AIRecruiter, SyntheticUsers, BulkExportProgressModal

Fixes WS Promise leak: polling survives disconnects, background tabs, page reloads.
WS events retained as zero-payload hints for near-zero latency when connected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 16:46:58 +00:00
Vadym Samoilenko
c00728f375 Fix Gemini LLM AssertionError: force httpx transport over aiohttp
google-genai SDK uses aiohttp when it's available in the environment
(installed via llama-index-core), causing AssertionError (connector is None)
on async requests. Pass httpx_async_client in HttpOptions to bypass aiohttp.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 13:05:27 +00:00
Vadym Samoilenko
3e1865edbd Apply Jintech security audit remediation (sprint 3) — 87/92 findings fixed
- Fix missing await on FocusGroup.get_messages() (N-L1)
- Replace time.sleep with asyncio.sleep in key_theme_service and focus_group_service (N-P10)
- Replace flask import with quart in focus_groups.py (N-S3)
- Add logger.error before all 500 returns in focus_groups.py (N-P6)
- Add logging to silent except blocks across routes (N-M10, N-M11)
- Add @rate_limit to 6 remaining AI endpoints (N-H4)
- Add --confirm flag to populate scripts before delete_many (S-H2)
- Remove hardcoded Azure ID fallbacks from msal_service.py and msalConfig.ts (A-M2, F-H4)
- Centralize make_serializable() in utils.py, remove duplicates from 3 route files (N-P7)
- Replace all datetime.utcnow() with datetime.now(timezone.utc) across entire backend (M-L2)
- AuthContext.tsx: only mark token validated on 200 success, not on non-401 errors (F-H2)
- Rename authType → auth_type in auth.py (N-S4)
- Add security_report.md and security_report.pdf with full 92-finding status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 12:51:18 +00:00
michael
b1be8f8c38 Add model alias for legacy gpt-5 database entries
Focus groups created before the gpt-5.2 rename have llm_model='gpt-5'
stored in MongoDB. Without an alias, the backend falls through to the
Gemini provider and fails with an aiohttp AssertionError.

Adds MODEL_ALIASES mapping and _resolve_model() helper so gpt-5 is
transparently resolved to gpt-5.2. Also updates all llm_model checks
to accept both values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:06:10 -06:00
michael
b0445de18b Update GPT-5 to GPT-5.2 and lower default reasoning effort to low
Swap model ID from gpt-5 to gpt-5.2 across all backend services,
frontend components, and documentation. Change default reasoning
effort from medium to low for faster responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:24:01 -06:00
michael
d4ce5d99bc Fix audience_brief and research_objective dropped in Stage 2 persona generation
Stage 2 (detailed persona generation) was ignoring the audience brief and
research objective, causing the LLM to guess research context from demographics
alone. Now passes both values through to generate_persona() in all three
endpoints (generate-personas-full, complete-and-save-persona, complete-persona)
and auto-generates prompt customization via customize_persona_prompt() when
they are provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:40:17 -06:00
michael
850cb25067 Fix GPT-5 Responses API crash on reasoning items with None content
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:05:52 -06:00
michael
1708bd75a4 Remove SDK version logging on every Gemini call
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:30:42 -06:00
michael
d79c202e8f Fix NameError: use print instead of logger in get_gemini_client
logger is not defined at module level where get_gemini_client() lives.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:59:30 -06:00
michael
c9848210bb Add full traceback and SDK version logging for AssertionError debug
This will help identify where exactly the AssertionError is occurring
in the google-genai SDK and what version is installed on the server.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:49:47 -06:00
michael
5a825934f8 Add verbose exception debugging for empty error messages
Log full exception details: type, module, str, repr, args, and __dict__
to diagnose why Gemini errors are producing empty messages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:39:15 -06:00
michael
b50c0fa2a4 Fix empty error messages from Google GenAI SDK
Catch genai_errors.APIError specifically and extract e.code and e.message
attributes for proper error logging. The generic str(e) was returning empty
strings for Google API errors, making debugging impossible.

- Import google.genai.errors for specific exception handling
- Add APIError catch before generic Exception in generate_content()
- Add APIError catch before generic Exception in generate_contextual_response()
- Properly categorize errors by HTTP code for retry logic (429/500+ retryable)
- Fix time.sleep to await asyncio.sleep in contextual response handler

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:26:55 -06:00
michael
6ee80e67aa Create fresh LLM clients per call instead of caching
The previous event loop tracking approach still caused issues - when replacing
a cached client, its garbage collection triggers aclose() which tries to close
the aiohttp session on the wrong event loop.

Simplest fix: create a fresh client for each call. The overhead is minimal
compared to the actual LLM API call, and this completely avoids all event
loop mismatch issues in ASGI environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 16:56:36 -06:00
michael
94f98b837b Add event loop tracking to LLM client getters
The previous lazy initialization fix wasn't sufficient - the genai.Client
internally caches async structures bound to the event loop at creation time.
With ASGI servers like Hypercorn, subsequent requests may come on different
event loop contexts, causing "Future attached to a different loop" errors.

Now tracks which event loop the client was created on and recreates it if
the loop has changed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 16:46:35 -06:00
michael
36e1752fea Remove __pycache__ files from git tracking
These files are already in .gitignore but were committed previously.
Removing them from tracking to prevent future conflicts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 13:28:19 -06:00
Michael Clervi
893b537b67 changed permissions 2025-12-19 19:26:16 +00:00
michael
45c7a52aeb Fix asyncio event loop mismatch in LLM service
The genai.Client and AsyncOpenAI clients were being created at module
import time, before the Quart/Hypercorn event loop existed. This caused
"Future attached to a different loop" errors when async calls were made,
resulting in autonomous focus group conversations stopping with
"excessive_silence".

Changed to lazy initialization - clients are now created on first use
within the running event loop context via get_gemini_client() and
get_openai_client() helper functions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 13:24:23 -06:00
michael
96b9bfeedd Add LLM-generated one-line summaries for focus groups in list view
- Create focus_group_summary_service.py to generate concise summaries
- Add prompt template for summary generation
- Integrate summary generation after discussion guide creation
- Display summary under focus group title in list view with fallback to description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:51:28 -06:00
michael
25fcbc826f Redesign Enhance Brief to auto-apply changes with assumptions modal
- Replace suggestions panel with auto-apply: enhanced text now populates form fields automatically
- Add modal showing assumptions made during enhancement (3-5 bullet points)
- Reduce prompt scope by 50% for more focused, impactful enhancements
- Update backend to return enhanced text + assumptions instead of suggestions array

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 15:57:11 -06:00
michael
4d9b0afde7 upgraded to Gemini 3.0 Pro (gemini-3-pro-preview) from Gemini 2.5 Pro
- Upgraded google-genai package from 1.31.0 to 1.52.0
- Updated DEFAULT_MODEL in llm_service.py to gemini-3-pro-preview
- Updated all backend routes, services, and models with new model string
- Updated all frontend components with new model string and display labels
- Updated CLAUDE.md documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 10:37:36 -06:00
michael
d3aca7c309 added retry logic and smart field completion for persona generation, increased OpenAI timeout
- persona generation now retries up to 2 times on failure
- missing persona fields are intelligently completed based on context
- expanded required field validation from 5 to 9 fields
- prompt now explicitly lists all required fields with validation instructions
- fixed discussion guide cancellation check to handle object responses

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 09:39:58 -06:00
michael
f1cc094318 added full persona profile export in bulk actions menu (CSV, JSON and Markdown formats) 2025-09-10 22:38:35 -05:00
michael
e29d2a0bb9 made long actions cancellable (like persona generation, etc.), increased variety of persona generation with prompt changes and temperature variable, reduced length of key theme quotes, bug fixes 2025-09-10 16:24:05 -05:00
michael
8288cb9f5e fixed age to be a single number instead of range, fixed language for AI model to include thematic analysis, and added review/revert/save function to AI persona modification mechanism 2025-09-08 16:10:03 -05:00
michael
6a40936508 major refactor of entire application - migrate sync -> async including pymongo -> motor, flask -> quart, google-generativeai -> google-genai 2025-08-27 15:20:56 -05:00
michael
065c368aeb added persona modification service - users can adjust individual personas with NLP via LLM 2025-08-26 12:45:15 -05:00
michael
fbef4f42f6 added drag and drop for reordering questions/activities in discussion guide edit mode 2025-08-26 09:52:24 -05:00
michael
3e73259ff3 refactored file uploader and bug fixes 2025-08-13 11:32:56 -05:00
michael
8a5c50cacb refactored image/asset attachment to discussion guide and discussion messages to avoid back end filenames being displayed to user - use metadata system instead. Also added a detailed persona export to markdown. Also, bug fixes 2025-08-12 15:43:34 -05:00
michael
1c6e2e62e8 bug fixes related to websockets implementation 2025-08-11 09:33:09 -05:00
michael
9a3baa9d97 converted message polling to websockets 2025-08-10 18:08:34 -05:00
michael
da8639aee8 fixed folders again, bug fixes for gpt-5, adjusted response length calculation, cosmetic UI changes, other bug fixes 2025-08-09 10:08:45 -05:00
michael
fbb444037a fixed folders to be database instead of local storage based, implemented gpt-5, fixed key theme export quotes 2025-08-09 06:38:49 -05:00
michael
3c9518e3ec various bug fixes and UI tweaks 2025-08-07 16:34:37 -05:00
michael
8dcbe7efee various bug fixes and feature additions per Alec's requests 2025-08-06 19:29:27 -05:00
michael
b649793013 added gpt-4.1 support among other things 2025-08-05 17:38:13 -05:00
michael
da7b2c0448 initial commit 2025-08-04 09:07:59 -05:00