Commit graph

44 commits

Author SHA1 Message Date
Vadym Samoilenko
7b6a7c7347 Fix admin filters: ISO Z parsing crash + All time period returning month data
Two bugs caused filters to show 0 and period selector to have no effect:

1. Python < 3.11 can't parse JS toISOString() Z suffix — every request with a
   period filter threw ValueError → 500 → frontend received no data. Fixed with
   _parse_iso() helper that replaces Z with +00:00 before fromisoformat().

2. 'All time' sends no from/to params, but backend defaulted to _month_start()
   instead of omitting the ts filter. Fixed with _period_match() helper that
   returns {} (no filter) when both from and to are absent.

Also: stale _user_mtd_cost reference in get_user route replaced with
_user_period_cost(user_id, None, None); adminApi types updated with from/to.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 19:17:57 +01:00
Vadym Samoilenko
57508e8e55 Add period selector to all cost-bearing admin tabs
- New usePeriod hook (day/week/month/all/custom presets) with from/to ISO string outputs
- New PeriodSelector component (button group + custom date inputs)
- UsersTab, UsageTab, FocusGroupsTab all wired up with period state
- Backend /admin/users and /admin/focus-groups now accept from/to query params
- MTD Cost column header now reflects selected period label (e.g. "Cost (MTD)")
- Logout clears local state only (no account sign-out)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 19:03:16 +01:00
Vadym Samoilenko
bc4138f332 Final pieces: decorators on LLM routes, usage self-service, billing page, WS events
Backend:
- @active_required + @with_user_context applied to all LLM-invoking routes
  in personas.py, focus_group_ai.py, ai_personas.py
- backend/app/routes/usage.py: GET /api/usage/me (MTD summary by feature),
  GET /api/usage/focus-groups/<id> (owner or admin)
- Registered usage_bp in app/__init__.py
- llm_service._record_usage now emits usage_update WS event to focus group room

Frontend:
- useMyUsage + useFocusGroupUsage hooks
- MyUsage.tsx: personal billing dashboard (cost cards + per-feature table)
- /billing route (ProtectedRoute) + Billing nav link
- FocusGroupSession: quota_warning amber banner with Progress bar,
  quota_exceeded + quota_warning WS events wired via websocketServiceNew

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:43:13 +01:00
Vadym Samoilenko
915c81b8f1 Complete phases D–G: quota enforcement, token invalidation, admin writes, backfill
Backend:
- token_version in JWT (bump_token_version, get_token_version on User model);
  jwt_required checks tv claim → 401 on mismatch; login routes embed version
- Quota pre-flight in all 3 LLM public methods (QuotaExceededError bubbles up)
- AI runner catches QuotaExceededError → sets status paused_quota + emits WS event
- Admin routes: POST /users (create), POST /users/<id>/reset-password,
  POST /pricing, GET /focus-groups with aggregated cost; PUT /users/<id>
  now bumps token_version on disable or role change
- backfill_usage.py: idempotent estimated-event generator for historical data,
  tiktoken for GPT models, char/3.8 for Gemini, --dry-run flag

Frontend:
- 402 interceptor dispatches quota_exceeded CustomEvent
- adminApi: createUser, resetPassword, createPricing, listFocusGroups
- UsersTab: New User dialog + Reset Password in edit dialog
- PricingTab: New Price dialog (model, provider, input/output/cached prices)
- FocusGroupsTab: focus groups table sorted by total cost
- Admin.tsx: 4th tab (Focus Groups)
- FocusGroupSession: admin-only cost badge + dismissable quota exceeded banner

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:34:48 +01:00
Vadym Samoilenko
015e6cc5cc Add Phase D admin panel: user management + usage analytics
Backend: /api/admin/* blueprint with user CRUD (list, get, update,
disable/enable), usage summary aggregation (group by user/model/feature/
day/focus_group), usage event drill-down, and pricing list. Fixed
admin_required decorator (async-safe). Added find_all/count/update
helpers to User model.

Frontend: /admin page (AdminRoute guard, 3 tabs) — Users table with
search/filter/edit dialog, Usage tab with KPI cards + bar chart +
events table, Pricing tab showing active model rows with tier details.
Admin nav link visible only to admin role.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:26:05 +01:00
Vadym Samoilenko
3e9ccafad2 Add LLM usage tracking infrastructure (Phases A-C)
- Model renames: gpt-5.2 → gpt-5.4-2026-03-05, gemini-3-pro-preview → gemini-3.1-pro-preview; retire gpt-4.1 via alias fallback
- New: llm_usage_context.py (ContextVar-based attribution), model_pricing.py (tiered pricing + 60s cache), usage_event.py (append-only telemetry), quota.py (user/FG quota enforcement with 80% warning)
- Wire _record_usage into all 3 LLM methods; set_llm_context at every service entry point
- Fix admin_required decorator (was sync, never awaited User.find_by_id); add active_required and with_user_context decorators
- Inject user_id into ContextVar from JWT on every authenticated request
- Add DB indexes for usage_events, model_pricing, users collections
- Seed script for model pricing (gpt-5.4 single-tier, gemini-3.1 two-tier 200k threshold)
- Fix parse_json_response NameError (logger undefined at module level)
- 70 passing tests: conftest.py with sys.modules stubs, test_usage_infrastructure.py (52 tests), rewrite stale test_llm_service.py (18 tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-24 18:08:27 +01:00
Vadym Samoilenko
283b31e786 Fix AI mode: race condition, split-brain UI, and stuck local state
- Backend: set status to ai_mode in the route handler before submitting
  to AI runner, eliminating the race condition where frontend's immediate
  status poll read the old status
- Frontend: replace all raw isAiModeActive prop usages with
  effectiveAiModeActive in DiscussionPanel (13 locations) so ReasoningPanel,
  status text, loading indicator, and manual/AI controls all reflect the
  correct state instantly on Start AI Mode click
- Frontend: add useEffect to sync localAiModeActive back to null once
  the parent prop catches up, preventing permanent override after natural
  session end
- These fixes also unblock the 3-second AI message polling which was
  never activating due to isAiModeActive staying false

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 19:02:15 +00:00
Vadym Samoilenko
33272cc677 Allow document uploads (PDF, DOCX, TXT, etc.) as focus group assets
- Expand allowed file types from images-only to also include:
  PDF, DOCX, DOC, TXT, MD, CSV, XLSX, XLS, PPTX, PPT, RTF
- validate_asset_file: skip PIL validation for non-image files; 50MB limit for docs / 10MB for images
- Correct MIME type detection for document extensions
- Store asset_type: "document"|"image" in metadata
- ImageDescriptionService: text files → LLM summary; binary docs → label; images → existing multimodal flow

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 17:08:30 +00:00
Vadym Samoilenko
4b47b334d7 Fix data isolation + conversation/decision 500 errors
Data isolation:
- GET /tasks/<id>: verify requesting user owns the task (403 if not)
- DELETE /tasks/<id>: same ownership check
- GET /tasks/status: add @jwt_required()
- GET /personas/<id>: add ownership check (403 if created_by != user)
- GET /focus-groups/<id>: add ownership check
- GET /focus-groups/<id>/messages: add ownership check
- POST/DELETE /focus-groups/<id>/participants: add ownership check

Fix conversation/decision 500:
- Convert POST /conversation/decision to async 202+background (was synchronous LLM → timed out / LLM errors → 500)
- Frontend polls waitForTaskResult for decision result before calling generateResponseAsync
- GET /conversation/insights: return empty insights (200) on LLM error instead of 500

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 17:02:10 +00:00
Vadym Samoilenko
1b387daacf Migrate task result delivery from WebSocket to HTTP polling
Backend:
- task_manager.py: add result/error/completed_at storage, TTL sweeper (5min), store_task_result() helper
- tasks.py: add GET /<task_id> endpoint returning stored result; cancel route stores 'cancelled' status
- __init__.py: start TTL sweeper on app startup
- All 8 bg functions: store result before emitting lightweight WS hint (no payload data)

Frontend:
- src/lib/taskPolling.ts: waitForTaskResult() — polls GET /tasks/{id} every 2s, WS hint triggers immediate poll, 5min timeout
- src/hooks/useTaskPolling.ts: drop-in replacement for useCancellableGeneration using polling
- Migrate 6 Promise-based WS listeners → waitForTaskResult() in DiscussionPanel, FocusGroupSession (×2), PersonaProfile, PersonaModificationModal, useDiscussionGuideGeneration
- Migrate 3 hook-based consumers → useTaskPolling in AIRecruiter, SyntheticUsers, BulkExportProgressModal

Fixes WS Promise leak: polling survives disconnects, background tabs, page reloads.
WS events retained as zero-payload hints for near-zero latency when connected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 16:46:58 +00:00
Vadym Samoilenko
c7034634e3 Fix all async LLM routes: bypass GCP 30s load balancer timeout
Convert 6 synchronous LLM routes to async 202+WebSocket pattern:
- generate-response (focus_group_ai): persona chat response
- generate-key-themes (focus_group_ai): discussion analysis
- modify-with-ai (personas): AI persona modification
- export-profile (personas): markdown profile export
- describe-asset (focus_groups): image AI description

Each route now returns 202 + task_id immediately, runs LLM in
asyncio background task, delivers result via WebSocket task_completed
event. Frontend listeners updated to wait for ws:task_completed
instead of HTTP response body.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:56:54 +00:00
Vadym Samoilenko
f4a587c4f7 Fix 500: add current_app import to focus_groups route
Missing import caused NameError when starting background discussion
guide generation task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:35:29 +00:00
Vadym Samoilenko
d8a5d6643f Fix discussion guide 504: async flow + WebSocket delivery
- Backend: /generate-discussion-guide now returns task_id immediately (202)
  and runs generation as a background asyncio task, delivering the guide
  via WebSocket task_completed event (bypasses GCP LB 30s timeout)
- Frontend: useDiscussionGuideGeneration awaits ws:task_completed event
  to resolve the guide Promise instead of waiting on the HTTP response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:32:08 +00:00
Vadym Samoilenko
6917518d11 Fix NameError: _fg_logger undefined in update_focus_group route
_fg_logger was used but never defined, causing a NameError on every
PUT /focus-groups/:id request that included llm_model (i.e. all autosave
and handleSubmit updates) — resulting in a 500 Internal Server Error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:26:08 +00:00
Vadym Samoilenko
f60d86e8cb Fix task_completed WebSocket payload too large
Don't send full persona objects in WS event — only send counts.
Frontend navigates to list page where personas load from API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 14:11:14 +00:00
Vadym Samoilenko
aa4090888d Fix persona generation 504: async flow + remove debug logging
- Backend: /generate-personas-full now returns task_id immediately (202)
  and runs generation as a background asyncio task, delivering results
  via WebSocket task_completed event (bypasses GCP LB 30s timeout)
- Frontend: AIRecruiter listens for ws:task_completed to process personas
  instead of awaiting the long HTTP response
- Remove 53 debug console.log calls from websocketServiceNew.ts including
  session_id exposure and a self-test emit that was firing fake events
- Remove debug logs from WebSocketContextNew, AIRecruiter, personaGenerator

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 14:00:07 +00:00
Vadym Samoilenko
3e1865edbd Apply Jintech security audit remediation (sprint 3) — 87/92 findings fixed
- Fix missing await on FocusGroup.get_messages() (N-L1)
- Replace time.sleep with asyncio.sleep in key_theme_service and focus_group_service (N-P10)
- Replace flask import with quart in focus_groups.py (N-S3)
- Add logger.error before all 500 returns in focus_groups.py (N-P6)
- Add logging to silent except blocks across routes (N-M10, N-M11)
- Add @rate_limit to 6 remaining AI endpoints (N-H4)
- Add --confirm flag to populate scripts before delete_many (S-H2)
- Remove hardcoded Azure ID fallbacks from msal_service.py and msalConfig.ts (A-M2, F-H4)
- Centralize make_serializable() in utils.py, remove duplicates from 3 route files (N-P7)
- Replace all datetime.utcnow() with datetime.now(timezone.utc) across entire backend (M-L2)
- AuthContext.tsx: only mark token validated on 200 success, not on non-401 errors (F-H2)
- Rename authType → auth_type in auth.py (N-S4)
- Add security_report.md and security_report.pdf with full 92-finding status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 12:51:18 +00:00
michael
b1be8f8c38 Add model alias for legacy gpt-5 database entries
Focus groups created before the gpt-5.2 rename have llm_model='gpt-5'
stored in MongoDB. Without an alias, the backend falls through to the
Gemini provider and fails with an aiohttp AssertionError.

Adds MODEL_ALIASES mapping and _resolve_model() helper so gpt-5 is
transparently resolved to gpt-5.2. Also updates all llm_model checks
to accept both values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:06:10 -06:00
michael
b0445de18b Update GPT-5 to GPT-5.2 and lower default reasoning effort to low
Swap model ID from gpt-5 to gpt-5.2 across all backend services,
frontend components, and documentation. Change default reasoning
effort from medium to low for faster responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:24:01 -06:00
michael
d4ce5d99bc Fix audience_brief and research_objective dropped in Stage 2 persona generation
Stage 2 (detailed persona generation) was ignoring the audience brief and
research objective, causing the LLM to guess research context from demographics
alone. Now passes both values through to generate_persona() in all three
endpoints (generate-personas-full, complete-and-save-persona, complete-persona)
and auto-generates prompt customization via customize_persona_prompt() when
they are provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:40:17 -06:00
michael
36e1752fea Remove __pycache__ files from git tracking
These files are already in .gitignore but were committed previously.
Removing them from tracking to prevent future conflicts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 13:28:19 -06:00
Michael Clervi
893b537b67 changed permissions 2025-12-19 19:26:16 +00:00
michael
96b9bfeedd Add LLM-generated one-line summaries for focus groups in list view
- Create focus_group_summary_service.py to generate concise summaries
- Add prompt template for summary generation
- Integrate summary generation after discussion guide creation
- Display summary under focus group title in list view with fallback to description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:51:28 -06:00
michael
25fcbc826f Redesign Enhance Brief to auto-apply changes with assumptions modal
- Replace suggestions panel with auto-apply: enhanced text now populates form fields automatically
- Add modal showing assumptions made during enhancement (3-5 bullet points)
- Reduce prompt scope by 50% for more focused, impactful enhancements
- Update backend to return enhanced text + assumptions instead of suggestions array

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 15:57:11 -06:00
michael
4d9b0afde7 upgraded to Gemini 3.0 Pro (gemini-3-pro-preview) from Gemini 2.5 Pro
- Upgraded google-genai package from 1.31.0 to 1.52.0
- Updated DEFAULT_MODEL in llm_service.py to gemini-3-pro-preview
- Updated all backend routes, services, and models with new model string
- Updated all frontend components with new model string and display labels
- Updated CLAUDE.md documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 10:37:36 -06:00
michael
f1cc094318 added full persona profile export in bulk actions menu (CSV, JSON and Markdown formats) 2025-09-10 22:38:35 -05:00
michael
abc9731e4a added hierarchical folders (just two levels) with drag and drop management 2025-09-10 19:53:06 -05:00
michael
e29d2a0bb9 made long actions cancellable (like persona generation, etc.), increased variety of persona generation with prompt changes and temperature variable, reduced length of key theme quotes, bug fixes 2025-09-10 16:24:05 -05:00
michael
8288cb9f5e fixed age to be a single number instead of range, fixed language for AI model to include thematic analysis, and added review/revert/save function to AI persona modification mechanism 2025-09-08 16:10:03 -05:00
michael
1b977ec517 graceful handling of expired JWT token 2025-09-03 13:15:00 -05:00
michael
6a40936508 major refactor of entire application - migrate sync -> async including pymongo -> motor, flask -> quart, google-generativeai -> google-genai 2025-08-27 15:20:56 -05:00
michael
3c1b082046 fixed messages about parallel generation of personas when only one persona is being generated - actually just removed the parallel language from all the notifications 2025-08-26 14:23:27 -05:00
michael
ece469d15e added generation prompt tracking and viewing in persona detail view (new tab) for audience brief and research objective 2025-08-26 14:15:50 -05:00
michael
065c368aeb added persona modification service - users can adjust individual personas with NLP via LLM 2025-08-26 12:45:15 -05:00
michael
fbef4f42f6 added drag and drop for reordering questions/activities in discussion guide edit mode 2025-08-26 09:52:24 -05:00
michael
3e73259ff3 refactored file uploader and bug fixes 2025-08-13 11:32:56 -05:00
michael
8a5c50cacb refactored image/asset attachment to discussion guide and discussion messages to avoid back end filenames being displayed to user - use metadata system instead. Also added a detailed persona export to markdown. Also, bug fixes 2025-08-12 15:43:34 -05:00
michael
d5dc0bc3af fixed bug related to moderator position status in discussion guide 2025-08-11 16:39:59 -05:00
michael
1c6e2e62e8 bug fixes related to websockets implementation 2025-08-11 09:33:09 -05:00
michael
9a3baa9d97 converted message polling to websockets 2025-08-10 18:08:34 -05:00
michael
da8639aee8 fixed folders again, bug fixes for gpt-5, adjusted response length calculation, cosmetic UI changes, other bug fixes 2025-08-09 10:08:45 -05:00
michael
fbb444037a fixed folders to be database instead of local storage based, implemented gpt-5, fixed key theme export quotes 2025-08-09 06:38:49 -05:00
michael
b649793013 added gpt-4.1 support among other things 2025-08-05 17:38:13 -05:00
michael
da7b2c0448 initial commit 2025-08-04 09:07:59 -05:00