Commit graph

61 commits

Author SHA1 Message Date
Vadym Samoilenko
1b387daacf Migrate task result delivery from WebSocket to HTTP polling
Backend:
- task_manager.py: add result/error/completed_at storage, TTL sweeper (5min), store_task_result() helper
- tasks.py: add GET /<task_id> endpoint returning stored result; cancel route stores 'cancelled' status
- __init__.py: start TTL sweeper on app startup
- All 8 bg functions: store result before emitting lightweight WS hint (no payload data)

Frontend:
- src/lib/taskPolling.ts: waitForTaskResult() — polls GET /tasks/{id} every 2s, WS hint triggers immediate poll, 5min timeout
- src/hooks/useTaskPolling.ts: drop-in replacement for useCancellableGeneration using polling
- Migrate 6 Promise-based WS listeners → waitForTaskResult() in DiscussionPanel, FocusGroupSession (×2), PersonaProfile, PersonaModificationModal, useDiscussionGuideGeneration
- Migrate 3 hook-based consumers → useTaskPolling in AIRecruiter, SyntheticUsers, BulkExportProgressModal

Fixes WS Promise leak: polling survives disconnects, background tabs, page reloads.
WS events retained as zero-payload hints for near-zero latency when connected.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 16:46:58 +00:00
Vadym Samoilenko
c7034634e3 Fix all async LLM routes: bypass GCP 30s load balancer timeout
Convert 6 synchronous LLM routes to async 202+WebSocket pattern:
- generate-response (focus_group_ai): persona chat response
- generate-key-themes (focus_group_ai): discussion analysis
- modify-with-ai (personas): AI persona modification
- export-profile (personas): markdown profile export
- describe-asset (focus_groups): image AI description

Each route now returns 202 + task_id immediately, runs LLM in
asyncio background task, delivers result via WebSocket task_completed
event. Frontend listeners updated to wait for ws:task_completed
instead of HTTP response body.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:56:54 +00:00
Vadym Samoilenko
f4a587c4f7 Fix 500: add current_app import to focus_groups route
Missing import caused NameError when starting background discussion
guide generation task.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:35:29 +00:00
Vadym Samoilenko
d8a5d6643f Fix discussion guide 504: async flow + WebSocket delivery
- Backend: /generate-discussion-guide now returns task_id immediately (202)
  and runs generation as a background asyncio task, delivering the guide
  via WebSocket task_completed event (bypasses GCP LB 30s timeout)
- Frontend: useDiscussionGuideGeneration awaits ws:task_completed event
  to resolve the guide Promise instead of waiting on the HTTP response

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:32:08 +00:00
Vadym Samoilenko
6917518d11 Fix NameError: _fg_logger undefined in update_focus_group route
_fg_logger was used but never defined, causing a NameError on every
PUT /focus-groups/:id request that included llm_model (i.e. all autosave
and handleSubmit updates) — resulting in a 500 Internal Server Error.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:26:08 +00:00
Vadym Samoilenko
f359157949 Fix focus group create: 500 on update + 400 on autosave
- FocusGroup.update: use matched_count > 0 instead of modified_count > 0
  so updates succeed even when data is unchanged (was returning 500)
- useFocusGroupAutoSave: skip save if name is empty (not all-fields-empty)
  preventing 400 Bad Request when autosave fires before name is filled

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 15:20:40 +00:00
Vadym Samoilenko
f60d86e8cb Fix task_completed WebSocket payload too large
Don't send full persona objects in WS event — only send counts.
Frontend navigates to list page where personas load from API.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 14:11:14 +00:00
Vadym Samoilenko
aa4090888d Fix persona generation 504: async flow + remove debug logging
- Backend: /generate-personas-full now returns task_id immediately (202)
  and runs generation as a background asyncio task, delivering results
  via WebSocket task_completed event (bypasses GCP LB 30s timeout)
- Frontend: AIRecruiter listens for ws:task_completed to process personas
  instead of awaiting the long HTTP response
- Remove 53 debug console.log calls from websocketServiceNew.ts including
  session_id exposure and a self-test emit that was firing fake events
- Remove debug logs from WebSocketContextNew, AIRecruiter, personaGenerator

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 14:00:07 +00:00
Vadym Samoilenko
c00728f375 Fix Gemini LLM AssertionError: force httpx transport over aiohttp
google-genai SDK uses aiohttp when it's available in the environment
(installed via llama-index-core), causing AssertionError (connector is None)
on async requests. Pass httpx_async_client in HttpOptions to bypass aiohttp.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-23 13:05:27 +00:00
Vadym Samoilenko
7f0df54de3 Fix domain typo: oliver.solution → oliver.solutions across all files
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 13:40:00 +00:00
Vadym Samoilenko
cf698e1e87 Remove create_default_user call from run.py (method removed in security remediation) 2026-03-20 13:34:26 +00:00
Vadym Samoilenko
05d7ea68e2 Fix make_serializable import — move to utils/__init__.py (was shadowed by utils/ package) 2026-03-20 13:33:11 +00:00
Vadym Samoilenko
d1788a4017 Fix .dockerignore — exclude *.txt but keep requirements.txt 2026-03-20 13:28:32 +00:00
Vadym Samoilenko
4a6b4d6fe0 Dockerize backend — replace systemd service with docker-compose
- Add backend/Dockerfile (python:3.12-slim)
- Add docker-compose.yml (backend :5137 + mongo:7)
- Add backend/.dockerignore
- Rewrite deploy.sh: build frontend locally, rsync dist/, docker compose up --build
- Remove semblance.service (no longer needed)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 13:21:34 +00:00
Vadym Samoilenko
bb4dca0fe8 Update production URL to optical-dev.oliver.solution
Replace ai-sandbox.oliver.solutions with optical-dev.oliver.solution
across all config, env, docs, and source files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 12:55:45 +00:00
Vadym Samoilenko
3e1865edbd Apply Jintech security audit remediation (sprint 3) — 87/92 findings fixed
- Fix missing await on FocusGroup.get_messages() (N-L1)
- Replace time.sleep with asyncio.sleep in key_theme_service and focus_group_service (N-P10)
- Replace flask import with quart in focus_groups.py (N-S3)
- Add logger.error before all 500 returns in focus_groups.py (N-P6)
- Add logging to silent except blocks across routes (N-M10, N-M11)
- Add @rate_limit to 6 remaining AI endpoints (N-H4)
- Add --confirm flag to populate scripts before delete_many (S-H2)
- Remove hardcoded Azure ID fallbacks from msal_service.py and msalConfig.ts (A-M2, F-H4)
- Centralize make_serializable() in utils.py, remove duplicates from 3 route files (N-P7)
- Replace all datetime.utcnow() with datetime.now(timezone.utc) across entire backend (M-L2)
- AuthContext.tsx: only mark token validated on 200 success, not on non-401 errors (F-H2)
- Rename authType → auth_type in auth.py (N-S4)
- Add security_report.md and security_report.pdf with full 92-finding status

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-20 12:51:18 +00:00
michael
c7ff1755ee Add architecture document generator and PDF
Create comprehensive technical architecture document (PDF) with 11
chapters covering system architecture, frontend/backend design, data
model, auth, WebSocket communication, LLM integration, and core
feature flows. Includes 11 Mermaid diagrams rendered as PNGs.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-24 11:39:50 -06:00
michael
b1be8f8c38 Add model alias for legacy gpt-5 database entries
Focus groups created before the gpt-5.2 rename have llm_model='gpt-5'
stored in MongoDB. Without an alias, the backend falls through to the
Gemini provider and fails with an aiohttp AssertionError.

Adds MODEL_ALIASES mapping and _resolve_model() helper so gpt-5 is
transparently resolved to gpt-5.2. Also updates all llm_model checks
to accept both values.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 12:06:10 -06:00
michael
b0445de18b Update GPT-5 to GPT-5.2 and lower default reasoning effort to low
Swap model ID from gpt-5 to gpt-5.2 across all backend services,
frontend components, and documentation. Change default reasoning
effort from medium to low for faster responses.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:24:01 -06:00
michael
d4ce5d99bc Fix audience_brief and research_objective dropped in Stage 2 persona generation
Stage 2 (detailed persona generation) was ignoring the audience brief and
research objective, causing the LLM to guess research context from demographics
alone. Now passes both values through to generate_persona() in all three
endpoints (generate-personas-full, complete-and-save-persona, complete-persona)
and auto-generates prompt customization via customize_persona_prompt() when
they are provided.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:40:17 -06:00
michael
850cb25067 Fix GPT-5 Responses API crash on reasoning items with None content
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 08:05:52 -06:00
michael
1708bd75a4 Remove SDK version logging on every Gemini call
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:30:42 -06:00
michael
7f4f659501 Bump google-genai to >=1.56.0 to fix aiohttp AssertionError
Version 1.52.0 has a known bug where aiohttp connector is None.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 14:07:38 -06:00
michael
d79c202e8f Fix NameError: use print instead of logger in get_gemini_client
logger is not defined at module level where get_gemini_client() lives.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:59:30 -06:00
michael
c9848210bb Add full traceback and SDK version logging for AssertionError debug
This will help identify where exactly the AssertionError is occurring
in the google-genai SDK and what version is installed on the server.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:49:47 -06:00
michael
5a825934f8 Add verbose exception debugging for empty error messages
Log full exception details: type, module, str, repr, args, and __dict__
to diagnose why Gemini errors are producing empty messages.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:39:15 -06:00
michael
b50c0fa2a4 Fix empty error messages from Google GenAI SDK
Catch genai_errors.APIError specifically and extract e.code and e.message
attributes for proper error logging. The generic str(e) was returning empty
strings for Google API errors, making debugging impossible.

- Import google.genai.errors for specific exception handling
- Add APIError catch before generic Exception in generate_content()
- Add APIError catch before generic Exception in generate_contextual_response()
- Properly categorize errors by HTTP code for retry logic (429/500+ retryable)
- Fix time.sleep to await asyncio.sleep in contextual response handler

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-07 13:26:55 -06:00
michael
6ee80e67aa Create fresh LLM clients per call instead of caching
The previous event loop tracking approach still caused issues - when replacing
a cached client, its garbage collection triggers aclose() which tries to close
the aiohttp session on the wrong event loop.

Simplest fix: create a fresh client for each call. The overhead is minimal
compared to the actual LLM API call, and this completely avoids all event
loop mismatch issues in ASGI environments.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 16:56:36 -06:00
michael
94f98b837b Add event loop tracking to LLM client getters
The previous lazy initialization fix wasn't sufficient - the genai.Client
internally caches async structures bound to the event loop at creation time.
With ASGI servers like Hypercorn, subsequent requests may come on different
event loop contexts, causing "Future attached to a different loop" errors.

Now tracks which event loop the client was created on and recreates it if
the loop has changed.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 16:46:35 -06:00
michael
36e1752fea Remove __pycache__ files from git tracking
These files are already in .gitignore but were committed previously.
Removing them from tracking to prevent future conflicts.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 13:28:19 -06:00
Michael Clervi
893b537b67 changed permissions 2025-12-19 19:26:16 +00:00
michael
45c7a52aeb Fix asyncio event loop mismatch in LLM service
The genai.Client and AsyncOpenAI clients were being created at module
import time, before the Quart/Hypercorn event loop existed. This caused
"Future attached to a different loop" errors when async calls were made,
resulting in autonomous focus group conversations stopping with
"excessive_silence".

Changed to lazy initialization - clients are now created on first use
within the running event loop context via get_gemini_client() and
get_openai_client() helper functions.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-19 13:24:23 -06:00
michael
bb16165a83 Remove unused eventlet import and dead emit_with_ack function
Legacy code from Flask-SocketIO migration that's no longer used.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-09 11:51:23 -06:00
michael
1b98757676 Replace flask_jwt_extended imports with quart_jwt
Fixes ModuleNotFoundError on server by using the custom
Quart-compatible JWT implementation.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-09 11:49:22 -06:00
michael
517f033ede Pin all Python dependency versions in requirements.txt
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-09 11:34:48 -06:00
michael
96b9bfeedd Add LLM-generated one-line summaries for focus groups in list view
- Create focus_group_summary_service.py to generate concise summaries
- Add prompt template for summary generation
- Integrate summary generation after discussion guide creation
- Display summary under focus group title in list view with fallback to description

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-04 08:51:28 -06:00
michael
25fcbc826f Redesign Enhance Brief to auto-apply changes with assumptions modal
- Replace suggestions panel with auto-apply: enhanced text now populates form fields automatically
- Add modal showing assumptions made during enhancement (3-5 bullet points)
- Reduce prompt scope by 50% for more focused, impactful enhancements
- Update backend to return enhanced text + assumptions instead of suggestions array

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 15:57:11 -06:00
michael
4d9b0afde7 upgraded to Gemini 3.0 Pro (gemini-3-pro-preview) from Gemini 2.5 Pro
- Upgraded google-genai package from 1.31.0 to 1.52.0
- Updated DEFAULT_MODEL in llm_service.py to gemini-3-pro-preview
- Updated all backend routes, services, and models with new model string
- Updated all frontend components with new model string and display labels
- Updated CLAUDE.md documentation

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 10:37:36 -06:00
michael
d3aca7c309 added retry logic and smart field completion for persona generation, increased OpenAI timeout
- persona generation now retries up to 2 times on failure
- missing persona fields are intelligently completed based on context
- expanded required field validation from 5 to 9 fields
- prompt now explicitly lists all required fields with validation instructions
- fixed discussion guide cancellation check to handle object responses

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-12-01 09:39:58 -06:00
michael
7c68f5d0e4 truncated long folder names so the buttons are still visible, migrated legacy folders to new format 2025-09-11 10:40:01 -05:00
michael
f1cc094318 added full persona profile export in bulk actions menu (CSV, JSON and Markdown formats) 2025-09-10 22:38:35 -05:00
michael
abc9731e4a added hierarchical folders (just two levels) with drag and drop management 2025-09-10 19:53:06 -05:00
michael
e29d2a0bb9 made long actions cancellable (like persona generation, etc.), increased variety of persona generation with prompt changes and temperature variable, reduced length of key theme quotes, bug fixes 2025-09-10 16:24:05 -05:00
michael
8288cb9f5e fixed age to be a single number instead of range, fixed language for AI model to include thematic analysis, and added review/revert/save function to AI persona modification mechanism 2025-09-08 16:10:03 -05:00
michael
1b977ec517 graceful handling of expired JWT token 2025-09-03 13:15:00 -05:00
michael
6a40936508 major refactor of entire application - migrate sync -> async including pymongo -> motor, flask -> quart, google-generativeai -> google-genai 2025-08-27 15:20:56 -05:00
michael
3c1b082046 fixed messages about parallel generation of personas when only one persona is being generated - actually just removed the parallel language from all the notifications 2025-08-26 14:23:27 -05:00
michael
ece469d15e added generation prompt tracking and viewing in persona detail view (new tab) for audience brief and research objective 2025-08-26 14:15:50 -05:00
michael
065c368aeb added persona modification service - users can adjust individual personas with NLP via LLM 2025-08-26 12:45:15 -05:00
michael
fbef4f42f6 added drag and drop for reordering questions/activities in discussion guide edit mode 2025-08-26 09:52:24 -05:00