Commit graph

21 commits

Author SHA1 Message Date
Vadym Samoilenko
290d5e32e6 fix: 7 caption/AD quality bugs + retranslation error handling
Bug fixes:
- Bug 1a: source_has_ad flag prevents AI generating AD over existing professional AD;
  JobBrief/Job models, gemini service prompt conditional, NewBrief UI checkbox
- Bug 1b: disable native textTracks on video element to prevent double captions
- Bug 2: caption ALL audible speech including off-screen narrators (prompt fix)
- Bug 3: DCMP §6.01 disfluency removal for EN/ES/FR/DE/IT (prompt + post-pass)
- Bug 4: VTT cue settings (line:0%, position:) preserved through parser round-trip
- Bug 5: Whisper word-level timestamp alignment via new caption_aligner service
- Bug 6: assert_cue_alignment used .start/.end; renamed to .start_time/.end_time
- New migration: backfill source_has_ad=False on existing jobs and job_briefs

Also fix retranslation error handling to preserve existing GCS URIs on failure
so video_native captions remain accessible if retranslation fails.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-07 15:38:20 +01:00
Vadym Samoilenko
43ef3a6cd8 fix(migrations): correct listCollections cursor parsing, add processing_failed+cancelled to status enum
Previous migrations used async-for on a dict (Atlas returns firstBatch, not
async cursor) — silently failed. New migration reads firstBatch correctly and
sets the complete status list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 20:47:21 +01:00
Vadym Samoilenko
2e8cf8269e fix(tts): fetch job_doc before gcs_path call in _generate_language_tts; add cancelled migration
- translate_and_synthesize.py: fetch job_doc from DB right before the combined
  MP3 upload so gcs_path() has the gcs_prefix needed for newer jobs; removes the
  duplicate fetch that existed later in the same function
- migration_2026-04-30-000001: add 'cancelled' to MongoDB $jsonSchema validator
  enum so cancel_job writes no longer fail Document validation
- Dashboard.tsx: include all active processing statuses in the Processing counter

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 20:36:03 +01:00
Vadym Samoilenko
31199f8705 chore: push all session changes — backend hardening, tests, apache config, deploy scripts
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-30 15:52:14 +01:00
Vadym Samoilenko
ff372c7322 fix(security): close MT-17/18/19, restore cross-tenant tests, quick wins
Blocks 1–5 of stabilization plan:

SECURITY
- validation.py: restore settings.upload_max_video_bytes (T-14 regression fix)
  and JSON object key validation that was incorrectly removed
- MT-18: add accessible_org_ids filter to list_for_reviewer/list_for_linguist
  so reviewers/linguists only see jobs from their own org in QC queue
- MT-17: add Membership.team_ids[], write to it on invitation acceptance and
  direct team add/remove; migration backfills from Team.member_user_ids
- MT-19: validate all target_team_ids belong to invitation's org_id at creation

TESTS
- Restore test_cross_tenant_isolation.py (was deleted, only .pyc remained)
- Extend with MT-18 reviewer org isolation tests

QUICK WINS
- W-8: remove time.sleep(1) + dead debug block from POST /jobs (task was
  undefined — would have caused NameError → HTTP 500 on every job creation)
- T-13: warn at startup when REDIS_URL configured but connection failed
- T-16: skip language_qc lifespan migration when count=0 (no DB scan on startup)

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-30 10:32:23 +01:00
Vadym Samoilenko
f4a82dcf76 fix(migrations): replace relative imports with absolute in PR-7 migrations
Migration runner executes scripts outside package context — relative
imports fail. Pattern matches all other migration files.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-29 22:05:32 +01:00
Vadym Samoilenko
595897e61a feat(w-12): JobBrief model, endpoints, migration + brief→job linkage
- JobBrief model (DRAFT→SUBMITTED→APPROVED→FULFILLED) with 6 CRUD
  endpoints: list, create, get, patch (DRAFT only), submit, approve
- All endpoints use MembershipContext; read=VIEWER, mutate=MANAGER,
  approve=ADMIN for org-scoped access
- create_job accepts brief_id Form field; validates APPROVED brief,
  copies organization_id/project_id/deadline from brief, marks brief
  FULFILLED after job insert
- organization_id now populated from project client_id on job create
  (fixes missing multi-tenant field on new jobs)
- migration_2026-04-29-000001: job_briefs collection + 4 indexes
- Wired briefs router into main.py

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:38:08 +01:00
Vadym Samoilenko
3e3be935c6 feat(w-13): structured Job.failure schema, PROCESSING_FAILED status, audit actions
Add JobFailure model (step, type, message, retriable, occurred_at,
retry_count) to job.py. Add PROCESSING_FAILED to JobStatus (legacy
TTS_FAILED/RENDER_FAILED preserved for back-compat).

Add missing Job fields that existed in DB but not the Pydantic model:
organization_id, brief_id, gcs_prefix, initial_linguist_id,
initial_reviewer_id, failure, retry_count.

Add JOB_TASK_FAILED, JOB_RETRY, JOB_BULK_RETRY to AuditAction enum.

Add migration 2026-04-29-000000: processing_failed in schema validator +
compound indexes (failure.step/status) and (status/org_id/created_at).

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
2026-04-29 20:27:28 +01:00
Vadym Samoilenko
1563714454 feat(saas): Phase 3 — membership-based authz + Mailgun + job.organization_id
authz.py (new):
- MembershipContext — per-request membership dict for the current user
- get_membership_context FastAPI dependency
- require_org_role(min_role) — dependency factory keyed off org_id path param
- require_platform_admin()
- OrgScopedQuery — adds organization_id filter; platform admin passes through
- bump_user_membership_cache — invalidates Redis key on membership writes

dependencies.py:
- get_accessible_project_ids now queries memberships collection first;
  legacy pm_client_ids / team.member_user_ids fallback retained until migration runs
  (four job-route access checks at lines 608/1054/1181/1538 are fixed via this function)

routes_clients.py:
- _assert_pm_or_admin and _assert_client_access are now async and query memberships
- All 10 call sites updated with await + db arg

emailer.py:
- Switched from SendGrid to Mailgun REST API via httpx (already in requirements)
- _send() is now fully async; same public method signatures preserved
- send_completion_email uses _send()

config.py:
- Added mailgun_api_key, mailgun_domain, mailgun_from settings
- sendgrid_api_key kept with empty default for backward compat

migration_2026-04-28-000003:
- Backfills job.organization_id from project.client_id
- Creates (organization_id, status, created_at) sparse index on jobs

routes_organizations.py / routes_invitations.py:
- Call bump_user_membership_cache after every membership write

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:56:42 +01:00
Vadym Samoilenko
00fb1aacc6 feat(saas): Phase 2 — invitation flow, email templates, MS SSO zero-membership
Backend:
- models/invitation.py — Invitation model + create/accept/preview schemas
- routes_invitations.py — org-scoped POST/GET/DELETE + public preview/accept endpoints
  Single-use token via find_one_and_update; sha256(token) stored in DB, plaintext in email URL
- emailer.py — _send() helper; send_invitation_email, send_welcome_email, send_password_reset_email
  send_completion_email refactored to use _send()
- migration_2026-04-28-000002 — creates invitations collection with TTL index (30d audit trail)
- routes_auth.py — new MS SSO users provisioned with zero memberships instead of role=PRODUCTION;
  they land on "no access" page until an admin invites them
- main.py — registers invitations_org_router and invitations_router

Frontend:
- routes/AcceptInvite.tsx — public page at /accept-invite?token=...
  Four states: new user (name+password), existing user (confirm), MS user, already-member
- App.tsx — /accept-invite route outside RequireAuth
- types/api.ts — Invitation, InvitationCreate, InvitationPreview, InvitationAcceptRequest/Response
- lib/api.ts — listInvitations, createInvitation, revokeInvitation, previewInvitation, acceptInvitation
- hooks/useClients.ts — useInvitations, useCreateInvitation, useRevokeInvitation

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:52:08 +01:00
Vadym Samoilenko
6f1be645ce feat(saas): Phase 0+1 — Organization/Membership entities and dev branch
Introduces the multi-tenant SaaS foundation alongside the existing
client/team/project model (zero-downtime shim period):

Backend:
- app/models/organization.py — Organization + OrgRole enum (OWNER/ADMIN/MANAGER/MEMBER/VIEWER)
- app/models/membership.py — Membership model with MemberDetail for enriched responses
- app/services/membership_service.py — upsert/remove/list/has_org_role helpers
- app/api/v1/routes_organizations.py — /organizations CRUD + /members sub-resource + /me/memberships
- main.py — registers organizations router
- migrations: create memberships collection (unique index) + backfill from pm_client_ids/team members

Frontend:
- types/api.ts — Organization, OrgRole, Membership, OrganizationCreateRequest types; Client marked @deprecated
- hooks/useClients.ts — useOrganizations, useOrganization, useOrgMembers, useAddOrgMember,
  useUpdateOrgMember, useRemoveOrgMember, useMyMemberships
- lib/api.ts — listOrganizations, getOrganization, createOrganization, updateOrganization,
  listOrgMembers, addOrgMember, updateOrgMember, removeOrgMember, getMyMemberships

Reads fall back to the clients collection during transition; all writes go to organizations.
Existing /clients endpoints and hooks are untouched.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:46:24 +01:00
Vadym Samoilenko
723bbbc695 fix: add project_manager migration + add migration step to full-deploy.sh
- New migration updates MongoDB users collection validator to accept
  project_manager role and pm_client_ids field
- full-deploy.sh was missing the run_migrations step entirely; added it
  after rebuild_containers so new role/field validators apply on every deploy

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-27 16:05:17 +01:00
Vadym Samoilenko
cf761c4bb6 feat: add linguist role and user management navigation
- Add LINGUIST role to UserRole enum (backend + frontend)
- Grant linguists access to QC Review, Final Review, review notes, and VTT editing
- Add MongoDB migration to update schema validator with linguist role
- Add admin seed: vadymsamoilenko@oliver.agency is promoted to admin on startup
- Add User Management sidebar link for admin users
- Fix Login.tsx role type cast to use UserRole instead of hardcoded union

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-04-16 11:46:33 +01:00
michael
bcfc026e61 feat: add migration for rendering_qc status in MongoDB schema
The rendering_qc status was added to the Python model but was missing
from the MongoDB schema validator, causing WriteError when setting
job status during QC re-rendering.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-11 10:05:05 -06:00
michael
76c4c60b0d fix: add tts_failed and render_failed to MongoDB schema validator
MongoDB was rejecting status updates to 'tts_failed' and 'render_failed'
because these values weren't in the schema validator's enum, even though
they were defined in the Python JobStatus model.

This caused TTS failures to leave jobs stuck in 'tts_generating' status
with no error feedback to users - the WriteError from MongoDB prevented
the status and error fields from being updated.

The migration adds both failed statuses to the jobs collection validator.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2026-01-05 14:09:41 -06:00
michael
3ca70a7c03 fix: add rendering_video status to MongoDB schema validator
The rendering_video status was defined in job.py and frontend types but
was missing from the MongoDB schema validator, causing document update
failures when jobs transitioned to the rendering_video state.

Changes:
- Add migration script to update existing databases
- Update mongodb-init.js for new database setups

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-27 08:40:23 -06:00
michael
e6578e0ccf add approved_source and qc_feedback job statuses to MongoDB schema
- Add migration to update jobs collection validator with new statuses
- Update mongodb-init.js for fresh deployments
- Fix deploy.sh to properly run migrations with 'python migrate.py up'

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
2025-12-22 13:12:14 -06:00
michael
d25fb921a1 fixed dates on scheme validator migration 2025-10-10 10:59:20 -05:00
michael
92169d047b added scheme validator 2025-10-10 10:55:54 -05:00
michael
665b49c3f1 added MSAL microsoft authentication 2025-10-10 09:19:39 -05:00
michael
af2562096a initial commit 2025-08-24 16:28:33 -05:00