ai_qc/CLAUDE_AXA.md
nickviljoen 29ee941037 refactor(formatting_diff): narrow scope to bold + italic only
First real-data test against the AXA car-insurance PDFs surfaced a
noise problem: the new document is a brand refresh — every page flips
font (PublicoBanner-Bold→PublicoHeadline-Bold) and colour
(#893f4a→#2e3092). At medium-per-finding that crashed the diff score
to 0.0 and drowned the bold-regression signal AXA actually flagged.

Drop font, size, colour comparators. Keep bold + italic — the
attributes the vision-LLM consistently misses on dense layouts. The
LLM already narrates colour-scheme rebrands and font swaps in its
Modified / Style-changes blocks; running both layers on the same
visual change just double-counts it.

Tests inverted from "X change is flagged" to "X change is NOT
flagged" to lock the scope decision in.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-19 12:37:19 +02:00

6.5 KiB

AXA Client Documentation

Referenced from main CLAUDE.md. Detailed AXA QC profile descriptions, document-mode pipeline notes, and status.

Overview

AXA QC is built around document-mode — multi-page PDF analysis (policy documents, forms, brochures), not single-asset image checks. The document-mode subsystem (backend/document_mode/) was built for AXA and is now reused by Boots Production Pack.

Status (2026-05-10): Phases 1, 3, 4, 5, 6 merged to develop and live on dev (https://optical-dev.oliver.solutions/ai_qc/). Phase 6 wires veraPDF into the accessibility check (PAC-equivalent PDF/UA-1 validation) and splits accessibility into its own dedicated profile. Email to AXA pending — explains Adobe vs PAC + veraPDF parity findings + requests the original axa-transaction-charges-100326.pdf so we can run a true apples-to-apples comparison. Not yet on prod — held for AXA show-and-tell + email response. Full plan in backend/AXA_DOCUMENT_MODE_PLAN.md.

AXA Profiles

axa_policy_document — single-document mode (7 checks)

Multi-page policy document QC. mode: document, scopes vary per check. Accessibility validation lives in the dedicated axa_accessibility profile, not here.

Check What it does Weight
axa_font_inventory Per-page font extraction + brand-font compliance against AXA's approved font list 1.0
axa_phone_inventory Extracts phone numbers across pages, validates format and approved-list membership 1.0
axa_bold_words_definitions Bold-word inventory + definition cross-check (seed list at backend/document_mode/data/axa_bold_words_seed.json) 2.0
axa_page_numbering Page numbering format and continuity 1.0
axa_print_preflight Print-preflight checks (color space, embedded fonts, image resolution) 1.0
axa_print_code Print code presence + format 1.0
axa_omg_versioning OMG version footer/header presence and consistency 1.0

axa_accessibility — accessibility-only mode (1 check, strict-grade)

mode: document, strict_grade: true. Standalone PDF/UA-1 validation for users who only need to check accessibility compliance without the full policy-document content suite. Mirrors how axes4 PAC is used — single-purpose, binary verdict.

Check What it does Weight
axa_pdf_accessibility PDF/UA-1 validation via veraPDF (matches axes4 PAC), with deterministic PyMuPDF fallback if veraPDF is not installed 1.0

axa_policy_document_diff — old-vs-new diff mode (1 check)

mode: document_diff — compares two PDFs (old vs new policy version) and reports structured changes.

Check What it does Weight
axa_pdf_diff Detects added/removed/modified pages, paragraphs, defined terms, phone numbers 1.0

Document-mode infrastructure

AXA's document-mode subsystem is the foundation for all multi-page PDF QC in this app:

  • document_mode/ingest.py — PDF ingestion, page rendering, span/font/color extraction via PyMuPDF
  • document_mode/dispatcher.py — Orchestrates per-check execution against pages, supports scopes: document / targeted / page_sample / page_pair / page_each
  • document_mode/checks.py, print_preflight_checks.py, accessibility_checks.py — AXA check implementations
  • document_mode/diff_engine.py, diff_report_writer.py — Old-vs-new diff handling
  • document_mode/result_writer.py — HTML report rendering with per-page sections

Boots Production Pack reuses this entire spine — so any infra changes here affect both clients.

AI usage across AXA tools

For client-facing context: 8 of 9 AXA tools are deterministic (no LLM, $0 cost, runs in seconds). Only axa_pdf_diff uses AI — Gemini 2.5 Pro vision-LLM page-pair comparison at ~$0.40-0.80 per pair, supplemented by a deterministic PyMuPDF span comparator that catches bold/italic flips the vision-LLM misses (font/size/colour changes are left to the LLM narrative diff — flagging them deterministically drowns out the bold/italic regressions on re-branded documents). The accessibility check uses veraPDF, which is a rule-based open-source PDF/UA-1 validator — not AI. This framing matters when clients conflate "automation" with "AI".

Tool Type Engine
axa_font_inventory, axa_phone_inventory, axa_bold_words_definitions, axa_page_numbering, axa_print_code, axa_omg_versioning Deterministic PyMuPDF (text + font extraction, regex)
axa_print_preflight Deterministic PyMuPDF (page geometry, image colour spaces, DPI, transparency, PDF/X)
axa_pdf_accessibility Deterministic (rule-based) veraPDF subprocess (PDF/UA-1 / Matterhorn Protocol) + PyMuPDF fallback
axa_pdf_diff AI + deterministic Gemini 2.5 Pro vision-LLM (content + font/size/colour narrative) + PyMuPDF span comparator (bold/italic flip detection)

Open items

  • AXA show-and-tell pending — feedback will drive the next round of tuning
  • Awaiting axa-transaction-charges-100326.pdf from AXA (the file PAC was run against) — needed to fully confirm veraPDF↔PAC parity on the Structure Elements rule bucket
  • Phase 2 (any further check expansion) deferred until after show-and-tell
  • Canonical AXA font list / approved phone list / OMG version reference data may need expansion as test PDFs surface gaps
  • Prod deployment of veraPDF + axa_accessibility profile — held until AXA confirms findings on dev

veraPDF deployment

axa_pdf_accessibility runs the veraPDF PDF/UA-1 validator as a subprocess when the binary is available. veraPDF implements the Matterhorn Protocol — the same rule set axes4 PAC uses — so its verdict is the closest open-source equivalent to PAC.

Binary resolution order (in accessibility_checks._resolve_verapdf_binary):

  1. VERAPDF_BIN env var
  2. verapdf on PATH
  3. /opt/ai_qc/vendor/verapdf/verapdf (project-local production install)

If veraPDF isn't installed the check falls back to the 9-criterion deterministic PyMuPDF layer — no breakage, just less depth. Production install pattern is a project-local bundled-JRE tarball under /opt/ai_qc/vendor/verapdf/ to avoid touching system Java or other projects on shared servers.

Key files

  • backend/AXA_DOCUMENT_MODE_PLAN.md — full design plan and phase breakdown
  • backend/document_mode/ — pipeline implementation
  • backend/profiles/axa_policy_document.json, axa_accessibility.json, axa_policy_document_diff.json
  • backend/document_mode/data/axa_bold_words_seed.json — bold-word seed list