wiki: auto-compile 2026-05-18 (1 log(s), 319 articles)
This commit is contained in:
parent
5f81909886
commit
4ffd637ba5
16 changed files with 1363 additions and 3 deletions
127
01 Projects/ac-helper/DEVELOPER_MANUAL.md
Normal file
127
01 Projects/ac-helper/DEVELOPER_MANUAL.md
Normal file
|
|
@ -0,0 +1,127 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Activation Calendar Helper — Developer Manual
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The application is a **hybrid stack**:
|
||||
|
||||
1. **Backend**: Python FastAPI application running in Docker.
|
||||
- Handles API endpoints (`/api/*`), WebSocket connections (`/ws`), and authentication.
|
||||
- Manages database interactions (PostgreSQL) and AI service integrations.
|
||||
2. **Frontend**: React application (TypeScript) served via Apache/Nginx as static files.
|
||||
- Handles UI, state management (Zustand), and MSAL authentication.
|
||||
- Communicates with the backend via REST API and WebSockets.
|
||||
3. **Infrastructure**: Docker Compose for local/dev environment orchestration.
|
||||
- `postgres`: Data persistence.
|
||||
- `app`: FastAPI container.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Backend**: Python 3.11+, FastAPI, SQLAlchemy, MSAL (Azure AD), OpenAI/Gemini/Claude SDKs.
|
||||
- **Frontend**: React 18+, TypeScript, Vite, Handsontable (Spreadsheets), MSAL React, Zustand (State).
|
||||
- **Database**: PostgreSQL 16.
|
||||
- **Auth**: Azure AD (MSAL PKCE flow).
|
||||
|
||||
## Local Setup
|
||||
|
||||
### Prerequisites
|
||||
- Docker & Docker Compose.
|
||||
- Node.js 18+.
|
||||
- Python 3.11+.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Clone Repository**:
|
||||
```bash
|
||||
git clone <repo-url>
|
||||
cd ac-helper
|
||||
```
|
||||
|
||||
2. **Environment Configuration**:
|
||||
```bash
|
||||
cp .env.example .env
|
||||
cp frontend/.env.example frontend/.env
|
||||
```
|
||||
- Fill in `AZURE_TENANT_ID`, `AZURE_CLIENT_ID`, `AZURE_REDIRECT_URI`.
|
||||
- Add API keys for desired AI providers (`OPENAI_API_KEY`, `GEMINI_API_KEY`, etc.).
|
||||
- Set `ADMIN_EMAIL` for initial bootstrap.
|
||||
|
||||
3. **Start Backend**:
|
||||
```bash
|
||||
docker-compose up -d
|
||||
```
|
||||
- DB will be available at `localhost:5432`.
|
||||
- API will be available at `http://localhost:${APP_PORT:-8100}`.
|
||||
|
||||
4. **Start Frontend**:
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
npm run dev
|
||||
```
|
||||
- Frontend will run on `localhost:5173` and proxy API requests to `localhost:${APP_PORT}`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
| Variable | Description | Required |
|
||||
|----------|-------------|----------|
|
||||
| `AZURE_TENANT_ID` | Azure AD Tenant ID | Yes |
|
||||
| `AZURE_CLIENT_ID` | Azure AD App Client ID | Yes |
|
||||
| `AZURE_REDIRECT_URI` | Callback URL | Yes |
|
||||
| `OPENAI_API_KEY` | OpenAI API Key | Optional |
|
||||
| `GEMINI_API_KEY` | Google Gemini API Key | Optional |
|
||||
| `ANTHROPIC_API_KEY` | Anthropic API Key | Optional |
|
||||
| `EMERGENCY_TOKEN` | Token for SSO bypass | Optional |
|
||||
| `POSTGRES_PASSWORD` | Database password | Yes |
|
||||
|
||||
## Key Services & Entry Points
|
||||
|
||||
- **`backend/main.py`**: FastAPI app entry point. Registers routers (`/api/auth`, `/api/data`, `/api/ai`, etc.).
|
||||
- **`frontend/src/main.tsx`**: React app entry point. Initializes MSAL provider and Handsontable modules.
|
||||
- **`frontend/src/stores/useAuthStore.ts`**: Manages user state and token refresh logic.
|
||||
- **`frontend/src/api/client.ts`**: Axios instance with interceptors for token injection and error handling.
|
||||
|
||||
## API Reference
|
||||
|
||||
- **`POST /api/auth/login`**: Initiates Microsoft Login.
|
||||
- **`GET /api/auth/config`**: Returns MSAL configuration for the frontend.
|
||||
- **`GET /api/data/sheets`**: Lists all sheets.
|
||||
- **`GET /api/data/sheets/{id}`**: Retrieves sheet data.
|
||||
- **`POST /api/data/sheets`**: Creates a new sheet.
|
||||
- **`POST /api/ai/extract`**: Uploads a PDF for AI extraction.
|
||||
- **`WS /ws`**: WebSocket endpoint for real-time updates.
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production (Apache/Nginx)
|
||||
|
||||
1. **Build Frontend**:
|
||||
```bash
|
||||
cd frontend
|
||||
npm run build
|
||||
```
|
||||
Output goes to `frontend/dist`.
|
||||
|
||||
2. **Configure Web Server**:
|
||||
- Serve `frontend/dist` as static files.
|
||||
- Proxy `/api/*` and `/ws/*` to the Docker container's `APP_PORT` (default 8100).
|
||||
|
||||
3. **Docker**:
|
||||
- Set `APP_PORT` in `.env` to match your host port binding.
|
||||
- Ensure `POSTGRES_PASSWORD` is secure.
|
||||
- Configure `AZURE_REDIRECT_URI` to the production URL.
|
||||
|
||||
4. **Restart**: `docker-compose up -d`.
|
||||
|
||||
## Known Gotchas
|
||||
|
||||
- **MSAL Redirects**: Ensure `AZURE_REDIRECT_URI` exactly matches the registered Azure App URI. Dev mode requires `http://localhost:5173/ac-helper/` (if proxied) or similar.
|
||||
- **CORS**: The backend must be configured to allow requests from the frontend origin.
|
||||
- **File Permissions**: The `data/` volume in Docker must be writable by the app user.
|
||||
- **AI Rate Limits**: High usage of AI extraction may hit API rate limits. Implement retry logic (already partially implemented via `OPENAI_MAX_RETRIES`).
|
||||
- **Dependent Dropdowns**: Changing a "Category" or "Media" may invalidate dependent cells. Validate data integrity on import/export.
|
||||
78
01 Projects/ac-helper/USER_MANUAL.md
Normal file
78
01 Projects/ac-helper/USER_MANUAL.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Activation Calendar Helper — User Manual
|
||||
|
||||
## What This Tool Does
|
||||
|
||||
The **Activation Calendar Helper** is a web-based platform for marketing teams to manage deliverables, coordinate campaigns, and extract information from briefs using AI automation.
|
||||
|
||||
### Core Capabilities
|
||||
- **Smart Spreadsheet**: An interactive grid (powered by Handsontable) for managing deliverables with sorting, filtering, and dependent dropdowns.
|
||||
- **AI Command Interface**: Natural language commands to create, update, or query deliverables.
|
||||
- **Brief Extraction**: Upload PDF briefs and use AI (OpenAI, Gemini, or Claude) to automatically populate the spreadsheet with extracted data.
|
||||
- **Multi-Sheet Management**: Create, rename, duplicate, or delete multiple data sheets.
|
||||
- **Export**: Export current data to CSV.
|
||||
|
||||
## Who Uses It
|
||||
|
||||
- **Marketing Managers/Coordinators**: To track deliverables across multiple campaigns.
|
||||
- **Creative Teams**: To upload briefs and manage asset statuses.
|
||||
- **Admins**: To manage users, dropdown lists (Categories, Media, etc.), and clients.
|
||||
|
||||
## How to Access
|
||||
|
||||
1. **Login**: Navigate to the application URL (e.g., `https://ai-sandbox.oliver.solutions/ac-helper/`).
|
||||
2. **Microsoft SSO**: Click "Sign in with Microsoft" to authenticate via Azure AD.
|
||||
3. **Emergency Access**: If SSO is unavailable, click "Emergency access" at the bottom of the login screen and enter the provided token.
|
||||
4. **Dev Mode**: If running locally in development (`npm run dev`), login is automatic.
|
||||
|
||||
## Main Workflows
|
||||
|
||||
### 1. Managing the Spreadsheet
|
||||
|
||||
The main interface is the spreadsheet grid.
|
||||
|
||||
1. **Navigate Sheets**: Use the sidebar or top bar to switch between sheets.
|
||||
2. **Create/Duplicate Sheets**: Click the `+` icon or "Duplicate" to create a new dataset.
|
||||
3. **Edit Data**: Click any cell to edit. Dropdowns are dependent:
|
||||
- **Category** must be selected first.
|
||||
- **Media** options depend on the selected Category.
|
||||
- **Sub-media** options depend on the selected Media.
|
||||
4. **Sort/Filter**: Use the column headers to sort or apply filters.
|
||||
5. **Export**: Click "Export CSV" to download the current sheet's data.
|
||||
|
||||
### 2. Processing Briefs (AI Extraction)
|
||||
|
||||
1. **Go to Brief Upload**: Navigate to the "Briefs" section.
|
||||
2. **Upload File**: Click "Upload Brief" and select a PDF.
|
||||
3. **Review Extraction**: The AI will parse the document and present the extracted data.
|
||||
4. **Edit & Confirm**: Verify the data in the review view. Edit any fields if necessary.
|
||||
5. **Import**: Click "Import to Sheet" to add the deliverables to your active spreadsheet.
|
||||
|
||||
### 3. Using AI Commands
|
||||
|
||||
Use the command input (usually a floating button or top bar) to:
|
||||
- **Create**: "Create a new deliverable for Category X, Media Y."
|
||||
- **Update**: "Change the status of Deliverable #123 to 'In Progress'."
|
||||
- **Query**: "Show me all pending items in the US region."
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: How do I create new dropdown options (e.g., new Media types)?**
|
||||
A: Go to **Admin > Dropdowns** (if you have admin rights) to add new categories, media types, or sub-media options.
|
||||
|
||||
**Q: Can I recover a deleted sheet?**
|
||||
A: No, deletion is permanent. Please ensure you have backups (CSV exports) before deleting.
|
||||
|
||||
**Q: Why is my AI command failing?**
|
||||
A: Ensure you have the correct API keys configured in the environment. If running locally, check that the AI service keys are valid.
|
||||
|
||||
**Q: How do I export data?**
|
||||
A: Click the "Export CSV" button in the toolbar. Note: This exports only the *current view* (including any active filters).
|
||||
|
||||
**Q: Is my data secure?**
|
||||
A: Yes. Data is stored in a PostgreSQL database. Access is restricted to authenticated users. Sensitive files (briefs) are stored securely and not publicly accessible.
|
||||
124
01 Projects/build-a-squad/DEVELOPER_MANUAL.md
Normal file
124
01 Projects/build-a-squad/DEVELOPER_MANUAL.md
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Build-A-Squad Developer Manual
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
Build-A-Squad is a single-page application (SPA) built with React. It follows a client-side architecture where all state management and calculations occur in the browser.
|
||||
|
||||
### Key Components
|
||||
1. **Auth Gate:** Manages Microsoft identity integration via MSAL.
|
||||
2. **App Container:** Orchestrates state between the three main tabs (Scoping, Configurator, Squad).
|
||||
3. **Data Layer:** Static JSON files for assets and routing logic, parsed at build/initialization time.
|
||||
4. **AI Service:** Client-side integration with Google Gemini API for scope analysis and auditing.
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Framework:** React 19 (using Hooks: `useState`, `useMemo`, `useCallback`).
|
||||
- **Language:** TypeScript 5.8.
|
||||
- **Build Tool:** Vite 6.
|
||||
- **Styling:** Tailwind CSS (via CDN).
|
||||
- **Auth:** `@azure/msal-react` and `@azure/msal-browser`.
|
||||
- **AI:** `@google/genai` (Gemini 3.1 Pro).
|
||||
- **Icons:** Lucide React.
|
||||
|
||||
## Local Setup
|
||||
|
||||
### Prerequisites
|
||||
- Node.js (LTS version recommended)
|
||||
- npm or yarn
|
||||
|
||||
### Installation
|
||||
1. Clone the repository.
|
||||
2. Install dependencies:
|
||||
```bash
|
||||
npm install
|
||||
```
|
||||
3. Create a `.env` file in the root directory (see **Environment Variables**).
|
||||
4. Start the development server:
|
||||
```bash
|
||||
npm run dev
|
||||
```
|
||||
The app will run at `http://localhost:3000`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
The application relies on Vite's `define` mechanism to inject environment variables at build time. Create a `.env` file with the following keys:
|
||||
|
||||
```env
|
||||
# Google Gemini API Key for AI features
|
||||
GEMINI_API_KEY=your_gemini_api_key_here
|
||||
|
||||
# Microsoft Azure AD B2B/C2B Configuration
|
||||
AZURE_CLIENT_ID=your_azure_client_id
|
||||
AZURE_TENANT_ID=your_azure_tenant_id
|
||||
AZURE_REDIRECT_URI=http://localhost:3000/auth/callback # Or your prod URI
|
||||
```
|
||||
|
||||
*Note: Ensure the Redirect URI in the Azure Portal matches the one in your `.env` file.*
|
||||
|
||||
## Key Services & Entry Points
|
||||
|
||||
### `index.tsx`
|
||||
Entry point. Initializes MSAL, checks authentication status, and renders the `App` component wrapped in `MsalProvider`.
|
||||
|
||||
### `auth.ts`
|
||||
Configures the `PublicClientApplication` instance for Azure MSAL. Exports `msalInstance` and `loginScopes`.
|
||||
|
||||
### `App.tsx`
|
||||
The core application logic:
|
||||
- **State Management:** Manages `selectedAssets`, `overrides`, `manualStaff`, `scenarios`, and tab navigation.
|
||||
- **Tab Logic:**
|
||||
- **Scoping:** Calls Gemini API to analyze `scopingInput` and map to `ASSETS_DATA`.
|
||||
- **Configurator:** Renders filtered lists of `ASSETS_DATA`.
|
||||
- **Squad:** Calculates FTEs using `STAFFING_ROUTES` and `ROLE_DISCIPLINE_MAP`.
|
||||
- **Scenario Management:** Saves/loads `Scenario` objects to local state (and supports CSV export/import).
|
||||
|
||||
### `mockData.ts`
|
||||
Loads static data from JSON files:
|
||||
- `data/assets.json`: Full catalog of 30+ asset types.
|
||||
- `data/staffingRoutes.json`: Map of `uniques` to `{ role: hours }`.
|
||||
- `data/roleDisciplineMap.json`: Map of `role` to `discipline`.
|
||||
|
||||
*Note on Naming:* Despite the filename `mockData.ts`, this serves as the primary data source for the production catalog. The assets are parsed from `documents/Latest Asset List.md` using `scripts/parse-assets.ts`.
|
||||
|
||||
### `scripts/parse-assets.ts`
|
||||
A utility script to update the catalog data. It parses a pipe-delimited markdown table and outputs the three required JSON files into the `data/` directory.
|
||||
|
||||
Run: `npm run parse-catalog`
|
||||
|
||||
## API Reference
|
||||
|
||||
### AI Integration
|
||||
The app uses `@google/genai` with `gemini-3.1-pro-preview`.
|
||||
|
||||
**Scope Analysis:**
|
||||
- **Input:** Project brief text.
|
||||
- **Output:** JSON schema-based array of `ScopingResult` objects containing `scopeItem`, `volume`, `catalogId`, and `rationale`.
|
||||
- **Usage:** `GoogleGenAI` instance calls `models.generateContent` with structured output schema.
|
||||
|
||||
**Operational Audit:**
|
||||
- **Input:** Current `selectedAssets`, `overrides`, `manualStaff`.
|
||||
- **Output:** Strategic text insights regarding the squad composition.
|
||||
|
||||
## Deployment
|
||||
|
||||
1. **Build:**
|
||||
```bash
|
||||
npm run build
|
||||
```
|
||||
This generates static files in the `dist/` directory.
|
||||
2. **Base Path:** In `vite.config.ts`, the base path is set to `/build-a-squad/` for production. Ensure your web server (e.g., IIS, Nginx, Apache) is configured to serve the app from this subdirectory or adjust the `base` config if deploying to the root.
|
||||
3. **Azure AD:** Update the `AZURE_REDIRECT_URI` in the `.env.production` file to point to your production deployment URL before building for production.
|
||||
|
||||
## Known Gotchas
|
||||
|
||||
1. **State Persistence:** The app does **not** save state to a database. Scenarios are saved only within the current session or via exported CSV. Reloading the page resets all data.
|
||||
2. **Asset Catalog Updates:** The catalog is static JSON. You **must** run `npm run parse-catalog` after updating `documents/Latest Asset List.md` to see changes in the app.
|
||||
3. **Billable Hours:** The default FTE calculation uses 1,600 hours/year. Change `billableHoursTarget` in the Squad tab if your agency uses a different divisor (e.g., 2,080).
|
||||
4. **Tailwind CDN:** The app uses Tailwind via CDN for simplicity in this demo/prototype phase. For production, consider building a full Tailwind CSS pipeline to avoid FOUC (Flash of Unstyled Content) and enable PurgeCSS for smaller bundle sizes.
|
||||
5. **MSAL Redirect:** If you encounter auth loops, verify that the `redirectUri` in `auth.ts` (and `.env`) exactly matches the one registered in the Azure Portal App Registration under "Authentication".
|
||||
85
01 Projects/build-a-squad/USER_MANUAL.md
Normal file
85
01 Projects/build-a-squad/USER_MANUAL.md
Normal file
|
|
@ -0,0 +1,85 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Build-A-Squad User Manual
|
||||
|
||||
## What This Tool Does
|
||||
|
||||
Build-A-Squad is a client-side staffing calculator designed for creative agencies. It helps you estimate the team size and composition needed for a project by:
|
||||
1. **Analyzing your project brief** using AI to suggest relevant creative assets.
|
||||
2. **Selecting specific deliverables** from a catalog of 30+ asset types.
|
||||
3. **Calculating FTE (Full-Time Equivalent) projections** by discipline and role based on standardized staffing routes.
|
||||
|
||||
All calculations happen in your browser; no data is sent to a server for storage, though AI features connect to Google Gemini.
|
||||
|
||||
## Who Uses It
|
||||
|
||||
- **Resourcing Managers:** To quickly estimate headcount needs for new proposals.
|
||||
- **Creative Directors:** To understand the volume and type of creative work involved.
|
||||
- **Strategy Teams:** To align project scope with budget and resource capabilities.
|
||||
|
||||
## How to Access
|
||||
|
||||
1. Navigate to the application URL provided by your agency.
|
||||
2. Click the **"Sign in with Microsoft"** button.
|
||||
3. Authenticate using your Oliver Microsoft account.
|
||||
4. Once authenticated, you will be directed to the **Scoping** tab.
|
||||
|
||||
## Main Workflows
|
||||
|
||||
The tool follows a three-tab workflow:
|
||||
|
||||
### 1. Scoping (AI-Assisted)
|
||||
This is the starting point for most projects.
|
||||
|
||||
1. **Paste Brief:** Copy and paste the text of a project brief into the input area.
|
||||
2. **Analyze:** Click "Analyze Scope." The AI (Google Gemini) will read the brief and suggest relevant assets from the catalog.
|
||||
3. **Upload Screenshot (Optional):** If you have a PDF or image of the brief, use the upload button to extract text via OCR.
|
||||
4. **Review Suggestions:** The AI returns a list of suggested assets with rationales. Click the **"Add to Squad"** button on any suggestion to bulk-add it to your selected list.
|
||||
5. **Adjust:** You can still manually edit volumes or add/remove items later.
|
||||
|
||||
### 2. Configurator (Manual Selection)
|
||||
Use this tab to browse the full catalog or fine-tune your selections.
|
||||
|
||||
1. **Browse Assets:** View the full list of 30+ asset types.
|
||||
2. **Filter:** Use the filters to sort by:
|
||||
- **Category:** (e.g., Brand Communication, Video, Digital).
|
||||
- **Type:** Master vs. Adapt.
|
||||
- **Pencil Pro Flag:** Highlighted assets with specific internal flags.
|
||||
3. **Add Assets:** Click **"Add"** on any asset. A modal will appear where you can set the **Volume** (quantity) if multiple units are needed.
|
||||
4. **Review Squad:** Switch to the **Squad Projection** tab to see the results of your selections.
|
||||
|
||||
### 3. Squad Projection (FTE Calculation)
|
||||
This tab displays the calculated staffing requirements.
|
||||
|
||||
1. **View Breakdown:** See the total FTE and hours required, broken down by Discipline (e.g., Creative, Strategy) and specific Roles (e.g., Senior Designer, Planner).
|
||||
2. **Adjust Parameters:**
|
||||
- **Billable Hours:** Change the annual billable hours target (default: 1,600) if your agency uses a different standard.
|
||||
- **Manual Overrides:** Click the pencil icon next to any role's FTE to manually override the calculated number.
|
||||
- **Bespoke Staff:** Add manual staff entries for roles not in the standard catalog using the "Add Manual Staff" feature.
|
||||
- **Exclude Roles:** Use the checkmarks to exclude specific roles from the calculation if they are handled by external partners or other departments.
|
||||
3. **AI Audit:** Click **"Generate AI Audit"** to get strategic insights and recommendations based on your current squad composition.
|
||||
4. **Save Scenarios:**
|
||||
- Enter a name in the "Save Scenario" field and click **Save**.
|
||||
- Select multiple scenarios to compare them side-by-side. Delta indicators will show differences between saved snapshots.
|
||||
5. **Export:** Use the **Download** button to export your current squad data as a CSV file.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Why do my FTE numbers look different from the project?**
|
||||
A: The calculator uses standardized "staffing routes" (base hours per asset volume) averaged across the agency. It provides a baseline estimation. Manual overrides are encouraged for specific project nuances.
|
||||
|
||||
**Q: Can I save my work and come back later?**
|
||||
A: The application does not auto-save to a server. Use the **Save Scenario** feature to create named snapshots of your squad. You can re-load a scenario by pasting the saved data if you have exported it, but note that the primary persistence is within your current browser session via local state or manual save/compare features.
|
||||
|
||||
**Q: How accurate is the AI suggestion?**
|
||||
A: The AI uses Google Gemini to interpret intent, not just keywords. It is a strong starting point but should always be reviewed. You can manually adjust any suggested asset or volume.
|
||||
|
||||
**Q: Is my project brief data secure?**
|
||||
A: Yes. The text is sent only to the Google Gemini API for analysis and is not stored by the application itself. No backend database is used.
|
||||
|
||||
**Q: What is the "Pencil Pro" flag?**
|
||||
A: This indicates assets that have specific internal criteria or flags within the studio's operational model. It is primarily used for internal filtering and reporting.
|
||||
181
01 Projects/ford_qc/DEVELOPER_MANUAL.md
Normal file
181
01 Projects/ford_qc/DEVELOPER_MANUAL.md
Normal file
|
|
@ -0,0 +1,181 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Ford BnP QC System Developer Manual
|
||||
|
||||
## Architecture Overview
|
||||
The system follows a modular architecture:
|
||||
1. **Entry Points:**
|
||||
* `qc_engine.py`: Core engine for running QC checks locally or programmatically.
|
||||
* `ford_qc_box_hotfolder_process.py`: Production daemon that monitors Box folders, orchestrates download, processing, and upload.
|
||||
2. **Core Logic:**
|
||||
* `qc_module.py`: Contains `run_qc_checks()` which orchestrates the loading of profiles and execution of check scripts.
|
||||
* `utils/config.py`: Centralized configuration management, loading from `.env` files.
|
||||
* `utils/path_resolver.py`: Handles dynamic path resolution for profiles and assets.
|
||||
3. **Checks Library (`checks/`):**
|
||||
* Individual Python modules for each QC rule (e.g., `special_requirements_mec_bau.py`).
|
||||
* `html_reporter.py` / `html_error_reporter.py`: Logic for generating human-readable and error-specific reports.
|
||||
4. **Configuration:**
|
||||
* `.env.example`: Template for environment variables.
|
||||
* `profiles/`: JSON files defining the sequence and parameters of QC checks.
|
||||
|
||||
## Tech Stack
|
||||
* **Language:** Python 3.8+
|
||||
* **Box SDK:** `boxsdk` for cloud storage interaction.
|
||||
* **PDF Generation:** `reportlab` (used in `generate_pdf.py` for documentation, not core QC).
|
||||
* **Logging:** `logging` module, with optional `systemd` journal integration.
|
||||
* **Configuration:** `python-dotenv` (implied by `.env.example` usage).
|
||||
* **Service Management:** `systemd` for production deployment.
|
||||
|
||||
## Local Setup
|
||||
|
||||
### 1. Clone and Install
|
||||
```bash
|
||||
git clone <repo_url>
|
||||
cd ford_qc_system
|
||||
python -m venv venv
|
||||
source venv/bin/activate
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
### 2. Environment Variables
|
||||
Copy `.env.example` to `.env.dev` and configure:
|
||||
|
||||
```ini
|
||||
FORD_QC_ENV=development
|
||||
# Box Folders (Can be dummy IDs for local-only testing without Box)
|
||||
BOX_SOURCE_FOLDER_ID=0
|
||||
BOX_REPORT_FOLDER_ID=0
|
||||
BOX_PROCESSED_FOLDER_ID=0
|
||||
# Optional: Set Box credentials if testing Box integration locally
|
||||
BOX_CONFIG_FILE=ford_box_config.json
|
||||
# Paths (relative to script dir)
|
||||
DOWNLOAD_PATH=download_tmp
|
||||
WORKING_DIR=working
|
||||
QC_PROFILE_PATH=profiles/ford_bnp.json
|
||||
```
|
||||
|
||||
### 3. Box JWT Config
|
||||
Place a valid `ford_box_config.json` (from IT or Box Admin) in the project root if testing Box integration.
|
||||
|
||||
## Environment Variables Reference
|
||||
| Variable | Description | Default/Example |
|
||||
| :--- | :--- | :--- |
|
||||
| `FORD_QC_ENV` | Environment mode | `production` or `development` |
|
||||
| `BOX_SOURCE_FOLDER_ID` | Box folder ID for incoming assets | `332861865120` |
|
||||
| `BOX_REPORT_FOLDER_ID` | Box folder ID for QC reports | `332864939558` |
|
||||
| `BOX_PROCESSED_FOLDER_ID` | Box folder ID for processed assets | `332861653811` |
|
||||
| `BOX_CONFIG_FILE` | Path to Box JWT JSON config | `ford_box_config.json` |
|
||||
| `DOWNLOAD_PATH` | Local dir for downloads | `download_tmp` |
|
||||
| `WORKING_DIR` | Local dir for extraction | `working` |
|
||||
| `QC_PROFILE_PATH` | Path to QC checks definition | `profiles/ford_bnp.json` |
|
||||
| `BOX_CONNECTION_TIMEOUT` | Box API connect timeout | (see code) |
|
||||
| `BOX_READ_TIMEOUT` | Box API read timeout | (see code) |
|
||||
| `MAX_RETRIES` | Max Box API retry attempts | (see code) |
|
||||
| `WATCHDOG_INTERVAL` | Time between Box folder polls | (see code) |
|
||||
|
||||
## Key Services & Entry Points
|
||||
|
||||
### `run_qc_checks(profile_path, input_file)`
|
||||
* **Location:** `qc_module.py`
|
||||
* **Description:** The core function. Loads the QC profile (list of check definitions), iterates through them, and calls the respective check scripts. It aggregates results into a JSON structure.
|
||||
* **Inputs:**
|
||||
* `profile_path`: Path to JSON file.
|
||||
* `input_file`: Path to the `.zip` asset pack.
|
||||
* **Output:** JSON string containing aggregated results.
|
||||
|
||||
### `run_qc_profile(profile_path, input_file)`
|
||||
* **Location:** `qc_engine.py`
|
||||
* **Description:** High-level API for running the profile. Handles JSON loading validation and result aggregation. Used by command-line interface.
|
||||
|
||||
### `ford_qc_box_hotfolder_process.py`
|
||||
* **Description:** The main production loop.
|
||||
1. Authenticates with Box.
|
||||
2. Polls `BOX_SOURCE_FOLDER_ID` every `WATCHDOG_INTERVAL` seconds.
|
||||
3. Downloads new `.zip` files to `DOWNLOAD_PATH`.
|
||||
4. Calls `run_qc_checks` via `qc_module`.
|
||||
5. Generates HTML/JSON reports.
|
||||
6. Uploads reports to `BOX_REPORT_FOLDER_ID`.
|
||||
7. Moves processed `.zip` to `BOX_PROCESSED_FOLDER_ID`.
|
||||
8. Handles errors via `HTMLErrorReporter`.
|
||||
|
||||
### `HTMLReporter` and `HTMLErrorReporter`
|
||||
* **Location:** `checks/html_reporter.py` / `checks/html_error_reporter.py`
|
||||
* **Description:** Utility classes/functions to format JSON results into Bootstrap-styled HTML. `HTMLErrorReporter` handles specific error types (network, file access, QC failure).
|
||||
|
||||
## API Reference (Internal)
|
||||
|
||||
### QC Profile JSON Structure (`profiles/ford_bnp.json`)
|
||||
```json
|
||||
[
|
||||
{
|
||||
"script": "checks.special_requirements_mec_bau",
|
||||
"config": {
|
||||
"some_param": "value"
|
||||
}
|
||||
},
|
||||
...
|
||||
]
|
||||
```
|
||||
* `script`: Module path to the check function (e.g., `checks.module_name.check_function`).
|
||||
* `config`: Dict of parameters passed to the check.
|
||||
|
||||
### Result JSON Structure
|
||||
```json
|
||||
{
|
||||
"profile": "ford_bnp.json",
|
||||
"timestamp": "2023-10-27T10:00:00Z",
|
||||
"checks": [
|
||||
{
|
||||
"script": "checks.special_requirements_mec_bau",
|
||||
"result": {
|
||||
"status": "pass",
|
||||
"details": { "pack_type": "MEC" }
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production (Systemd)
|
||||
1. Copy `ford_qc_box_hotfolder_process.py` to `/opt/ford_qc/bin/`.
|
||||
2. Create service file `/etc/systemd/system/ford_qc.service`:
|
||||
```ini
|
||||
[Unit]
|
||||
Description=Ford BnP QC Box Hotfolder Processor
|
||||
After=network.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=box-cli
|
||||
Group=box-cli
|
||||
WorkingDirectory=/opt/ford_qc
|
||||
EnvironmentFile=/opt/ford_qc/.env.prod
|
||||
ExecStart=/usr/bin/python3 /opt/ford_qc/bin/ford_qc_box_hotfolder_process.py
|
||||
Restart=on-failure
|
||||
# Enable systemd journal logging
|
||||
StandardOutput=journal
|
||||
StandardError=journal
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
```
|
||||
3. Enable and start:
|
||||
```bash
|
||||
sudo systemctl daemon-reload
|
||||
sudo systemctl enable ford_qc.service
|
||||
sudo systemctl start ford_qc.service
|
||||
```
|
||||
|
||||
## Known Gotchas
|
||||
1. **Path Resolution:** The script uses `os.path.dirname(os.path.abspath(__file__))` to determine the base directory. Hardcoded paths in `qc_engine.py` (fallback) can cause issues if deployed to a different location. Always rely on `config` or relative paths.
|
||||
2. **Box Authentication:** JWT keys expire. Ensure the Box JWT key file is rotated regularly. The `BOX_CONFIG_FILE` must be up-to-date.
|
||||
3. **File Locking:** The system assumes no other process is modifying files in `WORKING_DIR`. It uses basic `os.makedirs` checks but lacks advanced file locking. Ensure exclusive access.
|
||||
4. **Memory Usage:** Large asset packs can consume significant RAM during extraction. The `FALLBACK_WORKING_DIR` is a safety net if the primary working directory lacks space.
|
||||
5. **Test Script Specifics:** `test_mec_bau_determination.py` is a debugging helper. Do not run in production. It assumes specific example files exist in `example packs/`.
|
||||
6. **PDF Generation:** `generate_pdf.py` is for documentation only and has dependencies (`reportlab`, `PIL`, `mmdc`/`mermaid-cli`) not required for the core QC system.
|
||||
121
01 Projects/ford_qc/USER_MANUAL.md
Normal file
121
01 Projects/ford_qc/USER_MANUAL.md
Normal file
|
|
@ -0,0 +1,121 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# Ford BnP QC System User Manual
|
||||
|
||||
## What This Tool Does
|
||||
The Ford BnP (Build and Price) QC System is an automated quality control tool designed to validate automotive asset packs before they are deployed to customer-facing applications. It ensures that images, metadata, and configuration files meet strict Ford quality standards.
|
||||
|
||||
**Key Capabilities:**
|
||||
* **Automated Validation:** Runs 13 specialized checks on asset packs (MEC, BAU, Color Chips, Lifestyle, etc.) to detect errors in image format, metadata correctness, and business rules.
|
||||
* **Cloud Integration:** Automatically monitors a Box cloud folder for new `.zip` asset packs, downloads them, processes them, and uploads detailed HTML and JSON reports back to a designated Box folder.
|
||||
* **Detailed Reporting:** Generates rich, Bootstrap-styled HTML reports that clearly list issues found, their severity, and actionable steps to fix them. JSON reports are also generated for programmatic consumption.
|
||||
* **Error Recovery:** Includes robust error handling for network timeouts, file access issues, and QC failures, with automatic fallback mechanisms.
|
||||
|
||||
## Who Uses It
|
||||
* **Asset Managers & Creators:** Submitting new vehicle imagery or configuration data and needing confirmation that their uploads are valid.
|
||||
* **DevOps/IT Operations:** Monitoring the health and status of the asset upload pipeline via logs and Box folder contents.
|
||||
* **Developers/QA Engineers:** Debugging specific validation failures or adding new QC rules.
|
||||
|
||||
## How to Access
|
||||
### Production Access (Automated Workflow)
|
||||
For most users, interaction is indirect. You upload asset packs to a specific Box folder (`BOX_SOURCE_FOLDER_ID`), and the system processes them automatically.
|
||||
|
||||
* **Source Folder (Upload Here):** `332861865120`
|
||||
* **Reports Folder (View Results Here):** `332864939558`
|
||||
* **Processed Folder (Archived Files):** `332861653811`
|
||||
|
||||
### Local/Development Access (Manual Workflow)
|
||||
Developers and QA engineers can run the system locally to test changes or validate specific files.
|
||||
|
||||
**Prerequisites:**
|
||||
1. Python 3.8+
|
||||
2. A valid Ford Box JWT Configuration file (`ford_box_config.json`)
|
||||
3. The project codebase cloned.
|
||||
|
||||
## Main Workflows (Step-by-Step)
|
||||
|
||||
### Workflow 1: Manual QC Check (Local Development)
|
||||
Use this to test a specific asset pack file locally.
|
||||
|
||||
1. **Prepare your environment:**
|
||||
```bash
|
||||
# Clone the repository
|
||||
git clone <repo_url>
|
||||
cd ford_qc_system
|
||||
|
||||
# Set up virtual environment (recommended)
|
||||
python -m venv venv
|
||||
source venv/bin/activate # On Windows: venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
2. **Create a local configuration:**
|
||||
```bash
|
||||
cp .env.example .env.dev
|
||||
# Edit .env.dev and ensure FORD_QC_ENV=development
|
||||
# You may not need Box credentials for local testing if not using Box,
|
||||
# but BOX_CONFIG_FILE should still point to a valid (or dummy) file if required by imports.
|
||||
```
|
||||
|
||||
3. **Run the QC Engine:**
|
||||
```bash
|
||||
# Basic usage with the default profile
|
||||
python qc_engine.py profiles/ford_bnp.json --input_file /path/to/your/asset_pack.zip --reports_dir ./local_reports
|
||||
```
|
||||
|
||||
4. **Review Results:**
|
||||
Check the `./local_reports` directory for `qc_report.html` and `qc_report.json`.
|
||||
|
||||
5. **Generate a specific HTML report from existing JSON (Optional):**
|
||||
```bash
|
||||
python -m checks.html_reporter ./local_reports/qc_report.json
|
||||
```
|
||||
|
||||
### Workflow 2: Monitoring the Box Cloud Folder (Production/Dev Server)
|
||||
The system runs as a background service to monitor Box.
|
||||
|
||||
1. **Configuration:**
|
||||
Ensure `.env.prod` is correctly set up with:
|
||||
* `FORD_QC_ENV=production`
|
||||
* Correct `BOX_SOURCE_FOLDER_ID`, `BOX_REPORT_FOLDER_ID`, etc.
|
||||
* Valid `BOX_CONFIG_FILE` path.
|
||||
|
||||
2. **Start the Process:**
|
||||
```bash
|
||||
python ford_qc_box_hotfolder_process.py
|
||||
```
|
||||
*For systemd deployment, use the provided service file.*
|
||||
|
||||
3. **Check Logs:**
|
||||
```bash
|
||||
journalctl -u ford_qc_system -f
|
||||
```
|
||||
|
||||
4. **Monitor Box:**
|
||||
* Upload `.zip` files to `BOX_SOURCE_FOLDER_ID`.
|
||||
* Wait for processing.
|
||||
* Check `BOX_REPORT_FOLDER_ID` for new HTML/JSON reports.
|
||||
* Original `.zip` files are moved to `BOX_PROCESSED_FOLDER_ID`.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Where do I find the list of QC checks?**
|
||||
A: The checks are defined in `profiles/ford_bnp.json`. Each entry specifies a script and configuration.
|
||||
|
||||
**Q: How do I fix a failed QC check?**
|
||||
A: Open the generated HTML report in `BOX_REPORT_FOLDER_ID` or your local `reports_dir`. The report provides specific details and "Fix Instructions" for each error.
|
||||
|
||||
**Q: What if the Box API connection fails?**
|
||||
A: The system includes retry logic (`MAX_RETRIES`, `RETRY_BACKOFF_BASE`). If it exceeds retries, it logs the error and moves the file to an error state (often a specific error report folder or leaving it in place with a logged error) depending on configuration. Check logs for details.
|
||||
|
||||
**Q: Can I add custom QC checks?**
|
||||
A: Yes. The system is modular. Create a new Python script in the `checks/` directory, define it in `profiles/ford_bnp.json` with the appropriate `script` name and `config` parameters, and restart the service.
|
||||
|
||||
**Q: Where are the logs stored?**
|
||||
A:
|
||||
* **Production:** Systemd Journal (`journalctl -u ford_qc_system`)
|
||||
* **Local/Dev:** Standard output or log files configured in the logging setup section of `ford_qc_box_hotfolder_process.py`.
|
||||
163
01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md
Normal file
163
01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md
Normal file
|
|
@ -0,0 +1,163 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# GMAL Scope Builder Developer Manual
|
||||
|
||||
## Architecture Overview
|
||||
|
||||
The application is a Dockerized, three-service architecture:
|
||||
1. **Frontend**: React + Vite + TypeScript SPA.
|
||||
2. **Backend**: Python FastAPI application handling business logic, AI integration, and database interactions.
|
||||
3. **Database**: PostgreSQL 16 storing project data, GMAL assets, and user information.
|
||||
|
||||
Services communicate via HTTP (Frontend <-> Backend) and network (Backend <-> Database).
|
||||
|
||||
## Tech Stack
|
||||
|
||||
- **Containerization**: Docker & Docker Compose
|
||||
- **Database**: PostgreSQL 16 (Alpine image)
|
||||
- **Backend**: Python, FastAPI, `asyncpg` (async driver), `psycopg2` (sync driver)
|
||||
- **Frontend**: React 18+, Vite, TypeScript, `react-router-dom`, `@azure/msal-react` (Authentication)
|
||||
- **AI**: Anthropic Claude API (Opus 4.6)
|
||||
- **Auth**: Microsoft Entra ID (Azure AD) via MSAL
|
||||
|
||||
## Local Setup
|
||||
|
||||
### Prerequisites
|
||||
- Docker & Docker Compose installed.
|
||||
- Git.
|
||||
- An Active Anthropic API Key.
|
||||
|
||||
### Steps
|
||||
|
||||
1. **Clone & Setup Env**:
|
||||
```bash
|
||||
git clone git@bitbucket.org:zlalani/gmal-scope-builder.git
|
||||
cd gmal-scope-builder
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
- Set `ANTHROPIC_API_KEY`.
|
||||
- Set `POSTGRES_PASSWORD`.
|
||||
- For local dev, set `DEV_AUTH_BYPASS=true` and `VITE_DEV_AUTH_BYPASS=true` to skip Azure login.
|
||||
|
||||
2. **Add Data**:
|
||||
Place the master GMAL Excel file (e.g., `U-Studio GMAL Asset Job Routes Apr25 ForFranky.xlsx`) in the `data/` directory.
|
||||
|
||||
3. **Build & Run**:
|
||||
```bash
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
4. **Wait for Services**:
|
||||
Wait for PostgreSQL to be healthy and Backend to startup.
|
||||
```bash
|
||||
docker compose logs backend --tail 5
|
||||
```
|
||||
|
||||
5. **Ingest GMAL Data**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8001/api/gmal/ingest
|
||||
```
|
||||
|
||||
6. **Access App**:
|
||||
Open `http://localhost:3010`.
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### `.env` (Root)
|
||||
- `POSTGRES_PASSWORD`: Password for the `scope_user` DB user.
|
||||
- `ANTHROPIC_API_KEY`: Key for Claude API access.
|
||||
- `AZURE_TENANT_ID` & `AZURE_CLIENT_ID`: For MSAL authentication.
|
||||
- `DEV_AUTH_BYPASS`: Set to `true` to disable SSO in local dev.
|
||||
- `VITE_DEV_AUTH_BYPASS`: Frontend flag to bypass auth UI.
|
||||
- `DATA_DIR`: Absolute path to directory containing the GMAL Excel file.
|
||||
|
||||
### `docker-compose.yml` (Backend Service)
|
||||
- `DATABASE_URL`: Async SQLAlchemy connection string.
|
||||
- `DATABASE_URL_SYNC`: Sync SQLAlchemy connection string.
|
||||
- Environment variables are passed through from the root `.env`.
|
||||
|
||||
## Key Services & Entry Points
|
||||
|
||||
### Backend
|
||||
- **Entry Point**: Implicit in `./backend` Docker build context (likely `main.py` or `app.py` not shown, but standard FastAPI structure).
|
||||
- **GMAL Ingestion**: Endpoint `POST /api/gmal/ingest`. Reads Excel from `DATA_DIR`, parses assets/roles/hours, and populates the DB.
|
||||
- **AI Integration**: Uses `ANTHROPIC_API_KEY`. Exposes `/ai/usage` and `/ai/debug` endpoints for frontend tracking.
|
||||
- **Projects**: CRUD endpoints at `/projects`.
|
||||
|
||||
### Frontend
|
||||
- **Entry Point**: `frontend/src/main.tsx` -> `App.tsx`.
|
||||
- **Auth**: `frontend/src/auth/AuthProvider.tsx`.
|
||||
- Uses `@azure/msal-react`.
|
||||
- Checks `import.meta.env.VITE_DEV_AUTH_BYPASS` to render raw children if true.
|
||||
- **Routing** (`frontend/src/App.tsx`):
|
||||
- `/`: Dashboard (List Projects).
|
||||
- `/new`: NewProject Form.
|
||||
- `/projects/:id`: ProjectView.
|
||||
- `/gmal`: GMAL Browser.
|
||||
- `/gmal-editor`: GMAL Editor (Admin only).
|
||||
- `/users`: UserManagement (Admin only).
|
||||
- `/help`: Help Page.
|
||||
- **Key Components**:
|
||||
- `Dashboard.tsx`: Displays stats and project list.
|
||||
- `NewProject.tsx`: Form for creating scopes.
|
||||
- `AiCostTracker.tsx`: Polls `/ai/usage` every 5s.
|
||||
- `DebugPanel.tsx`: Polls `/ai/debug` every 3s for LLM debug info.
|
||||
|
||||
## API Reference (Key Endpoints)
|
||||
|
||||
| Method | Path | Description | Auth |
|
||||
| :--- | :--- | :--- | :--- |
|
||||
| POST | `/api/projects` | Create a new project scope | Auth |
|
||||
| GET | `/api/projects` | List all projects | Auth |
|
||||
| POST | `/api/gmal/ingest` | Re-ingest GMAL Excel file | Admin |
|
||||
| GET | `/api/gmal/stats` | Get GMAL asset/role counts | Auth |
|
||||
| GET | `/api/ai/usage` | Get cumulative AI cost/token usage | Internal/Dev |
|
||||
| GET | `/api/ai/debug` | Get recent LLM debug logs | Internal/Dev |
|
||||
|
||||
## Deployment
|
||||
|
||||
### On a Remote Server
|
||||
|
||||
1. **Copy Code & Data**:
|
||||
```bash
|
||||
git clone git@bitbucket.org:zlalani/gmal-scope-builder.git
|
||||
cd gmal-scope-builder
|
||||
# Copy Excel file and DB backup if restoring
|
||||
scp "data/*.xlsx" user@server:/path/to/gmal-scope-builder/data/
|
||||
```
|
||||
|
||||
2. **Configure `.env`**:
|
||||
```bash
|
||||
cp .env.example .env
|
||||
nano .env
|
||||
```
|
||||
Ensure production values for passwords and keys. **Never** set `DEV_AUTH_BYPASS=true` in production.
|
||||
|
||||
3. **Adjust Ports**:
|
||||
Edit `docker-compose.yml` ports if `127.0.0.1:8002:8000`, `5433`, and `3010` conflict with existing services.
|
||||
|
||||
4. **Start**:
|
||||
```bash
|
||||
docker compose build
|
||||
docker compose up -d
|
||||
```
|
||||
|
||||
5. **Initial Ingest**:
|
||||
```bash
|
||||
curl -X POST http://localhost:8001/api/gmal/ingest # Or your mapped port
|
||||
```
|
||||
|
||||
## Known Gotchas
|
||||
|
||||
1. **Double-Counting Efficiency**: When using "AI efficiency" model types in the frontend, do not manually apply additional AI efficiency adjustments. The model type already accounts for this.
|
||||
2. **Data Directory Path**: The backend mounts `${DATA_DIR:-./data}`. Ensure this directory exists and is writable by the container user, and contains the valid Excel file. Relative paths in `.env` are resolved relative to the repo root.
|
||||
3. **Auth Bypass**: `DEV_AUTH_BYPASS` and `VITE_DEV_AUTH_BYPASS` must remain empty/false in production to ensure security.
|
||||
4. **AI Cost**: The AI costs can accumulate quickly during development/debugging due to frequent calls. Monitor the `AiCostTracker` widget.
|
||||
5. **Database Restore**: If restoring from `scope_builder_deploy.sql`, ensure the DB user and database name (`scope_builder`, `scope_user`) match the environment variables.
|
||||
6. **Excel Format**: The GMAL ingestion logic depends on the specific structure of the "U-Studio GMAL Asset Job Routes..." Excel file. Changes to this file structure will break ingestion unless the backend parser is updated.
|
||||
86
01 Projects/gmal-scope-builder/USER_MANUAL.md
Normal file
86
01 Projects/gmal-scope-builder/USER_MANUAL.md
Normal file
|
|
@ -0,0 +1,86 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# GMAL Scope Builder User Manual
|
||||
|
||||
## What This Tool Does
|
||||
|
||||
The GMAL Scope Builder is a web application designed for agencies to efficiently scope new client ratecards and team structures. It works by matching client briefs against a standardized master asset database (GMAL) using AI (Claude Opus 4.6).
|
||||
|
||||
Key capabilities include:
|
||||
- **Automated Asset Matching**: AI matches client requirements to standardized GMAL assets.
|
||||
- **Ratecard Building**: Automatically calculates hours per role based on matched assets.
|
||||
- **Team Shaping**: Calculates team Full-Time Equivalent (FTE) headcount.
|
||||
- **Cost Tracking**: Monitors AI usage costs and token consumption.
|
||||
|
||||
## Who Uses It
|
||||
|
||||
- **Project Managers/Scopers**: Primary users who create new projects, input client briefs, and review AI-generated scopes.
|
||||
- **Administrators**: Users with 'admin' role who can manage GMAL data (ingest new master files), edit the GMAL database, and manage user accounts.
|
||||
- **Viewers**: Users who can view existing projects and GMAL assets but cannot create new scopes or modify data.
|
||||
|
||||
## How to Access
|
||||
|
||||
1. Navigate to the application URL in your web browser.
|
||||
2. **Authentication**: You will be redirected to the Microsoft login page. Sign in with your Oliver Agency Microsoft account.
|
||||
- *Note*: Local development environments can bypass SSO by setting `VITE_DEV_AUTH_BYPASS=true` in the `.env` file.
|
||||
3. Once logged in, you will see the **Dashboard** (Projects list).
|
||||
|
||||
## Main Workflows
|
||||
|
||||
### Workflow 1: Creating a New Project Scope
|
||||
|
||||
1. **Navigate**: Click the **"+ New Project"** button on the Dashboard (available to Editors/Admins).
|
||||
2. **Enter Details**:
|
||||
- **Project Name**: e.g., "Acme Corp Q2 2025 Scope".
|
||||
- **Client Name**: The client's name.
|
||||
- **Description**: Paste or type the client brief/requirements.
|
||||
- **Model Type**: Select the appropriate AI model type from the dropdown.
|
||||
- *Tip*: If selecting an "AI efficiency" model type, do not apply additional AI efficiency profiles to avoid double-counting hours.
|
||||
3. **Submit**: Click **"Create Project"**. You will be redirected to the **Project View** page.
|
||||
4. **Review**: The AI will process the brief. Review the matched assets, suggested hours, and team composition.
|
||||
|
||||
### Workflow 2: Viewing Projects
|
||||
|
||||
1. **Dashboard**: View all your projects with their current status (Draft, Parsing, Matching, Review, Building, Finalized).
|
||||
2. **Status Indicators**:
|
||||
- Green: Finalized
|
||||
- Blue: Matching/Parsing
|
||||
- Yellow: Draft/Building
|
||||
- Grey: Draft (Muted)
|
||||
3. **Click a Project**: Opens the detailed **Project View** to inspect or edit the scope.
|
||||
|
||||
### Workflow 3: Managing GMAL Data (Admins Only)
|
||||
|
||||
1. **Ingesting New Data**:
|
||||
- Click **"Ingest GMAL Data"** on the Dashboard.
|
||||
- Confirm the action. This reloads all assets from the master Excel file.
|
||||
- *Warning*: This process updates the entire database. Ensure the correct Excel file is in the `data/` directory.
|
||||
2. **Editing GMAL Assets**:
|
||||
- Navigate to **"GMAL Editor"** (Admin only).
|
||||
- Use the interface to search, view, and modify assets, roles, and hourly records in the GMAL master database.
|
||||
|
||||
### Workflow 4: Browsing Assets
|
||||
|
||||
1. Navigate to **"GMAL Browser"** from the sidebar.
|
||||
2. Search the standardized asset database to understand available resources and standard rates.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Why does the AI cost tracker show increasing costs?**
|
||||
A: The app uses Claude Opus 4.6 for AI matching. Costs accumulate based on input/output tokens and the number of API calls. You can monitor this in real-time via the "AI Cost" widget in the app.
|
||||
|
||||
**Q: Can I edit the client brief after creation?**
|
||||
A: Yes, on the Project View page, you can modify the description or other parameters and re-run the AI matching process.
|
||||
|
||||
**Q: What is the difference between "Model Type" options?**
|
||||
A: Different model types account for different levels of AI efficiency in the hourly rates. Some models have AI efficiency built-in; others do not. Selecting the correct type prevents double-counting efficiency gains.
|
||||
|
||||
**Q: I don't see the "Admin" menu items.**
|
||||
A: You need the 'admin' role assigned to your user account. Contact your system administrator if you believe you should have access.
|
||||
|
||||
**Q: How is FTE calculated?**
|
||||
A: FTE is calculated based on the total hours required for all matched assets, divided by the standard annual working hours for a full-time employee.
|
||||
124
01 Projects/olivas/DEVELOPER_MANUAL.md
Normal file
124
01 Projects/olivas/DEVELOPER_MANUAL.md
Normal file
|
|
@ -0,0 +1,124 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# OliVAS Developer Manual
|
||||
|
||||
## Architecture Overview
|
||||
OliVAS is a full-stack web application with a clear separation between frontend and backend services, managed via Docker Compose.
|
||||
|
||||
- **Frontend**: React 18 SPA with TypeScript, Vite, Tailwind CSS, Zustand (state), and React Router.
|
||||
- **Backend**: FastAPI (Python) with async SQLAlchemy and Pydantic v2. Handles API logic, file processing, and ML model inference.
|
||||
- **Database**: PostgreSQL 16 (via Docker).
|
||||
- **Auth**: Azure AD SSO via MSAL.
|
||||
- **ML**: Local GPU/CPU inference or offloaded to Google Cloud Run.
|
||||
|
||||
## Tech Stack
|
||||
| Layer | Technology |
|
||||
|-------|------------|
|
||||
| **Frontend** | React 18, TypeScript, Vite, Tailwind CSS, Zustand, React Router |
|
||||
| **Backend** | FastAPI, Python 3.12, SQLAlchemy (async), Pydantic v2 |
|
||||
| **Database** | PostgreSQL 16 |
|
||||
| **Auth** | Azure AD (MSAL) |
|
||||
| **ML** | DeepGaze Models, Optional: Anthropic API |
|
||||
|
||||
## Local Setup
|
||||
|
||||
### Prerequisites
|
||||
- Docker and Docker Compose
|
||||
- Python 3.12+ (for local backend dev if not using container)
|
||||
|
||||
### Step-by-Step
|
||||
1. **Clone the Repository**
|
||||
```bash
|
||||
git clone <repo-url>
|
||||
cd olivas
|
||||
```
|
||||
|
||||
2. **Configure Environment**
|
||||
Copy `.env.example` to `.env`:
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
3. **Start Services**
|
||||
```bash
|
||||
docker-compose up --build
|
||||
```
|
||||
This starts:
|
||||
- `postgres` on port `5453`
|
||||
- `backend` on port `8000`
|
||||
- `frontend` on port `1577`
|
||||
|
||||
4. **Access the App**
|
||||
- Frontend: `http://localhost:1577/olivas`
|
||||
- Backend API: `http://localhost:8000/docs`
|
||||
|
||||
## Environment Variables
|
||||
|
||||
### General
|
||||
- `DATABASE_URL`: PostgreSQL connection string (e.g., `postgresql+asyncpg://olivas:olivas@localhost:5453/olivas`)
|
||||
- `UPLOAD_DIR`: Local path for storing uploaded images (e.g., `./data/uploads`)
|
||||
- `DEVICE`: ML device (`auto`, `cpu`, `cuda`)
|
||||
- `CORS_ORIGINS`: Allowed frontend origins (e.g., `http://localhost:1577`)
|
||||
- `BACKEND_HOST/PORT`: Server binding settings
|
||||
|
||||
### Authentication (Azure AD)
|
||||
- `AZURE_AUTH_ENABLED`: `true`/`false`
|
||||
- `AZURE_TENANT_ID`: Your Azure AD Tenant ID
|
||||
- `AZURE_CLIENT_ID`: Your Azure AD Application Client ID
|
||||
- `VITE_AZURE_*`: Frontend Azure AD config (Tenant, Client, Redirect URI)
|
||||
|
||||
### Optional Services
|
||||
- `ANTHROPIC_API_KEY`: Enable AI Design Analysis via Claude Sonnet 4.6
|
||||
- `CLOUD_RUN_*`: Enable Google Cloud Run offloading for ML models
|
||||
|
||||
## Key Services & Entry Points
|
||||
|
||||
### Frontend (`frontend/src/`)
|
||||
- **`App.tsx`**: Route definitions. All routes are protected by `RequireAuth`.
|
||||
- **`auth/RequireAuth.tsx`**: Checks MSAL authentication status. Redirects to Login if unauthenticated.
|
||||
- **`auth/msalConfig.ts`**: MSAL instance configuration.
|
||||
- **`components/AuthImage.tsx`**: Helper to fetch images via authenticated API requests and convert to Blob URLs.
|
||||
|
||||
### Backend (`backend/`)
|
||||
- **Main Entry**: FastAPI app initialization (see `main.py` or `app.py` in backend dir).
|
||||
- **API Client**: Frontend uses a configured axios/fetch client with baseURL `/api`.
|
||||
- **ML Inference**:
|
||||
- Local: Uses DeepGaze models based on `DEVICE` env var.
|
||||
- Cloud: Sends requests to `CLOUD_RUN_SALIENCY_URL` if configured.
|
||||
|
||||
## API Reference
|
||||
|
||||
### Authentication
|
||||
- **Login**: Azure AD Login Redirect. Handled by MSAL.
|
||||
- **Protected Endpoints**: All API endpoints require a valid Azure AD bearer token.
|
||||
|
||||
### Endpoints (Examples)
|
||||
- `GET /api/projects`: List all projects.
|
||||
- `POST /api/projects`: Create a project.
|
||||
- `POST /api/analyses`: Trigger analysis on an uploaded image.
|
||||
- `GET /api/analyses/{id}`: Get analysis details (heatmap URLs, metrics, insights).
|
||||
- `GET /api/analyses/{id}/image`: Retrieve original image (authenticated blob).
|
||||
|
||||
## Deployment
|
||||
|
||||
### Production Considerations
|
||||
1. **HTTPS**: Ensure reverse proxy (Nginx/Traefik) handles SSL termination.
|
||||
2. **CORS**: Update `CORS_ORIGINS` to include production domain.
|
||||
3. **Secrets**: Do not commit `.env` files. Use CI/CD secrets or Vault.
|
||||
4. **Storage**: Map `UPLOAD_DIR` to persistent cloud storage (e.g., S3) in production.
|
||||
5. **ML Acceleration**: Set `DEVICE=cuda` and ensure GPU drivers are available on the host.
|
||||
|
||||
### Google Cloud Run Offload
|
||||
- Set `CLOUD_RUN_SALIENCY_URL` and `CLOUD_RUN_PROCESSING_URL` to point to deployed Cloud Run services.
|
||||
- Set `GOOGLE_CLOUD_PROJECT`.
|
||||
|
||||
## Known Gotchas
|
||||
1. **MSAL Redirect Issues**: Ensure `VITE_AZURE_REDIRECT_URI` matches the Azure AD portal configuration exactly. Note the `/olivas` suffix for the frontend basename.
|
||||
2. **Image Loading**: Images stored on the server must be fetched via `/api/analyses/{id}/image` to include auth tokens. Do not use direct filesystem paths in production.
|
||||
3. **Database Connection**: The backend uses asyncpg. Ensure `DATABASE_URL` uses the `postgresql+asyncpg://` dialect.
|
||||
4. **CORS**: During development, ensure `CORS_ORIGINS` includes `http://localhost:1577`. In production, specify your actual domain.
|
||||
5. **AI Insights Disabled**: If `ANTHROPIC_API_KEY` is empty, the AI Insights tab will show a message indicating AI analysis is unavailable.
|
||||
102
01 Projects/olivas/USER_MANUAL.md
Normal file
102
01 Projects/olivas/USER_MANUAL.md
Normal file
|
|
@ -0,0 +1,102 @@
|
|||
---
|
||||
auto_generated: true
|
||||
manual_updated_at: 2026-05-18
|
||||
modified: 2026-05-18
|
||||
---
|
||||
|
||||
# OliVAS User Manual
|
||||
|
||||
## What This Tool Does
|
||||
OliVAS (OLIVER Visual Attention Suite) is an open-source web application designed for creative teams, designers, and marketers. It predicts where human eyes will look in an image during the first 3-5 seconds of viewing.
|
||||
|
||||
Instead of using physical eye-tracking hardware, OliVAS uses advanced machine learning models to generate:
|
||||
- **Saliency Heatmaps**: Visual overlays showing predicted attention intensity.
|
||||
- **Gaze Sequence Predictions**: Numbered points showing the predicted order of scanning.
|
||||
- **Hotspot Detection**: Identification of the top 5 attention regions.
|
||||
- **AI Design Analysis**: Optional insights powered by Claude Sonnet 4.6, referencing specific visual elements to provide actionable recommendations.
|
||||
|
||||
## Who Uses It
|
||||
- **Designers**: To validate layout hierarchy and focal points.
|
||||
- **Marketers**: To optimize ad creatives for maximum engagement.
|
||||
- **Creative Directors**: To provide data-driven feedback on visual assets.
|
||||
|
||||
## How to Access
|
||||
1. Navigate to the OliVAS application URL.
|
||||
2. Click **"Sign in with Microsoft"**.
|
||||
3. If you do not have an Oliver agency account, access will be denied. Contact your IT administrator if you believe you should have access.
|
||||
|
||||
## Main Workflows
|
||||
|
||||
### 1. Creating a Project
|
||||
1. Log in to the Dashboard.
|
||||
2. Click **"Create Project"** in the top right corner.
|
||||
3. Enter a **Project Name** and optional **Description**.
|
||||
4. Click **"Create"**.
|
||||
5. You will be redirected to the Project Detail page.
|
||||
|
||||
### 2. Analyzing an Image
|
||||
1. From the **Dashboard** or **Project Detail** page, click **"Analyze"** or **"Create New Analysis"**.
|
||||
2. Upload an image file.
|
||||
3. Select a **Model**:
|
||||
- **DeepGaze I**: Fast, baseline saliency.
|
||||
- **DeepGaze IIE**: **Recommended** for best accuracy.
|
||||
- **DeepGaze III**: High-resolution detail.
|
||||
4. Click **"Analyze"**.
|
||||
5. Wait for the analysis to complete. You will be redirected to the **Analysis View**.
|
||||
|
||||
### 3. Interpreting Results (Analysis View)
|
||||
Once on the Analysis View page, you can explore several tabs:
|
||||
|
||||
#### Heatmap Tab
|
||||
- Toggle **Opacity** to adjust the visibility of the saliency heatmap overlay.
|
||||
- Select a **Colormap** (Jet, Viridis, Inferno, etc.) to change how intensity is displayed.
|
||||
- The red/hot areas indicate high predicted attention.
|
||||
|
||||
#### Gaze Sequence Tab
|
||||
- View **numbered fixation points** (1, 2, 3...) showing the predicted order viewers will scan the image.
|
||||
- Use this to understand the visual path created by your design.
|
||||
|
||||
#### Hotspots Tab
|
||||
- See the **top 5 attention regions** ranked by intensity.
|
||||
- Bounding boxes highlight these key areas.
|
||||
|
||||
#### AOI (Areas of Interest)
|
||||
- Draw rectangles over specific design elements.
|
||||
- The tool calculates **Attention %** (how much attention the element gets), **Area %** (size relative to image), and **Attention Density** for the selected area.
|
||||
|
||||
#### Insights Tab
|
||||
- **Rule-Based Insights**: Automatic metrics including:
|
||||
- **Attention Score**: 0-100 concentration score.
|
||||
- **Focal Dominance**: How much one area dominates the view.
|
||||
- **Spatial Balance**: Distribution of attention across the image.
|
||||
- **Edge Risk**: Potential for attention drifting off the edges.
|
||||
- **AI Design Analysis**: If enabled, view text-based insights from Claude Sonnet 4.6. These reference specific visual elements and provide actionable design recommendations.
|
||||
|
||||
#### PDF Report
|
||||
- Click **"Download PDF Report"** to generate a professional document containing:
|
||||
- All visualizations (heatmaps, gaze sequences).
|
||||
- Metrics (Hotspots, AOI data).
|
||||
- Rule-based and AI insights.
|
||||
- Montserrat typography for readability.
|
||||
|
||||
### 4. Comparing Designs
|
||||
1. Navigate to two different analyses.
|
||||
2. Use the **Comparison View** feature.
|
||||
3. View side-by-side metrics and heatmaps to evaluate performance differences.
|
||||
|
||||
## FAQ
|
||||
|
||||
**Q: Why is my analysis taking a long time?**
|
||||
A: Depending on the model selected (DeepGaze IIE/III) and server load, analysis may take several seconds. DeepGaze IIE is recommended for a balance of speed and accuracy.
|
||||
|
||||
**Q: Can I use OliVAS for external client work?**
|
||||
A: Yes, if you have an Oliver agency account. Ensure you comply with your organization's data privacy policies regarding client images.
|
||||
|
||||
**Q: What is the "Attention Score"?**
|
||||
A: It is a 0-100 metric measuring how focused the predicted attention is. A higher score (e.g., >55) indicates a strong focal point, while a lower score suggests a diffuse or scattered gaze pattern.
|
||||
|
||||
**Q: How do I get AI Insights?**
|
||||
A: AI Insights are optional and require an Anthropic API key to be configured in the backend. If available, they will appear in the Insights tab. If not, only rule-based insights will be shown.
|
||||
|
||||
**Q: Can I delete an analysis?**
|
||||
A: Yes, from the Analysis View or Project Detail page, use the **"Delete"** button. This action is usually irreversible.
|
||||
|
|
@ -23,7 +23,7 @@ This 3-hop pattern works for hundreds of articles without vector search.
|
|||
| [[wiki/tech-patterns/_index\|tech-patterns/]] | Recurring tech stacks: FastAPI, React/Vite, Next.js, Azure AD, AI, Box, One2Edit, Redis/Celery, cost-tracker, OMG API, Payload CMS | 39 |
|
||||
| [[wiki/architecture/_index\|architecture/]] | Cross-cutting architectural patterns: Docker Compose, multi-agent AI, GCP timeout, RAG, hotfolder, optical-dev deploy, cost-tracker, new-project checklist, troubleshooting playbooks, ADR log, Cloud Run Jobs | 53 |
|
||||
| [[wiki/client-knowledge/_index\|client-knowledge/]] | Per-client notes for Ford, H&M, L'Oréal, Barclays, Ferrero, 3M, BAIC | 7 |
|
||||
| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 202 |
|
||||
| [[wiki/concepts/_index\|concepts/]] | Atomic knowledge extracted from Claude Code sessions | 204 |
|
||||
| [[wiki/connections/_index\|connections/]] | Cross-cutting insights linking 2+ concepts: FastAPI+Azure AD+Docker trinity, AI→cost-tracker, Apache+Vite basePath, GCP→REST polling, Box+hotfolder, Docker DNS+AdGuard, Celery prefork×faster_whisper memory stacking | 10 |
|
||||
| [[wiki/qa/_index\|qa/]] | Filed answers to queries (saved with `--file-back`) | 0 |
|
||||
| [[wiki/homelab/_index\|homelab/]] | Self-hosted infra: Proxmox install, IOMMU/PCI passthrough, hypervisor setup, budget builds, HP Elitedesk G3, Homarr API + Apps + Boards + Certificates + Integrations + Settings + Tasks + AdGuard + Clock + Docker Stats + Docker Integration + Download Client + Firewall + Proxmox Integration + Radarr + Readarr + Sonarr + Bookmarks + Calendar + Icons + App Widget + Weather + GitHub + Nextcloud + qBittorrent + RSS Feed + Speedtest Tracker + System Health Monitoring + System Resources + Services Map + Media Stack | 43 |
|
||||
|
|
|
|||
|
|
@ -2,14 +2,14 @@
|
|||
title: "Concepts Index"
|
||||
description: "Atomic knowledge extracted from Claude Code sessions — gotchas, bugs, workarounds"
|
||||
tags: [index, concepts]
|
||||
updated: 2026-05-15
|
||||
updated: 2026-05-18
|
||||
---
|
||||
|
||||
# Concepts — Topic Index
|
||||
|
||||
Atomic knowledge articles from real Oliver Agency sessions. Each article = one specific gotcha, bug, or non-obvious behaviour discovered in production or testing.
|
||||
|
||||
**202 articles** — use Obsidian search or `grep` to find by keyword.
|
||||
**204 articles** — use Obsidian search or `grep` to find by keyword.
|
||||
|
||||
| Article | Title | Created |
|
||||
|---------|-------|---------|
|
||||
|
|
@ -39,6 +39,7 @@ Atomic knowledge articles from real Oliver Agency sessions. Each article = one s
|
|||
| [[wiki/concepts/celery-redis-queue-flush-on-deterministic-error\|celery-redis-queue-flush-on-deterministic-error]] | Celery + Redis: Must Flush Redis on Deterministic Errors | 2026-04-30 |
|
||||
| [[wiki/concepts/celery-worker-restart-inflight-task-loss\|celery-worker-restart-inflight-task-loss]] | Celery Worker Restart Orphans In-Flight Tasks | 2026-05-08 |
|
||||
| [[wiki/concepts/chartjs-time-axis-adapter\|chartjs-time-axis-adapter]] | Chart.js — type:time Axis Requires a Date Adapter | 2026-04-21 |
|
||||
| [[wiki/concepts/chromium-pdf-css-font-injection\|chromium-pdf-css-font-injection]] | Chromium PDF — @font-face Must Be in HTML <head> for CIDFont Embedding | 2026-05-18 |
|
||||
| [[wiki/concepts/claude-code-plugin-marketplace\|claude-code-plugin-marketplace]] | Claude Code Plugin Marketplace Install Pattern | 2026-04-29 |
|
||||
| [[wiki/concepts/claude-code-schedule-skill-account-type\|claude-code-schedule-skill-account-type]] | Claude Code — /schedule Skill Requires claude.ai Login Account | 2026-04-24 |
|
||||
| [[wiki/concepts/cline-lm-studio-openai-compatible\|cline-lm-studio-openai-compatible]] | Cline + LM Studio: Use OpenAI Compatible Provider | 2026-04-30 |
|
||||
|
|
@ -179,6 +180,7 @@ Atomic knowledge articles from real Oliver Agency sessions. Each article = one s
|
|||
| [[wiki/concepts/pydantic-v2-alias-id-gotcha\|pydantic-v2-alias-id-gotcha]] | Pydantic v2 — Field(alias='_id') Serializes as _id, Breaking Frontend .id Access | 2026-04-27 |
|
||||
| [[wiki/concepts/pydub-ffmpeg-silent-dependency\|pydub-ffmpeg-silent-dependency]] | pydub Silent ffmpeg Dependency in Docker | 2026-04-30 |
|
||||
| [[wiki/concepts/python-fastapi-module-level-singletons\|python-fastapi-module-level-singletons]] | Module-Level Singletons Break pytest — Use Lazy Initialisation | 2026-04-30 |
|
||||
| [[wiki/concepts/python-float-formatting-comparison\|python-float-formatting-comparison]] | Python Float — Equality Trap When Formatting Percentages (1.1×100≠110) | 2026-05-18 |
|
||||
| [[wiki/concepts/python-iso-z-suffix\|python-iso-z-suffix]] | Python — fromisoformat() Cannot Parse Z Suffix (Python < 3.11) | 2026-04-24 |
|
||||
| [[wiki/concepts/python-service-deployment-dotenv\|python-service-deployment-dotenv]] | Python Service Deployment — venv and .env Checklist | 2026-04-16 |
|
||||
| [[wiki/concepts/qbittorrent-proxy-profile-blocking\|qbittorrent-proxy-profile-blocking]] | qBittorrent Proxy Profile Enabled Without Proxy Host Blocks All Connections | 2026-05-03 |
|
||||
|
|
|
|||
78
wiki/concepts/chromium-pdf-css-font-injection.md
Normal file
78
wiki/concepts/chromium-pdf-css-font-injection.md
Normal file
|
|
@ -0,0 +1,78 @@
|
|||
---
|
||||
title: "Chromium PDF — CSS Font-Face Must Be in HTML <head>, Not SVG <defs>"
|
||||
tags: [chromium, pdf, css, fonts, svg, inkscape, indesign]
|
||||
created: 2026-05-18
|
||||
sources: daily/2026-05-18.md
|
||||
---
|
||||
|
||||
# Chromium PDF — CSS Font-Face Must Be in HTML `<head>`, Not SVG `<defs>`
|
||||
|
||||
## The Problem
|
||||
|
||||
When Chromium renders an SVG to PDF via `page.pdf()` (Playwright) or `--print-to-pdf` (headless), `@font-face` rules placed only inside the SVG's `<defs>` block produce **Type 3 (bitmap) fonts** in the PDF:
|
||||
|
||||
```xml
|
||||
<!-- SVG — @font-face here ONLY → bitmap Type 3 font in PDF -->
|
||||
<svg xmlns="http://www.w3.org/2000/svg">
|
||||
<defs>
|
||||
<style>
|
||||
@font-face {
|
||||
font-family: "CustomFont";
|
||||
src: url("CustomFont.woff2") format("woff2");
|
||||
}
|
||||
</style>
|
||||
</defs>
|
||||
<text font-family="CustomFont">Hello</text>
|
||||
</svg>
|
||||
```
|
||||
|
||||
**Symptom:** PDF opens fine in browsers but shows rasterised/pixelated text in InDesign, Acrobat, or print-preflight; font is listed as Type 3 (not TrueType/CIDFont).
|
||||
|
||||
## Root Cause
|
||||
|
||||
Chromium's PDF renderer uses the HTML document's font-loading pipeline, not the SVG inline stylesheet, to decide which fonts to embed as vector outlines. `@font-face` inside SVG `<defs>` is applied for on-screen rendering but is **not** picked up by the PDF font-embedding step.
|
||||
|
||||
## Fix
|
||||
|
||||
Inject the same `@font-face` declaration into the **HTML document's `<head>`** before calling `page.pdf()`:
|
||||
|
||||
```python
|
||||
# Playwright example
|
||||
await page.set_content(f"""
|
||||
<!DOCTYPE html>
|
||||
<html>
|
||||
<head>
|
||||
<style>
|
||||
@font-face {{
|
||||
font-family: "CustomFont";
|
||||
src: url("file:///path/to/CustomFont.woff2") format("woff2");
|
||||
}}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
{svg_content}
|
||||
</body>
|
||||
</html>
|
||||
""")
|
||||
await page.pdf(path="output.pdf", print_background=True)
|
||||
```
|
||||
|
||||
The SVG `<defs>` stylesheet can remain (no need to remove it); the HTML `<head>` declaration is additive and triggers proper CIDFont embedding.
|
||||
|
||||
## Verification
|
||||
|
||||
Open the resulting PDF in Acrobat → File → Properties → Fonts tab. The font should appear as **Type: TrueType (CID)** or **Type: OpenType (CID)**, not Type 3.
|
||||
|
||||
## Scope
|
||||
|
||||
Applies to:
|
||||
- Playwright `page.pdf()`
|
||||
- Puppeteer `page.pdf()`
|
||||
- Chrome headless `--print-to-pdf`
|
||||
- Any Chromium-based PDF renderer
|
||||
|
||||
Does **not** apply to wkhtmltopdf (uses QtWebKit) or WeasyPrint (uses Cairo).
|
||||
|
||||
## See Also
|
||||
|
||||
- `wiki/concepts/chromium-pdf-css-font-injection` — this article
|
||||
64
wiki/concepts/python-float-formatting-comparison.md
Normal file
64
wiki/concepts/python-float-formatting-comparison.md
Normal file
|
|
@ -0,0 +1,64 @@
|
|||
---
|
||||
title: "Python Float Formatting — Equality Trap with Percentages"
|
||||
tags: [python, float, formatting, gotcha]
|
||||
created: 2026-05-18
|
||||
sources: daily/2026-05-18.md
|
||||
---
|
||||
|
||||
# Python Float Formatting — Equality Trap with Percentages
|
||||
|
||||
## The Problem
|
||||
|
||||
Binary floating-point arithmetic breaks integer-equality checks on values that look whole:
|
||||
|
||||
```python
|
||||
1.1 * 10 # → 11.000000000000002
|
||||
(1.1 * 10) % 1 # → 2e-15 (not 0.0!)
|
||||
(1.1 * 10) % 1 == 0 # → False ← BUG
|
||||
```
|
||||
|
||||
A common context where this bites: formatting percentage strings.
|
||||
|
||||
```python
|
||||
def fmt_pct(value):
|
||||
"""Format 1.1 → '1.1%', 2.0 → '2%'"""
|
||||
pct = value * 100
|
||||
if pct % 1 == 0: # WRONG — floats lie here
|
||||
return f"{int(pct)}%" # never reached for 1.1
|
||||
return f"{pct:.2f}%" # outputs "1.10%" instead of "1.1%"
|
||||
```
|
||||
|
||||
**Symptom:** Values like `1.1`, `3.3`, `6.6` produce `"1.10%"`, `"3.30%"`, `"6.60%"` instead of `"1.1%"`, `"3.3%"`, `"6.6%"`.
|
||||
|
||||
## Root Cause
|
||||
|
||||
IEEE 754 double precision cannot represent most decimal fractions exactly. Multiplying by 100 amplifies the error. The `% 1 == 0` check compares against exact zero — which the fractional residue never is.
|
||||
|
||||
## Fix
|
||||
|
||||
Format to a fixed number of decimal places first, then strip trailing zeros:
|
||||
|
||||
```python
|
||||
def fmt_pct(value):
|
||||
pct = value * 100
|
||||
formatted = f"{pct:.10f}".rstrip("0").rstrip(".")
|
||||
return f"{formatted}%"
|
||||
```
|
||||
|
||||
Or use `Decimal` for exact arithmetic when the source is user-entered text:
|
||||
|
||||
```python
|
||||
from decimal import Decimal
|
||||
def fmt_pct(value: str) -> str:
|
||||
pct = Decimal(value) * 100
|
||||
return f"{pct.normalize():f}%"
|
||||
```
|
||||
|
||||
## General Rule
|
||||
|
||||
> Never use `float == integer` or `float % 1 == 0` for "is this whole?" checks.
|
||||
> Always format to fixed dp first, then apply string operations.
|
||||
|
||||
## See Also
|
||||
|
||||
- `wiki/concepts/python-iso-z-suffix` — another float/string representation trap
|
||||
10
wiki/log.md
10
wiki/log.md
|
|
@ -1,6 +1,16 @@
|
|||
|
||||
# Build Log
|
||||
|
||||
## [2026-05-18T21:30:00+01:00] compile | daily/2026-05-18.md — pass 3 (session 18:59 pimco-charts extraction)
|
||||
- Source: daily/2026-05-18.md (session 18:59 — pimco-charts PDF rendering + chart generation)
|
||||
- Articles created (2):
|
||||
- [[wiki/concepts/python-float-formatting-comparison]] — `1.1 * 100 ≠ 110` in binary float; `% 1 == 0` returns False; fix: format to fixed dp then `rstrip("0")`
|
||||
- [[wiki/concepts/chromium-pdf-css-font-injection]] — `@font-face` in SVG `<defs>` only → Type 3 bitmap fonts in PDF; must inject same CSS into HTML `<head>` to get CIDFont (TrueType) embedding
|
||||
- Mistakes appended (1):
|
||||
- [[wiki/mistakes/general]] — 2026-05-18 float-percent-formatting: `"1.10%"` instead of `"1.1%"` due to float arithmetic; fix: rstrip pattern
|
||||
- Index updates: concepts/_index.md (202→204, +2 entries), _master-index.md (concepts 202→204)
|
||||
- Note: gemini.md mistake and gemini-preview-vs-stable-rate-limits concept were already compiled in pass 2 (13:18 session); this pass covers the second session only
|
||||
|
||||
## [2026-05-18T21:00:00+01:00] compile | daily/2026-05-18.md — pass 2 (session 13:18 extraction)
|
||||
- Source: daily/2026-05-18.md (session 13:18 — BAIC Gemini 504/429 debugging)
|
||||
- Articles created (1):
|
||||
|
|
|
|||
|
|
@ -11,3 +11,18 @@ Running list. Newest first. Auto-populated from session flush.
|
|||
---
|
||||
|
||||
<!-- mistakes appended below by compile.py -->
|
||||
|
||||
## 2026-05-18 — float-percent-formatting
|
||||
|
||||
**Mistake:** Output `"1.10%"` instead of `"1.1%"` for percentage values like `1.1`.
|
||||
|
||||
**Symptom:** Formatted percentage strings showed unnecessary trailing zeros (e.g. `"3.30%"`, `"6.60%"`) even for values that are exact to one decimal place.
|
||||
|
||||
**Root cause:** `1.1 * 100 = 110.00000000000001` in binary float; `% 1 == 0` returns `False`; the "whole number" branch was never taken, falling through to `:.2f` formatting.
|
||||
|
||||
**Fix:** Format to fixed dp first, then `rstrip("0").rstrip(".")`:
|
||||
```python
|
||||
f"{value * 100:.10f}".rstrip("0").rstrip(".")
|
||||
```
|
||||
|
||||
**See also:** [[wiki/concepts/python-float-formatting-comparison]]
|
||||
|
|
|
|||
Loading…
Add table
Reference in a new issue