From 2ccb21102f7b2ba46af71dc2067df68608887fb1 Mon Sep 17 00:00:00 2001 From: Vadym Samoilenko Date: Tue, 19 May 2026 12:01:03 +0100 Subject: [PATCH] vault backup: 2026-05-19 12:01:03 --- 01 Projects/APAC ops Bot/DEVELOPER_MANUAL.md | 1 + 01 Projects/ac-helper/DEVELOPER_MANUAL.md | 3 + 01 Projects/ac-helper/USER_MANUAL.md | 1 + 01 Projects/ac-tool/DEVELOPER_MANUAL.md | 3 + 01 Projects/ac-tool/USER_MANUAL.md | 6 +- .../amazon-transcreation/DEVELOPER_MANUAL.md | 1 + .../baic_dashboard/DEVELOPER_MANUAL.md | 2 + .../barclays-rag-report/DEVELOPER_MANUAL.md | 164 +++++++++++++++ .../barclays-rag-report/USER_MANUAL.md | 115 +++++++++++ 01 Projects/build-a-squad/USER_MANUAL.md | 1 + 01 Projects/cc-dashboard/DEVELOPER_MANUAL.md | 8 +- .../DEVELOPER_MANUAL.md | 1 + .../cinema-studio-pro-kling/USER_MANUAL.md | 1 + .../cinema-studio-pro/DEVELOPER_MANUAL.md | 1 + 01 Projects/cinema-studio-pro/USER_MANUAL.md | 1 + .../ferrero-ac-creator/DEVELOPER_MANUAL.md | 1 + 01 Projects/ferrero-ac-creator/USER_MANUAL.md | 1 + 01 Projects/ford_qc/DEVELOPER_MANUAL.md | 1 + 01 Projects/ford_qc/USER_MANUAL.md | 1 + .../gmal-scope-builder/DEVELOPER_MANUAL.md | 8 +- 01 Projects/gmal-scope-builder/USER_MANUAL.md | 1 + .../hp-prod-tracker/DEVELOPER_MANUAL.md | 3 + .../loreal-sla-calculator/USER_MANUAL.md | 1 + 01 Projects/lusa-back-planner/USER_MANUAL.md | 6 +- 01 Projects/md to word/DEVELOPER_MANUAL.md | 6 +- 01 Projects/md to word/USER_MANUAL.md | 1 + 01 Projects/modcomms/DEVELOPER_MANUAL.md | 192 ++++++++++-------- 01 Projects/modcomms/USER_MANUAL.md | 104 ++++------ 01 Projects/olivas/DEVELOPER_MANUAL.md | 3 + .../DEVELOPER_MANUAL.md | 3 + 01 Projects/pimco-charts/DEVELOPER_MANUAL.md | 178 ++++++++-------- 01 Projects/pimco-charts/USER_MANUAL.md | 117 +++++------ .../DEVELOPER_MANUAL.md | 7 +- .../DEVELOPER_MANUAL.md | 2 + 99 Daily/2026-05-19.md | 1 + 35 files changed, 644 insertions(+), 302 deletions(-) create mode 100644 01 Projects/barclays-rag-report/DEVELOPER_MANUAL.md create mode 100644 01 Projects/barclays-rag-report/USER_MANUAL.md diff --git a/01 Projects/APAC ops Bot/DEVELOPER_MANUAL.md b/01 Projects/APAC ops Bot/DEVELOPER_MANUAL.md index 6339168..e384b6a 100644 --- a/01 Projects/APAC ops Bot/DEVELOPER_MANUAL.md +++ b/01 Projects/APAC ops Bot/DEVELOPER_MANUAL.md @@ -138,5 +138,6 @@ The following environment variables are required in `.env`: - **Sidebar Toggle**: The sidebar state is local to `AppContent`. Refreshing the page resets it to open. ## Related +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] - [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] - [[01 Projects/baic_dashboard/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ac-helper/DEVELOPER_MANUAL.md b/01 Projects/ac-helper/DEVELOPER_MANUAL.md index b1bb69a..132a8ce 100644 --- a/01 Projects/ac-helper/DEVELOPER_MANUAL.md +++ b/01 Projects/ac-helper/DEVELOPER_MANUAL.md @@ -144,5 +144,8 @@ The application is a **hybrid stack**: - **Dependent Dropdowns**: Changing a "Category" or "Media" may invalidate dependent cells. Validate data integrity on import/export. ## Related +- [[01 Projects/ac-tool/DEVELOPER_MANUAL]] +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] +- [[01 Projects/gmal-scope-builder/DEVELOPER_MANUAL]] - [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] - [[01 Projects/oliver-ai-assistant/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ac-helper/USER_MANUAL.md b/01 Projects/ac-helper/USER_MANUAL.md index 0ce1d81..cec8abe 100644 --- a/01 Projects/ac-helper/USER_MANUAL.md +++ b/01 Projects/ac-helper/USER_MANUAL.md @@ -94,4 +94,5 @@ A: Click the "Export CSV" button in the toolbar. Note: This exports only the *cu A: Yes. Data is stored in a PostgreSQL database. Access is restricted to authenticated users. Sensitive files (briefs) are stored securely and not publicly accessible. ## Related +- [[01 Projects/ac-tool/USER_MANUAL]] - [[01 Projects/ac-tool/USER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ac-tool/DEVELOPER_MANUAL.md b/01 Projects/ac-tool/DEVELOPER_MANUAL.md index f45f7d9..10c305f 100644 --- a/01 Projects/ac-tool/DEVELOPER_MANUAL.md +++ b/01 Projects/ac-tool/DEVELOPER_MANUAL.md @@ -180,5 +180,8 @@ location /ac-helper/ { ## Related +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/cc-dashboard/DEVELOPER_MANUAL]] +- [[01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL]] - [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] - [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ac-tool/USER_MANUAL.md b/01 Projects/ac-tool/USER_MANUAL.md index 9e5044c..cd898c1 100644 --- a/01 Projects/ac-tool/USER_MANUAL.md +++ b/01 Projects/ac-tool/USER_MANUAL.md @@ -97,4 +97,8 @@ A: The tool supports large uploads, but processing time and token usage will inc A: Yes, all data is stored in a PostgreSQL database with encrypted connections. Access is controlled via Azure AD and role-based permissions. **Q: Can I export my data?** -A: Yes, all sheets can be exported as CSV, and jobs include downloadable result files. \ No newline at end of file +A: Yes, all sheets can be exported as CSV, and jobs include downloadable result files. + +## Related + +- [[01 Projects/ac-helper/USER_MANUAL]] diff --git a/01 Projects/amazon-transcreation/DEVELOPER_MANUAL.md b/01 Projects/amazon-transcreation/DEVELOPER_MANUAL.md index 0061c0c..4976009 100644 --- a/01 Projects/amazon-transcreation/DEVELOPER_MANUAL.md +++ b/01 Projects/amazon-transcreation/DEVELOPER_MANUAL.md @@ -162,6 +162,7 @@ Frontend (Next.js) <--> FastAPI Backend <--> Celery Workers - **Redis/PostgreSQL**: Ensure both services are healthy before starting backend/celery. `depends_on` with `condition: service_healthy` helps, but add retries in app code. ## Related +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] - [[01 Projects/enterprise-ai-hub-nexus/DEVELOPER_MANUAL.md]] - [[01 Projects/oliver-ai-assistant/DEVELOPER_MANUAL.md]] - [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md b/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md index a0f68fe..1be3218 100644 --- a/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md +++ b/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md @@ -138,4 +138,6 @@ The frontend likely calls an internal proxy endpoint (e.g., `/api/data`) which f 6. **Path Trailing Slashes**: Ensure `BASE_PATH` ends with `/` as per config requirements. ## Related +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL]] - [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/barclays-rag-report/DEVELOPER_MANUAL.md b/01 Projects/barclays-rag-report/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..5ba2c64 --- /dev/null +++ b/01 Projects/barclays-rag-report/DEVELOPER_MANUAL.md @@ -0,0 +1,164 @@ +--- +auto_generated: true +manual_updated_at: 2026-05-19 +modified: 2026-05-19 +--- + +# RAG Test App Developer Manual + +## Architecture Overview +The application follows a modular architecture: +- **CLI Layer**: Handles argument parsing and user interaction +- **Core Logic**: RAGTester class manages testing workflows +- **Testing**: Pytest suite for component validation +- **Output**: Rich console formatting and JSON/CSV results + +## Tech Stack +- Python 3.8+ +- OpenAI Python SDK +- Pandas & NumPy for data analysis +- Rich for console formatting +- Matplotlib for visualizations +- Pytest for testing + +## Local Setup + +### Prerequisites +- Python 3.8+ +- OpenAI API key +- pip package manager + +### Installation +```bash +# Clone repository +git clone https://github.com/yourusername/rag-test-app.git +cd rag-test-app + +# Install dependencies +pip install -e . +``` + +## Environment Variables +Required: +- `OPENAI_API_KEY`: Your OpenAI API key (can also be passed via CLI) + +Optional: +- `CONFIG_PATH`: Default configuration file path +- `OUTPUT_DIR`: Default output directory + +## Key Services & Entry Points + +### Main Entry Point +```python +# cli.py +if __name__ == "__main__": + main() +``` + +### Core Classes +```python +# main.py +from openai import OpenAI + +class RAGTester: + def __init__(self, api_key, assistant_id, ...): + self.client = OpenAI(api_key=api_key) + self.document_content = self._load_documents() + + def run_test(self): + """Execute RAG testing workflow""" + questions = self._generate_questions() + results = self._evaluate_responses(questions) + return self._generate_report(results) +``` + +### Configuration Structure +```json +{ + "api_key": "sk-xxxxx", + "assistant_id": "asst_xxxxx", + "document": "path/to/doc.pdf", + "num_questions": 20, + "iterations": 3, + "model": "gpt-4o", + "prompt_type": "task-based" +} +``` + +## API Reference + +### CLI Arguments +| Argument | Description | Default | +|----------|-------------|---------| +| --config | Config file path | None | +| --api-key | OpenAI API key | env var | +| --assistant-id | Assistant ID | Required | +| --document | Document path | Required | +| --output-dir | Results directory | results | +| --num-questions | Number of questions | 20 | +| --model | OpenAI model | gpt-4o | +| --prompt-type | Question generation type | task-based | +| --parallel | Parallel workers | 5 | +| --verbose | Enable verbose output | False | + +### Core Methods +- `RAGTester.__init__()`: Initialize tester +- `RAGTester._load_documents()`: Load document content +- `RAGTester._generate_questions()`: Create test questions +- `RAGTester._evaluate_responses()`: Test assistant responses +- `RAGTester._generate_report()`: Create results report + +## Deployment + +### Docker Support +```dockerfile +FROM python:3.11-slim +WORKDIR /app +COPY requirements.txt . +RUN pip install -r requirements.txt +COPY . . +ENTRYPOINT ["rag-test"] +``` + +### Production Config +```json +{ + "parallel": 10, + "batch_size": 5, + "output_dir": "/data/results", + "verbose": false +} +``` + +## Known Gotchas +1. **Document Loading** + - DOCX files require `docx2txt` package + - Large PDFs may consume significant memory + - Text encoding issues may occur with special characters + +2. **API Limits** + - Rate limiting may occur with high parallel workers + - Token limits in document processing + - Assistant context window constraints + +3. **Performance** + - Parallel processing can cause resource contention + - Large document sets impact processing time + - Model selection affects speed/accuracy tradeoff + +4. **Testing** + - Mock OpenAI client in tests + - Use temporary directories for test artifacts + - Clear result directories between runs + +## Testing +```bash +# Run tests +pytest + +# Run with coverage +pytest --cov=rag_test_app + +# Specific test file +pytest tests/test_main.py +``` \ No newline at end of file diff --git a/01 Projects/barclays-rag-report/USER_MANUAL.md b/01 Projects/barclays-rag-report/USER_MANUAL.md new file mode 100644 index 0000000..de981e2 --- /dev/null +++ b/01 Projects/barclays-rag-report/USER_MANUAL.md @@ -0,0 +1,115 @@ +--- +auto_generated: true +manual_updated_at: 2026-05-19 +modified: 2026-05-19 +--- + +# RAG Test App User Manual + +## What This Tool Does +The RAG Test App is a comprehensive tool designed to evaluate the performance of OpenAI Assistants with Retrieval-Augmented Generation (RAG) capabilities. It allows users to: +- Test assistant responses against specific documents +- Generate questions based on different strategies (task-based, content-based, scenario-based) +- Evaluate assistant accuracy and performance +- Generate detailed reports and visualizations + +## Who Uses It +- AI Engineers testing RAG implementations +- Data Scientists evaluating model performance +- QA Engineers validating assistant behavior +- Researchers conducting AI experiment analysis + +## How to Access + +### Installation +```bash +pip install rag-test-app +``` + +### Configuration +Create a JSON configuration file (e.g., `config.json`): +```json +{ + "api_key": "your-openai-api-key", + "assistant_id": "asst_xxxxx", + "document": "path/to/document.txt", + "output_dir": "results", + "num_questions": 20, + "iterations": 3, + "model": "gpt-4o", + "prompt_type": "task-based", + "parallel": 5 +} +``` + +## Main Workflows + +### 1. Basic Evaluation +```bash +rag-test \ + --config config.json \ + --api-key sk-xxxxx \ + --assistant-id asst_xxxxx \ + --document docs/report.pdf +``` + +### 2. Generate Questions Only +```bash +rag-test \ + --config config.json \ + --generate-only +``` + +### 3. Parallel Testing +```bash +rag-test \ + --config config.json \ + --parallel 10 +``` + +### 4. Custom Model +```bash +rag-test \ + --config config.json \ + --model gpt-4-turbo +``` + +## Step-by-Step Example + +1. **Setup Configuration** + ```json + { + "api_key": "sk-your-key", + "assistant_id": "asst_123", + "document": "./company_policy.pdf", + "num_questions": 50, + "model": "gpt-4o" + } + ``` + +2. **Run Test** + ```bash + rag-test --config my_config.json + ``` + +3. **View Results** + - Check `results/` directory + - View `report.md` for summary + - Check `visualizations/` for charts + +## FAQ + +**Q: What formats are supported?** +A: TXT, PDF, DOCX, and Markdown files. + +**Q: How are results measured?** +A: Through accuracy scores, response times, and semantic similarity metrics. + +**Q: Can I test multiple documents?** +A: Yes, use the `documents` key in config with a list of paths. + +**Q: How do I run tests in parallel?** +A: Use the `--parallel` flag or set `parallel` in config. + +**Q: Where are the results stored?** +A: In a timestamped directory within the output_dir (default: `results/`) \ No newline at end of file diff --git a/01 Projects/build-a-squad/USER_MANUAL.md b/01 Projects/build-a-squad/USER_MANUAL.md index 8e96018..f12b6af 100644 --- a/01 Projects/build-a-squad/USER_MANUAL.md +++ b/01 Projects/build-a-squad/USER_MANUAL.md @@ -96,4 +96,5 @@ A: Yes. The text is sent only to the Google Gemini API for analysis and is not s A: This indicates assets that have specific internal criteria or flags within the studio's operational model. It is primarily used for internal filtering and reporting. ## Related +- [[01 Projects/gmal-scope-builder/USER_MANUAL]] - [[01 Projects/gmal-scope-builder/USER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/cc-dashboard/DEVELOPER_MANUAL.md b/01 Projects/cc-dashboard/DEVELOPER_MANUAL.md index d709a53..bef0e96 100644 --- a/01 Projects/cc-dashboard/DEVELOPER_MANUAL.md +++ b/01 Projects/cc-dashboard/DEVELOPER_MANUAL.md @@ -114,4 +114,10 @@ CC Dashboard is a full-stack web application consisting of: 2. **Database Health Check:** The DB depends on `service_healthy`. If the database takes longer than 30s (10 retries * 3s timeout) to start, the app container will fail. Adjust `interval` or `retries` if using slow hardware. 3. **Admin Auto-Role:** Only users in `ADMIN_EMAILS` get admin rights automatically on first login. Subsequent logins do not check this list again. 4. **Dev Bypass:** Never enable `DEV_AUTH_BYPASS=true` in production. It completely ignores SSO. -5. **Calendar Algorithm:** The lane assignment algorithm (`Be` function in `CalendarView`) assumes tasks are sorted by start time. Ensure backend returns tasks sorted chronologically. \ No newline at end of file +5. **Calendar Algorithm:** The lane assignment algorithm (`Be` function in `CalendarView`) assumes tasks are sorted by start time. Ensure backend returns tasks sorted chronologically. + +## Related + +- [[01 Projects/ac-tool/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL]] diff --git a/01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL.md b/01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL.md index 1d247aa..0522e84 100644 --- a/01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL.md +++ b/01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL.md @@ -135,4 +135,5 @@ Use `docker-compose.prod.yml` to deploy: 5. **Kling Integration:** The Kling workflow is optional. Ensure `klingWorkflow` state in `VideoGenTab.jsx` defaults correctly if Kling is not configured in `AdminSettings`. ## Related +- [[01 Projects/cinema-studio-pro/DEVELOPER_MANUAL]] - [[01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/cinema-studio-pro-kling/USER_MANUAL.md b/01 Projects/cinema-studio-pro-kling/USER_MANUAL.md index 66f4451..170057d 100644 --- a/01 Projects/cinema-studio-pro-kling/USER_MANUAL.md +++ b/01 Projects/cinema-studio-pro-kling/USER_MANUAL.md @@ -95,6 +95,7 @@ A: Ensure your browser supports the video codec used by the API (usually H.264/M A: Yes. Since storage is local (IndexedDB) and AI processing happens via Google's API, no generated assets are stored on Lux Studio's servers. Authentication is handled securely via MSAL if SSO is enabled. ## Related +- [[01 Projects/cinema-studio-pro/USER_MANUAL]] - [[01 Projects/cinema-studio-pro/USER_MANUAL.md]] - [[01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL.md]] - [[01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md b/01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md index a7e552d..7446a9f 100644 --- a/01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md +++ b/01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md @@ -152,5 +152,6 @@ The backend PHP scripts act as a proxy to AI providers and handle file managemen 5. **AI Latency**: Video generation can take several minutes. Ensure the UI shows clear loading states and does not timeout prematurely. ## Related +- [[01 Projects/cinema-studio-pro-kling/DEVELOPER_MANUAL]] - [[01 Projects/olivas/DEVELOPER_MANUAL.md]] - [[01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/cinema-studio-pro/USER_MANUAL.md b/01 Projects/cinema-studio-pro/USER_MANUAL.md index 1e3a80b..09117e3 100644 --- a/01 Projects/cinema-studio-pro/USER_MANUAL.md +++ b/01 Projects/cinema-studio-pro/USER_MANUAL.md @@ -101,4 +101,5 @@ A: Upload reference images in the Image Generation tab. The AI will analyze the A: Lux Studio is optimized for desktop workflows. While it may function on tablets, the complex interface is best experienced on a large screen. ## Related +- [[01 Projects/cinema-studio-pro-kling/USER_MANUAL]] - [[01 Projects/cinema-studio-pro/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL.md b/01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL.md index 5be2b59..81c74ca 100644 --- a/01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL.md +++ b/01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL.md @@ -142,5 +142,6 @@ pm2 start server.js --name "ferrero-booking" 5. **Local Path Dependency:** The README example uses an absolute local path (`/Users/pauljohns/...`). Ensure deployment instructions use relative paths or configurable directories. ## Related +- [[01 Projects/ferrero-ac-creator/USER_MANUAL]] - [[Ferrero Booking Tool]] - [[OMG API Integration]] \ No newline at end of file diff --git a/01 Projects/ferrero-ac-creator/USER_MANUAL.md b/01 Projects/ferrero-ac-creator/USER_MANUAL.md index e7fad43..728ce79 100644 --- a/01 Projects/ferrero-ac-creator/USER_MANUAL.md +++ b/01 Projects/ferrero-ac-creator/USER_MANUAL.md @@ -124,4 +124,5 @@ A: You can duplicate rows to keep backups, bulk-edit rows, or delete rows and st A: Click the **Logout** link/button in the toolbar if you are using the server version. ## Related +- [[01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL]] - [[01 Projects/ferrero-ac-creator/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/ford_qc/DEVELOPER_MANUAL.md b/01 Projects/ford_qc/DEVELOPER_MANUAL.md index b7f5635..cbed466 100644 --- a/01 Projects/ford_qc/DEVELOPER_MANUAL.md +++ b/01 Projects/ford_qc/DEVELOPER_MANUAL.md @@ -190,4 +190,5 @@ Place a valid `ford_box_config.json` (from IT or Box Admin) in the project root 6. **PDF Generation:** `generate_pdf.py` is for documentation only and has dependencies (`reportlab`, `PIL`, `mmdc`/`mermaid-cli`) not required for the core QC system. ## Related +- [[01 Projects/ford_qc/USER_MANUAL]] - [[ford_qc_system_box_integration]] \ No newline at end of file diff --git a/01 Projects/ford_qc/USER_MANUAL.md b/01 Projects/ford_qc/USER_MANUAL.md index 40c9544..f252448 100644 --- a/01 Projects/ford_qc/USER_MANUAL.md +++ b/01 Projects/ford_qc/USER_MANUAL.md @@ -131,4 +131,5 @@ A: * **Local/Dev:** Standard output or log files configured in the logging setup section of `ford_qc_box_hotfolder_process.py`. ## Related +- [[01 Projects/ford_qc/DEVELOPER_MANUAL]] - [[01 Projects/ford_qc/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md b/01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md index 544f94b..52f66ef 100644 --- a/01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md +++ b/01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md @@ -2,7 +2,7 @@ auto_generated: true created: 2026-05-18 manual_updated_at: 2026-05-18 -modified: 2026-05-18 +modified: 2026-05-19 name: Developer_Manual status: active tags: @@ -178,5 +178,9 @@ Services communicate via HTTP (Frontend <-> Backend) and network (Backend <-> Da 6. **Excel Format**: The GMAL ingestion logic depends on the specific structure of the "U-Studio GMAL Asset Job Routes..." Excel file. Changes to this file structure will break ingestion unless the backend parser is updated. ## Related +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/olivas/DEVELOPER_MANUAL]] +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] - [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL.md]] -- [[01 Projects/ac-tool/DEVELOPER_MANINAL.md]] \ No newline at end of file +- [[01 Projects/ac-tool/DEVELOPER_MANINAL.md]] +- [[01 Projects/gmal-scope-builder/USER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/gmal-scope-builder/USER_MANUAL.md b/01 Projects/gmal-scope-builder/USER_MANUAL.md index 57266ac..a4584e6 100644 --- a/01 Projects/gmal-scope-builder/USER_MANUAL.md +++ b/01 Projects/gmal-scope-builder/USER_MANUAL.md @@ -98,4 +98,5 @@ A: You need the 'admin' role assigned to your user account. Contact your system A: FTE is calculated based on the total hours required for all matched assets, divided by the standard annual working hours for a full-time employee. ## Related +- [[01 Projects/build-a-squad/USER_MANUAL]] - [[01 Projects/gmal-scope-builder/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/hp-prod-tracker/DEVELOPER_MANUAL.md b/01 Projects/hp-prod-tracker/DEVELOPER_MANUAL.md index 121d412..136087b 100644 --- a/01 Projects/hp-prod-tracker/DEVELOPER_MANUAL.md +++ b/01 Projects/hp-prod-tracker/DEVELOPER_MANUAL.md @@ -188,6 +188,9 @@ The `docker-compose.yml` defines: 5. **File Uploads:** Media uploads are stored in `uploads_data` volume. Ensure the Docker user has write permissions to `/data/uploads`. ## Related +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] +- [[01 Projects/enterprise-ai-hub-nexus/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] - [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] - [[01 Projects/olivas/DEVELOPER_MANUAL.md]] - [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/loreal-sla-calculator/USER_MANUAL.md b/01 Projects/loreal-sla-calculator/USER_MANUAL.md index 9025cf8..db85694 100644 --- a/01 Projects/loreal-sla-calculator/USER_MANUAL.md +++ b/01 Projects/loreal-sla-calculator/USER_MANUAL.md @@ -73,4 +73,5 @@ A: Non-developers can update assumptions, stage definitions, and default values A: The tool provides a "No" verdict and allows you to adjust inputs (like complexity or revisions) to see how changes impact the timeline. ## Related +- [[01 Projects/lusa-back-planner/USER_MANUAL]] - [[01 Projects/lusa-back-planner/USER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/lusa-back-planner/USER_MANUAL.md b/01 Projects/lusa-back-planner/USER_MANUAL.md index 0dee8ab..6efff90 100644 --- a/01 Projects/lusa-back-planner/USER_MANUAL.md +++ b/01 Projects/lusa-back-planner/USER_MANUAL.md @@ -78,4 +78,8 @@ A: Modern browsers (Chrome, Firefox, Safari, Edge) with JavaScript enabled. IE11 --- -*For technical support or deployment issues, please refer to the Developer Manual.* \ No newline at end of file +*For technical support or deployment issues, please refer to the Developer Manual.* + +## Related + +- [[01 Projects/loreal-sla-calculator/USER_MANUAL]] diff --git a/01 Projects/md to word/DEVELOPER_MANUAL.md b/01 Projects/md to word/DEVELOPER_MANUAL.md index 2700c2d..24b60ec 100644 --- a/01 Projects/md to word/DEVELOPER_MANUAL.md +++ b/01 Projects/md to word/DEVELOPER_MANUAL.md @@ -125,4 +125,8 @@ A Dockerfile can be created to containerize the application. Ensure system depen - **Font Loading**: Mermaid CSS loads Montserrat from Google Fonts. In offline environments, provide a local font file or use a different font family. - **Temporary Files**: The tool creates temp files for Mermaid rendering. These are not cleaned up automatically in all cases. - **Python Version**: Requires Python 3.9+. Do not use with Python <3.9 due to dependency compatibility. -- **Large Documents**: Performance may degrade with very large Markdown files due to repeated temp file creation and Mermaid rendering. \ No newline at end of file +- **Large Documents**: Performance may degrade with very large Markdown files due to repeated temp file creation and Mermaid rendering. + +## Related + +- [[01 Projects/md to word/USER_MANUAL]] diff --git a/01 Projects/md to word/USER_MANUAL.md b/01 Projects/md to word/USER_MANUAL.md index 31e11e0..7d8fa5b 100644 --- a/01 Projects/md to word/USER_MANUAL.md +++ b/01 Projects/md to word/USER_MANUAL.md @@ -108,5 +108,6 @@ A: The current CLI supports one input file per run. Use shell loops or scripts t A: The renderer may fail silently or produce broken images. Check your Mermaid syntax carefully before conversion. ## Related +- [[01 Projects/md to word/DEVELOPER_MANUAL]] - [[01 Projects/md to word/DEVELOPER_MANUAL.md]] - [[Mermaid CLI Installation]] \ No newline at end of file diff --git a/01 Projects/modcomms/DEVELOPER_MANUAL.md b/01 Projects/modcomms/DEVELOPER_MANUAL.md index 75cd83d..796330d 100644 --- a/01 Projects/modcomms/DEVELOPER_MANUAL.md +++ b/01 Projects/modcomms/DEVELOPER_MANUAL.md @@ -1,124 +1,148 @@ --- auto_generated: true -created: 2026-05-18 -manual_updated_at: 2026-05-18 -modified: 2026-05-18 +created: 2026-05-19 +manual_updated_at: 2026-05-19 +modified: 2026-05-19 name: Developer_Manual status: active tags: -- client/oliver -- domain/ai +- client/barclays - status/active - tech/azure-ad - tech/docker - tech/fastapi +- tech/gemini +- tech/llamaindex - tech/postgresql - tech/python -- tech/react -- tech/typescript +- tech/rag +- type/project type: project --- # Mod Comms Developer Manual ## Architecture Overview -Mod Comms is a multi-agent AI application with a Python backend and a React frontend. -### High-Level Flow -1. **Frontend**: React application handles UI, state management (via MSAL for auth), and WebSocket connections for real-time feedback. -2. **Backend**: FastAPI/Python service handles API requests, manages database interactions, and orchestrates the AI analysis. -3. **AI Engine**: A multi-agent system where four specialist agents (Legal, Brand, Tone, Channel) run in parallel, followed by a Lead Agent that synthesizes results. -4. **Database**: PostgreSQL stores user data, campaign info, proof metadata, and audit logs. -5. **Storage**: MinIO or local file system (configured via `FILE_STORAGE_PATH`) stores uploaded assets. +Mod Comms is a FastAPI-based backend application that interacts with a PostgreSQL database and utilizes a multi-agent AI system powered by Google Gemini. The architecture involves: + +* **Backend (FastAPI):** Handles API requests, authentication (Azure AD), and business logic. +* **Multi-Agent AI System:** Four specialist agents (Legal, Brand, Tone, Channel) analyze assets in parallel, with a Lead Agent synthesizing the final verdict. +* **Database (PostgreSQL):** Stores campaigns, proofs, user data, audit trails, and reference documents. +* **File Storage:** Local or cloud-based storage for uploaded proof assets. +* **Knowledge Base:** Optional integration with LlamaParse for advanced document processing. + +### Multi-Agent Flow + +``` +User Request -> Analysis Orchestrator -> [Legal, Brand, Tone, Channel Agents] -> Lead Agent -> Final Verdict +``` ## Tech Stack -- **Frontend**: React, TypeScript, Tailwind CSS, @azure/msal-react (Authentication), WebSocket client. -- **Backend**: Python, FastAPI, Asyncpg (PostgreSQL driver), Pydantic. -- **Database**: PostgreSQL 16. -- **AI**: Custom multi-agent orchestration logic (likely involving LLM calls and RAG). -- **Containerization**: Docker, Docker Compose. + +* **Backend:** Python, FastAPI +* **Database:** PostgreSQL 16 +* **AI Service:** Google Gemini API +* **Authentication:** Azure AD +* **ORM/Database Migrations:** SQLAlchemy, Alembic +* **Containerization:** Docker, Docker Compose +* **Documentation:** Pptx for presentation generation ## Local Setup ### Prerequisites -- Docker and Docker Compose -- Node.js (for frontend development) -- Python 3.11+ + +* Python 3.10+ +* Docker and Docker Compose +* Google Gemini API Key +* Azure AD Credentials (Tenant ID, Client ID) OR set `DISABLE_AUTH=true` for local dev. ### Steps -1. **Clone the repository**. -2. **Configure Environment Variables**: - - Copy `backend/.env.example` to `backend/.env` and fill in required values (especially `DATABASE_URL` and Azure AD config). -3. **Start the Infrastructure**: - ```bash - docker-compose up -d postgres - ``` - The backend will depend on the postgres service being healthy. -4. **Start the Backend**: - ```bash - docker-compose up --build backend - ``` - Or run locally: - ```bash - cd backend - pip install -r requirements.txt - uvicorn main:app --reload - ``` -5. **Start the Frontend**: - ```bash - cd frontend - npm install - npm start - ``` + +1. **Clone the repository.** +2. **Configure Environment Variables:** + * Copy `.env.example` to `.env` in the `backend/` directory. + * Set required variables (see Environment Variables section). +3. **Start Infrastructure:** + ```bash + docker-compose up -d postgres + ``` +4. **Run Migrations:** + ```bash + cd backend + alembic upgrade head + ``` +5. **Start Backend:** + ```bash + cd backend + uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload + ``` +6. **Access the App:** Open `http://localhost:8000` in your browser. ## Environment Variables -### Backend (backend/.env) -- `DATABASE_URL`: Connection string for PostgreSQL (e.g., `postgresql+asyncpg://user:pass@postgres:5432/db`). -- `FILE_STORAGE_PATH`: Path for storing uploaded files. -- `AZURE_CLIENT_ID`, `AZURE_CLIENT_SECRET`, `AZURE_TENANT_ID`: For MSAL authentication. -- `OPENAI_API_KEY` or other AI provider keys. - -### Docker Compose -- `POSTGRES_PORT`: Port for Postgres (default 5432). -- `BACKEND_PORT`: Port for the backend API (default 8000). -- `POSTGRES_USER`, `POSTGRES_PASSWORD`, `POSTGRES_DB`: Default credentials if not overridden. +| Variable | Description | Default | Required | +|----------|-------------|---------|----------| +| `GEMINI_API_KEY` | Google Gemini API Key | | Yes | +| `DISABLE_AUTH` | Bypass Azure AD for dev | `false` | No | +| `AZURE_TENANT_ID` | Azure AD Tenant ID | | Yes (if auth enabled) | +| `AZURE_CLIENT_ID` | Azure AD Client ID | | Yes (if auth enabled) | +| `DATABASE_URL` | PostgreSQL connection string | `postgresql+asyncpg://modcomms:modcomms_dev@localhost:5432/modcomms` | No | +| `REFERENCE_DOCS_PATH` | Path to reference docs for RAG | `../reference_docs` | No | +| `LLAMA_CLOUD_API_KEY` | LlamaParse API Key | | No | +| `MAILGUN_API_URL` | Mailgun API URL | | No | +| `POSTGRES_USER` | Postgres username | `modcomms` | No | +| `POSTGRES_PASSWORD` | Postgres password | `modcomms_dev` | No | +| `POSTGRES_DB` | Postgres database name | `modcomms` | No | +| `POSTGRES_PORT` | Postgres port | `5432` | No | +| `BACKEND_PORT` | Backend port | `8000` | No | ## Key Services & Entry Points -### Backend -- **`backend/main.py`**: Entry point for the FastAPI application. -- **`backend/services/ai_orchestrator.py`**: Contains the logic for spawning and managing the four specialist agents and the lead agent. -- **`backend/models/`**: SQLAlchemy/SQLModel models for Postgres tables. -- **`backend/routers/`**: API endpoints for campaigns, proofs, analytics, etc. +* **`app.main.py`**: FastAPI application entry point. Initializes services (Gemini, ReferenceDocs, Analysis, KnowledgeBase) on startup. +* **`app/config.py`**: Settings management via environment variables. +* **`app/services/gemini_service.py`**: Interface with Google Gemini API. +* **`app/services/analysis_service.py`**: Orchestrates multi-agent analysis. +* **`app/services/reference_docs.py`**: Manages RAG context from reference documents. +* **`app/models/database.py`**: Database session management and initialization. +* **`backend/alembic/`**: Database migrations. -### Frontend -- **`frontend/index.tsx`**: Initializes MSAL client and mounts the React app. -- **`frontend/App.tsx`**: Main component handling routing, auth state, and view switching. -- **`frontend/contexts/UserContext.tsx`**: Manages user role and permissions across the app. -- **`frontend/services/geminiService.ts`**: Handles communication with the AI backend (likely via HTTP/WebSocket). -- **`frontend/services/apiService.ts`**: Standard API service with MSAL token injection. +### Database Models -## API Reference (Key Endpoints) -- `GET /health`: Health check endpoint for the backend. -- `POST /api/v1/proofs/analyze`: Submit a proof for analysis. -- `GET /api/v1/campaigns`: List campaigns. -- `POST /api/v1/campaigns`: Create a campaign. -- `GET /api/v1/me`: Get current user details and permissions. -- `WebSocket /ws/analysis/{proof_id}`: Receive real-time updates during analysis. +* `Agency`, `User`, `Campaign`, `Proof`, `ProofVersion`, `FlaggedItem`, `ResolvedItem`, `ErrorItem`, `DropdownOption` + +## API Reference + +* **Base URL:** `http://localhost:8000` +* **Health Check:** `GET /health` +* **Authentication:** Azure AD OAuth2 (unless `DISABLE_AUTH=true`) +* **Endpoints:** See `app/api/router.py`, `app/api/kb_router.py`, `app/api/analysis_router.py` for specific endpoint details. ## Deployment -- **Docker Compose**: For development and staging. -- **Kubernetes**: Production deployment likely uses K8s, with managed PostgreSQL and object storage. -- **CI/CD**: Automated tests and linting are run on push to main. + +1. **Build Docker Images:** + ```bash + docker-compose build + ``` +2. **Set Production Environment Variables:** Ensure all required secrets are set in the production environment. +3. **Run Docker Compose:** + ```bash + docker-compose up -d + ``` +4. **Verify Health:** Check `http://:8000/health`. ## Known Gotchas -- **Auth**: Ensure MSAL config in `frontend/services/authConfig.ts` matches your Azure AD app registration. -- **Storage**: The `file_storage` volume is mapped in docker-compose. Ensure it has sufficient space. -- **Agents**: The AI agents run in parallel; ensure the AI provider has sufficient rate limits. -- **CORS**: Backend must allow origins from the frontend development server and production domain. + +* **Database Initialization:** The backend starts in a "stateless mode" if the database is not available during startup. Ensure `postgres` service is healthy before `backend` starts (handled by `depends_on` in `docker-compose.yml`). +* **Gemini API Key:** Must be valid and have sufficient quota. +* **Azure AD:** If not using `DISABLE_AUTH`, ensure the Azure AD app is correctly configured with redirect URIs. +* **Reference Docs:** Ensure `REFERENCE_DOCS_PATH` points to valid documents for RAG context. +* **LlamaParse:** If using the Knowledge Base feature, a valid `LLAMA_CLOUD_API_KEY` is required. +* **File Storage:** The `file_storage` volume is persistent. Clean up old files manually if needed. ## Related -- [[01 Projects/ac-helper/DEVELOPER_MANUAL.md]] -- [[01 Projects/enterprise-ai-hub-nexus/DEVELOPER_MANUAL.md]] -- [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL.md]] \ No newline at end of file + +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/modcomms/USER_MANUAL.md]] +- [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/modcomms/USER_MANUAL.md b/01 Projects/modcomms/USER_MANUAL.md index 8980508..88b0bd7 100644 --- a/01 Projects/modcomms/USER_MANUAL.md +++ b/01 Projects/modcomms/USER_MANUAL.md @@ -1,87 +1,69 @@ --- auto_generated: true -created: 2026-05-18 -manual_updated_at: 2026-05-18 -modified: 2026-05-18 -name: User_Manual -status: active -tags: -- client/barclays -- domain/ai -- tech/azure-ad -- tech/rag -- type/sop -type: sop +manual_updated_at: 2026-05-19 +modified: 2026-05-19 --- # Mod Comms User Manual ## What This Tool Does -Mod Comms is an AI-powered proof review tool designed for Barclays marketing materials. It utilizes a multi-agent AI system to analyze uploaded marketing assets for: -- **Legal Compliance**: Advertising standards, disclaimers, and financial promotion detection. -- **Brand Adherence**: Logo usage, color palette accuracy, and typo checking. -- **Tone of Voice**: Voice and clarity analysis. -- **Channel Suitability**: Platform-specific specifications. -The system provides real-time feedback via WebSocket streaming, allowing you to see the progress of analysis as it happens. +Mod Comms is an AI-powered proof review tool designed for Barclays marketing materials. It utilizes a multi-agent AI system to analyze uploaded marketing assets for: + +* **Legal Compliance:** Ensures adherence to advertising standards and detects financial promotions. +* **Brand Adherence:** Checks logo usage, color palettes, and visual identity. +* **Tone of Voice:** Analyzes clarity and voice consistency. +* **Channel Suitability:** Verifies platform-specific specifications. + +The tool provides a **RAG status** (Red, Amber, Green) for each agent, with detailed feedback, synthesized by a Lead Agent into a final verdict. It also offers real-time feedback via WebSocket streaming, campaign management, and PDF export capabilities. ## Who Uses It -- **Super Admins**: Full access to all features including user management, settings, and knowledge base. -- **Oversight Admins**: Can manage campaigns, view analytics, and audit trails. -- **Agency Admins**: Can manage agency-specific campaigns and view analytics. -- **Basic Users**: Can upload proofs, view feedback, and use the WIP Reviewer. -- **Agency Users**: Can upload proofs and collaborate on agency campaigns. + +* **Marketing Teams:** For submitting marketing assets for compliance and brand checks. +* **Legal & Brand Teams:** For reviewing flagged issues and resolutions. +* **WIP Reviewers:** For getting early-stage creative feedback on work-in-progress assets. ## How to Access -1. Navigate to the Mod Comms URL. -2. You will be prompted to log in using your corporate Microsoft account (SSO via Azure AD). -3. Once authenticated, you will be taken to the Home Dashboard. + +1. **Log In:** Access the application via the provided web interface. Authentication is handled via Azure AD. For development, auth can be disabled by setting `DISABLE_AUTH=true`. +2. **Navigate:** Use the dashboard to view analytics, campaign lists, and audit trails. +3. **Upload:** Select a campaign or create a new one to upload proof assets (PDFs, images, etc.). ## Main Workflows -### 1. Analyzing a Marketing Proof -1. From the Home Dashboard or Sidebar, select **"Home"** or **"Campaigns"**. -2. Click **"Upload Proof"** or drag and drop an asset (PDF, Image, etc.). -3. Select the relevant **Campaign**, **Channel**, **SubChannel**, and **Proof Type** from the dropdowns. -4. The multi-agent system will begin analyzing the asset in parallel. -5. Watch the **Real-Time Feedback** panel for live updates from the Legal, Brand, Tone, and Channel agents. -6. Review the final verdict (Red/Amber/Green) and detailed feedback from the Lead Agent. -7. Resolve flagged items by clicking on them and adding comments or marking them as resolved. -8. Optionally, generate a **PDF Report** of the analysis. +### Workflow 1: Submitting a Proof for Analysis -### 2. Managing Campaigns -1. Navigate to **"Campaigns"** in the sidebar. -2. Create a new campaign or select an existing one. -3. Organize proofs within the campaign. -4. View version history for each proof to track changes over time. +1. **Navigate to Campaigns:** Go to the "Campaigns" section or create a new campaign. +2. **Upload Asset:** Click "Add Proof" and upload your marketing asset. +3. **Initiate Analysis:** The system will automatically start the multi-agent analysis in parallel. +4. **Monitor Progress:** Watch the real-time WebSocket stream for live updates from the Legal, Brand, Tone, and Channel agents. +5. **Review Verdict:** Once complete, review the synthesized final verdict and detailed feedback from each agent. +6. **Export Report:** Click "Export PDF" to generate a professional report of the analysis results. -### 3. WIP Reviewer -1. Select **"WIP Reviewer"** from the sidebar. -2. Upload work-in-progress assets for early-stage creative feedback. -3. Receive targeted feedback from the AI agents specifically tailored for incomplete drafts. +### Workflow 2: Managing Campaigns and Versions -### 4. Viewing Analytics -1. Navigate to **"Analytics"**. -2. View compliance metrics, status breakdowns (Red/Amber/Green ratios), and trends. -3. Filter by campaign, channel, or date range. +1. **Create Campaign:** From the dashboard, click "New Campaign" to organize proofs. +2. **Upload Versions:** Upload multiple versions of proofs to the same campaign for version history tracking. +3. **Compare Results:** Use the campaign view to compare analysis results across different versions. -### 5. Auditing -1. Navigate to **"Auditing"** (Admins only). -2. View the full audit trail of all flagged issues and their resolutions. +### Workflow 3: Reviewing WIP Assets + +1. **Select WIP:** Identify a work-in-progress asset. +2. **Request Early Feedback:** Use the WIP Reviewer feature to get preliminary feedback before final submission. ## FAQ -**Q: How does the multi-agent system work?** -A: Four specialist agents (Legal, Brand, Tone, Channel) analyze the asset in parallel. A Lead Agent then synthesizes their findings into a final verdict with a RAG (Red/Amber/Green) status. +**Q: What file types are supported?** +A: The tool primarily supports PDFs for detailed analysis, but other asset types may be supported depending on the specific agent capabilities. -**Q: What types of files can I upload?** -A: Currently, PDFs and common image formats are supported for proof review. +**Q: How long does the analysis take?** +A: Analysis time varies based on asset complexity and content, but real-time WebSocket updates provide progress indicators. -**Q: Who can manage users?** -A: Only Super Admins and Oversight Admins have access to the "User Management" section. +**Q: Can I override the AI's verdict?** +A: Yes, users can flag items and add resolution notes, contributing to the audit trail. However, the AI provides the initial RAG status and feedback. -**Q: Can I export the analysis results?** -A: Yes, you can generate professional PDF reports of the analysis results for each proof. +**Q: Is my data secure?** +A: Yes, the system integrates with Azure AD for authentication and stores data in a secure PostgreSQL database. File storage is isolated. -**Q: What happens if the analysis fails?** -A: You will receive an error message. Check the "Profile" section for your account status or contact your Super Admin if issues persist. \ No newline at end of file +**Q: How do I get support?** +A: Contact support at BAICsupport@oliver.agency if you encounter issues. \ No newline at end of file diff --git a/01 Projects/olivas/DEVELOPER_MANUAL.md b/01 Projects/olivas/DEVELOPER_MANUAL.md index 4331a5d..de77beb 100644 --- a/01 Projects/olivas/DEVELOPER_MANUAL.md +++ b/01 Projects/olivas/DEVELOPER_MANUAL.md @@ -141,5 +141,8 @@ OliVAS is a full-stack web application with a clear separation between frontend 5. **AI Insights Disabled**: If `ANTHROPIC_API_KEY` is empty, the AI Insights tab will show a message indicating AI analysis is unavailable. ## Related +- [[01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] +- [[01 Projects/gmal-scope-builder/DEVELOPER_MANUAL]] - [[01 Projects/enterprise-ai-hub-nexus/DEVELOPER_MANUAL.md]] - [[01 Projects/oliver-ai-assistant/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL.md b/01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL.md index b1954f9..8af6456 100644 --- a/01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL.md +++ b/01 Projects/oliver-sales-ops-platform/DEVELOPER_MANUAL.md @@ -147,5 +147,8 @@ The platform is a full-stack web application with a React frontend and a Python 5. **Mailgun**: If `MAILGUN_API_KEY` is empty, emails are logged only. Ensure a key is set for production notifications. ## Related +- [[01 Projects/olivas/DEVELOPER_MANUAL]] +- [[01 Projects/cc-dashboard/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] - [[01 Projects/oliver-ai-assistant/DEVELOPER_MANUAL.md]] - [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/pimco-charts/DEVELOPER_MANUAL.md b/01 Projects/pimco-charts/DEVELOPER_MANUAL.md index 17d9dea..61ceb76 100644 --- a/01 Projects/pimco-charts/DEVELOPER_MANUAL.md +++ b/01 Projects/pimco-charts/DEVELOPER_MANUAL.md @@ -1,129 +1,137 @@ --- auto_generated: true -created: 2026-05-18 -manual_updated_at: 2026-05-18 -modified: 2026-05-18 -name: Developer_Manual -status: active -tags: -- client/oliver -- client/pimco -- domain/ai -- status/active -- tech/azure-ad -- tech/claude -- tech/docker -- tech/fastapi -- tech/python -- type/sop -type: sop +manual_updated_at: 2026-05-19 +modified: 2026-05-19 --- # PIMCO Chart Generator Developer Manual ## Architecture Overview +The PIMCO Chart Generator is a FastAPI-based web application that integrates an AI layer (Anthropic Claude) with a deterministic rendering engine. The architecture follows a pipeline pattern: -The PIMCO Chart Generator is a FastAPI-based web application that leverages the Claude AI model to interpret natural language requests and generate structured chart specifications. These specifications are then rendered into SVGs using a custom rendering engine. - -The system consists of the following key components: -- **Frontend**: HTML/JS templates for user interaction. -- **Backend**: FastAPI application handling requests, authentication, and orchestration. -- **AI Integration**: Uses Anthropic's Claude API to interpret briefs and refine chart specs. -- **Data Processing**: Pandas-based loaders and analyzers. -- **Rendering Engine**: Custom SVG generation using `drawsvg`. +1. **Input Layer**: Accepts file uploads and text briefs via HTTP. +2. **Data Processing Layer**: Loads, validates, and summarizes data using pandas. +3. **AI Layer**: Uses Claude Opus to interpret data summaries and briefs into a structured `ChartSpec` JSON object. +4. **Validation Layer**: Validates the `ChartSpec` against a schema and the underlying data. +5. **Rendering Layer**: Converts the `ChartSpec` into an SVG using the `drawsvg` library and custom layout algorithms. ## Tech Stack - - **Backend**: Python 3.10+, FastAPI, Uvicorn -- **Data Processing**: Pandas, OpenPyXL, NumPy -- **AI**: Anthropic Claude API (`claude-opus-4-6`) -- **Rendering**: `drawsvg` -- **Authentication**: Azure Active Directory (MSAL) -- **Deployment**: Docker, Docker Compose +- **Database/State**: In-memory session storage (Redis could be added for scaling) +- **AI**: Anthropic Claude API (specifically `claude-opus-4-6`) +- **Data Processing**: pandas, openpyxl +- **Rendering**: drawsvg (SVG generation) +- **Authentication**: Microsoft Azure AD (OAuth2) +- **Containerization**: Docker, Docker Compose ## Local Setup -1. **Prerequisites**: - - Docker and Docker Compose installed. - - Python 3.10+ (for local development without Docker). +### Prerequisites +- Python 3.10+ +- Docker and Docker Compose (for containerized setup) +- Git -2. **Configuration**: - - Copy `.env.example` to `.env`. - - Generate a `SESSION_SECRET_KEY` using: - ```bash - python -c "import secrets; print(secrets.token_hex(32))" - ``` - - Add your `ANTHROPIC_API_KEY` and Azure AD credentials to `.env`. +### Installation -3. **Dependencies**: +1. **Clone the Repository**: + ```bash + git clone + cd pimco-chart-generator + ``` + +2. **Environment Configuration**: + ```bash + cp .env.example .env + ``` + Edit `.env` with your specific credentials (see Environment Variables section). + +3. **Install Dependencies**: ```bash pip install -r requirements.txt ``` -4. **Run**: +4. **Run Locally**: ```bash - docker-compose up + uvicorn app.main:app --reload --host 0.0.0.0 --port 8569 + ``` + +5. **Run with Docker**: + ```bash + docker-compose up --build ``` ## Environment Variables | Variable | Description | Example | -|---|---|---| -| `ANTHROPIC_API_KEY` | API key for Claude AI | `sk-ant-xxx` | +|----------|-------------|---------| +| `ANTHROPIC_API_KEY` | API key for Anthropic Claude | `sk-ant-xxx...` | | `AZURE_TENANT_ID` | Azure AD Tenant ID | `e519c2e6-...` | | `AZURE_CLIENT_ID` | Azure AD Client ID | `9079054c-...` | -| `AZURE_REDIRECT_URI` | Azure OAuth Redirect URI | `https://ai-sandbox.oliver.solutions/Pimco-charts` | -| `SESSION_SECRET_KEY` | Secret for session signing | `` | +| `AZURE_REDIRECT_URI` | OAuth2 Redirect URI | `https://ai-sandbox.oliver.solutions/Pimco-charts` | +| `SESSION_SECRET_KEY` | Secret for session cookie encryption | `generate via python -c "import secrets; print(secrets.token_hex(32))"` | ## Key Services & Entry Points -- **`app/main.py`**: Entry point for the FastAPI application. Defines routes for authentication, data upload, and chart generation. -- **`app/data/loader.py`**: Handles loading Excel/CSV files. -- **`app/data/analyzer.py`**: Summarizes data characteristics for the AI model. -- **`app/ai/brief_interpreter.py`**: Interprets user briefs and refines chart specs using Claude. -- **`app/renderer/engine.py`**: Core rendering logic for generating SVGs. -- **`app/models/chart_spec.py`**: Dataclasses defining the structure of a chart specification. +### `app/main.py` +- **Entry Point**: FastAPI application initialization. +- **Key Routes**: + - `GET /`: Serves the login page or main dashboard based on auth state. + - `POST /generate`: Handles file upload, data loading, AI spec generation, and SVG rendering. + - `GET /static`: Serves frontend assets. + +### `app/data/loader.py` +- **Function**: `load_file(path) -> dict` +- **Purpose**: Reads Excel/CSV files and returns a dictionary of DataFrames (one per sheet). + +### `app/data/analyzer.py` +- **Function**: `summarize_data(df) -> dict` +- **Purpose**: Generates a statistical summary of the DataFrame (columns, dtypes, ranges) to feed into the AI prompt. + +### `app/ai/brief_interpreter.py` +- **Function**: `interpret_brief(data_summary, brief) -> ChartSpec` +- **Purpose**: Calls the Claude API with the system prompt, data summary, and user brief to generate the initial chart specification. + +### `app/renderer/engine.py` +- **Function**: `render_chart(spec, data) -> str` +- **Purpose**: Takes the `ChartSpec` and actual data, computes layout bounds, and generates the SVG string. + +### `app/models/chart_spec.py` +- **Purpose**: Defines Pydantic models for `ChartSpec`, `PanelSpec`, `AxisSpec`, `SeriesSpec`, etc., ensuring type safety and validation. ## API Reference -### Upload and Generate - -- **Endpoint**: `POST /generate` +### POST /generate +- **Content-Type**: `multipart/form-data` - **Parameters**: - - `file`: Uploaded file (Excel/CSV). - - `brief`: Natural language description of the chart. - - `sheet`: Name of the sheet in Excel files. - - `width`: Output SVG width (default: 2560). - - `height`: Output SVG height (default: 1440). -- **Response**: HTML response with the generated SVG embedded or links to download. - -### Authentication - -- **Endpoint**: `GET /` -- **Behavior**: Redirects to Azure AD login if not authenticated. Otherwise, renders the upload page. + - `file` (File, required): The data file (CSV/Excel). + - `brief` (str, required): The text description of the desired chart. + - `sheet` (str, optional): Specific sheet name if multiple sheets exist. + - `width` (int, optional): Output SVG width (default 2560). + - `height` (int, optional): Output SVG height (default 1440). +- **Response**: HTML page containing the rendered SVG. ## Deployment -1. **Docker**: - - Build the image: - ```bash - docker build -t pimco-charts:latest . - ``` - - Run with Docker Compose: - ```bash - docker-compose up -d - ``` +### Docker +The `Dockerfile` and `docker-compose.yml` are configured for production: +- Uses `uvicorn` with `--host 0.0.0.0`. +- Mounts `./output` to `/app/output` for persistent storage of rendered SVGs. +- Healthcheck monitors `http://localhost:8569/`. -2. **Production**: - - Ensure HTTPS is configured. - - Secure the `SESSION_SECRET_KEY` and `ANTHROPIC_API_KEY` in a production-grade secret manager. +### Nginx (Recommended for Production) +Reverse proxy the FastAPI app through Nginx to handle: +- SSL termination. +- Static file serving. +- Load balancing (if scaled). ## Known Gotchas -- **Font Rendering**: Ensure the fonts specified in `app/models/style.py` are installed on the rendering server to avoid fallback issues. -- **Memory Usage**: Large datasets may consume significant memory during the analysis phase. Monitor container resources. -- **Azure Redirect URI**: Must exactly match the deployed URL, including the `/Pimco-charts` root path. -- **Timezone Handling**: Dates are assumed to be in the timezone of the server unless specified in the data. Use UTC for consistency. +1. **Font Rendering**: The `drawsvg` library does not support custom font files directly. Ensure the system running the generator has the PIMCO brand fonts (Roboto Condensed, etc.) installed, or embed fonts if modifying the renderer. +2. **Session State**: Sessions are stored in-memory. Restarting the application clears all active sessions. For production, integrate a persistent session store (e.g., Redis). +3. **Azure Auth**: Ensure the `redirect_uri` in Azure Portal exactly matches the `AZURE_REDIRECT_URI` env var, including trailing slashes. +4. **Large Files**: Very large datasets (>100k rows) may slow down the data summarization and AI interpretation steps. Consider sampling or aggregating data before upload. +5. **SVG Complexity**: Extremely complex charts with many series may produce large SVG files. Monitor file sizes for web delivery. ## Related -- [[01 Projects/ac-helper/DEVELOPER_MANUAL.md]] \ No newline at end of file + +- [[01 Projects/pimco-charts/USER_MANUAL]] diff --git a/01 Projects/pimco-charts/USER_MANUAL.md b/01 Projects/pimco-charts/USER_MANUAL.md index 6116723..007e0ad 100644 --- a/01 Projects/pimco-charts/USER_MANUAL.md +++ b/01 Projects/pimco-charts/USER_MANUAL.md @@ -1,95 +1,84 @@ --- auto_generated: true -created: 2026-05-18 -manual_updated_at: 2026-05-18 -modified: 2026-05-18 -name: User_Manual -status: active -tags: -- client/pimco -- domain/ai -- domain/analytics -- status/active -- tech/azure-ad -- tech/claude -- tech/docker -- tech/fastapi -- type/sop -type: sop +manual_updated_at: 2026-05-19 +modified: 2026-05-19 --- # PIMCO Chart Generator User Manual ## What This Tool Does - -The PIMCO Chart Generator is an automated tool that creates publication-quality charts matching PIMCO's InDesign visual style. It allows users to upload data files and describe their desired chart in plain English. The system processes this input to generate pixel-perfect SVG charts that can be used across various platforms. - -Key features: -- **Natural Language Interaction**: Describe your chart requirements using everyday language. -- **Iterative Refinement**: Easily modify charts with commands like "make lines thicker" or "remove Japan". -- **Standardized Styling**: All charts automatically conform to PIMCO's corporate visual identity. -- **Multiple Data Formats**: Supports Excel and CSV file uploads. +The PIMCO Chart Generator is an automated tool that creates publication-quality charts matching PIMCO's InDesign visual style. It allows users to upload financial data and describe their visualization needs in plain English. The system then generates pixel-perfect SVG charts ready for any platform, with the ability to iterate on the design using natural language commands (e.g., "make lines thicker," "change the title," "remove Japan"). ## Who Uses It - -This tool is designed for: -- **Financial Analysts**: Who need to create consistent, branded charts for reports and presentations. -- **Communications Teams**: Who prepare materials requiring strict adherence to brand guidelines. -- **Data Scientists**: Who need to quickly visualize data in a professional format. +This tool is designed for financial analysts, data scientists, and communicators at PIMCO who need to quickly generate high-quality, branded charts from raw data without manual design work in tools like Excel or Illustrator. ## How to Access -1. **Local Deployment**: - - Clone the repository. - - Set up the virtual environment and install dependencies. - - Configure your `.env` file with API keys and Azure credentials. - - Run the application using `docker-compose up`. - - Access the application at `http://localhost:8569/Pimco-charts`. +### Prerequisites +- An active internet connection. +- Valid Azure credentials (provided by your organization). -2. **Cloud Deployment**: - - Deploy the Docker container to your preferred cloud provider. - - Ensure the `SESSION_SECRET_KEY` and API keys are securely configured. +### Login +1. Open your web browser and navigate to the application URL (e.g., `https://ai-sandbox.oliver.solutions/Pimco-charts`). +2. You will be redirected to the Microsoft Login page. +3. Sign in with your corporate Azure credentials. +4. Upon successful authentication, you will be redirected to the main dashboard. ## Main Workflows -### Workflow 1: Creating a Chart +### Workflow 1: Generate a New Chart -1. **Upload Data**: - - Navigate to the upload page. - - Select your Excel or CSV file. - - Choose the specific sheet if uploading an Excel file. +1. **Upload Data**: + - On the main dashboard, click "Upload File". + - Select your Excel (.xlsx) or CSV file containing your dataset. + - The system will automatically analyze the columns, data types, and ranges. 2. **Describe Your Chart**: - - In the text box, describe the chart you want. For example: "Plot U.S. and Germany bond yields over the last 10 years as a line chart with a yellow background." + - In the "Chart Brief" text area, describe what you want using plain English. + - Examples: + - "Show the trend of US and German 10-year bond yields from 2020 to 2025." + - "Create a bar chart comparing quarterly GDP growth by sector." -3. **Generate**: - - Click the "Generate" button. - - The system will process your request, interpret the data, and render the SVG. +3. **Review and Generate**: + - Click "Generate Chart". + - The system will use AI to interpret your brief and the data summary to create a `ChartSpec`. + - The SVG chart will be rendered and displayed on the screen. -4. **Review and Refine**: - - View the generated chart. - - If changes are needed, provide new instructions (e.g., "Change the title font to bold"). - - The system will update the chart based on your feedback. +4. **Iterate with Natural Language**: + - If the chart isn't exactly what you need, use the "Refine Chart" text area. + - Type instructions like: + - "Change the title to 'Global Bond Yields'." + - "Make the US line thicker and blue." + - "Remove Japan from the legend." + - Click "Refine" to see the updated chart. -5. **Download**: - - Save the final SVG file for use in your documents. +5. **Export**: + - Once satisfied, click "Download SVG" to save the file to your computer. + - The SVG is resolution-independent and can be inserted into InDesign, PowerPoint, or web pages. + +### Workflow 2: Iterate on an Existing Chart + +1. After generating a chart, use the refinement prompts below the chart preview. +2. The system maintains the context of the previous chart specification. +3. Apply changes iteratively until the output matches your requirements. ## FAQ -**Q: What data formats are supported?** -A: Currently, Excel (.xlsx, .xls) and CSV (.csv) files are supported. +**Q: What file formats are supported?** +A: The tool accepts CSV files and Excel files (.xlsx). Ensure your data has a clear header row. -**Q: Can I customize the colors beyond the PIMCO palette?** -A: The chart generator adheres to PIMCO's brand guidelines. You can specify colors using the PIMCO palette via natural language commands, but custom hex codes are not supported to maintain brand consistency. +**Q: Can I customize the colors beyond the default PIMCO palette?** +A: The tool currently uses the standard PIMCO color palette. For custom colors, you may need to edit the SVG directly or request this feature. -**Q: How do I handle large datasets?** -A: The application is optimized for performance. However, for very large datasets, consider aggregating the data before uploading to improve processing speed. +**Q: How accurate is the AI interpretation?** +A: The AI uses the data summary and your brief to generate a structured specification. While highly accurate for standard chart types, complex or ambiguous requests may require clarification in subsequent refinement steps. -**Q: Is my data stored?** -A: Data is temporarily stored in the `output` directory during the generation process. After the session or container restart, the data is typically cleared. Ensure sensitive data is handled according to your organization's security policies. +**Q: Where are my files stored?** +A: Uploaded files are processed temporarily and stored in the `output/` directory. They are not permanently stored on the server after the session ends. -**Q: How do I iterate on a chart?** -A: Simply provide new natural language instructions in the brief field after the initial generation. The system uses the previous chart as a baseline and applies your new instructions. +**Q: Is the output SVG responsive?** +A: Yes, SVGs are vector-based and scale infinitely without loss of quality. However, the generator produces fixed-width outputs (default 2560px) suitable for high-resolution printing and digital displays. ## Related -- [[01 Projects/pimco-charts/DEVELOPER_MANUAL.md]] \ No newline at end of file + +- [[01 Projects/pimco-charts/DEVELOPER_MANUAL]] diff --git a/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md b/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md index ff444fd..c3ab85f 100644 --- a/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md +++ b/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md @@ -2,7 +2,7 @@ auto_generated: true created: 2026-05-18 manual_updated_at: 2026-05-18 -modified: 2026-05-18 +modified: 2026-05-19 name: Developer_Manual status: active tags: @@ -143,7 +143,10 @@ The backend exposes three main tools via FastMCP: - **Port Mapping**: PostgreSQL is mapped to host port 5433, not 5432. Redis to 6380. Adjust local database clients accordingly. ## Related +- [[01 Projects/presenton/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] - [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] - [[01 Projects/enterprise-ai-hub-nexus/DEVELOPER_MANUAL]] - [[01 Projects/Oliver-ai-bot_2.0/DEVELOPER_MANUAL]] -- [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] +- [[01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/video-accessibility-old/DEVELOPER_MANUAL.md b/01 Projects/video-accessibility-old/DEVELOPER_MANUAL.md index 9686882..5e4fec6 100644 --- a/01 Projects/video-accessibility-old/DEVELOPER_MANUAL.md +++ b/01 Projects/video-accessibility-old/DEVELOPER_MANUAL.md @@ -195,6 +195,8 @@ celery -A celery_app worker -Q whisper -c 1 5. **MSAL Config**: Ensure `msalConfig.ts` matches the Azure AD redirect URIs exactly, including the `basename` (`/video-accessibility`) if deployed to a sub-path. ## Related +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL]] +- [[01 Projects/ac-helper/DEVELOPER_MANUAL]] - [[01 Projects/oliver-ai-assistant/DEVELOPER_MANUAL.md]] - [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] - [[01 Projects/ac-helper/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/99 Daily/2026-05-19.md b/99 Daily/2026-05-19.md index ab65066..9bced29 100644 --- a/99 Daily/2026-05-19.md +++ b/99 Daily/2026-05-19.md @@ -29,3 +29,4 @@ tags: [daily] - 11:30 | `aimpress` - **Asked:** Clean up PVE server by removing unused services and orphaned monitors. - **Done:** Deleted Media stack (CT111), Plane, Actual Budget, and removed Homarr monitor from Uptime Kuma database. +- 11:58 (<1min) — session ended | `ai_leed`