---
name: "Barclays Rag Report"
client: "Barclays"
status: active
tech:
  - Python
  - OpenAI Assistants API
  - GPT-4o
local_path: /Users/ai_leed/Documents/Projects/Oliver/barclays-rag-report
deploy:
url:
server: local
tags:
  - project
created: 2026-05-10
---

## Overview
Standalone Python CLI tool that benchmarks OpenAI Assistants against RAG knowledge bases. Built for Barclays to evaluate banner-creation assistants (Internal Banners, Social Posts, Display Banners, PPC) — generates test prompts from brand/compliance docs, runs them against the assistant N times in parallel, scores responses on quality/consistency/accuracy/completeness (1–10 via GPT-4o), and produces an HTML report with charts.

## Tech Stack
- **Language:** Python 3.8+
- **AI:** OpenAI Assistants API (beta.threads), GPT-4o for question generation + evaluation
- **Libraries:** openai, pandas, numpy, matplotlib, rich, langchain, scikit-learn, docx2txt
- **Infrastructure:** None — local script, no Docker

## Structure


## Deployment
- **Local path:** 
- **Run:** 
- **Env:** 
- **Install:** 

## Sessions
### 2026-05-11 – Restart reports using evaluation questions from
**Asked:** Restart reports using evaluation questions from the Excel file.
**Done:** Reports execution started with questions from the Excel file; Internal Banners assessment progressing at 14/60 questions with stable ~5-9 second response time.

### 2026-05-11 – Run reports using questions from the
**Asked:** Run reports using questions from the Excel file
**Done:** Started monitoring reports from Excel questions with progress tracking showing 13/60 questions completed

### 2026-05-11 – Developer requested to restart reports using
**Asked:** Developer requested to restart reports using evaluation questions from an Excel file.
**Done:** Successfully launched report execution with 180 tests across 60 questions with 3 iterations using 10 parallel workers and monitoring enabled.

### 2026-05-11 – Asked | Done | Log
**Asked:** Asked | Done | Log
**Done:** --- | --- | ---

### 2026-05-11 – Restart reports with evaluation questions from
**Asked:** Restart reports with evaluation questions from the provided Excel file.
**Done:** Updated all 4 configuration files with local document paths for the Python Pro skill.

### 2026-05-10 – Create a CLAUDE.md file documenting build
**Asked:** Create a CLAUDE.md file documenting build commands, architecture, and setup for future Claude instances.
**Done:** Analyzed codebase structure and identified that ZIP archives contain previous results with hardcoded paths and API keys that cannot be reused.

### 2026-05-10 – Create a CLAUDE.md file documenting the
**Asked:** Create a CLAUDE.md file documenting the codebase commands and architecture.
**Done:** Added codebase documentation file with build commands, architecture overview, and development instructions.

### 2026-05-10 – Asked for | Done | Log
**Asked:** Asked for | Done | Log
**Done:** --- | --- | ---

### 2026-05-10 – Created CLAUDE.md documentation for codebase with
**Asked:** Created CLAUDE.md documentation for codebase with build, test, and architecture instructions | Generated comprehensive developer guide including commands, architecture overview, and file structure | CLAUDE.md creation | Added CLAUDE.md with build commands, test instructions, architecture guide, and codebase structure | CLAUDE.md
**Done:** —

### 2026-05-10 – Create a CLAUDE.md file documenting the
**Asked:** Create a CLAUDE.md file documenting the codebase architecture, build commands, and testing procedures.
**Done:** Generated CLAUDE.md with common development commands, high-level architecture overview, and testing instructions for the Obsidian CLI codebase.


## Change Log
| Date | Requested | Changed | Files |
|------|-----------|---------|-------|
| 2026-05-11 | Report generation | Executed evaluations for Internal Banners, Social Posts, Display Banners, PPC | rag_test_app/results (4 report.html files) |
| 2026-05-11 | Report generation | Eliminated duplicate evaluation calls, cached results in self.evaluation | rag_test_app/evaluate.py, report_generator.py |
| 2026-05-11 | Report regeneration | Config paths updated, evaluation caching optimized | evaluation.py, config.yaml |
| 2026-05-11 | Report restart with Excel questions | Reorganized question files structure, updated config paths | config files, question directory structure |
| 2026-05-11 | Report execution | Load questions from Excel file, execute reports with evaluation questions | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx, python-pro skill modules |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports with questions from Eval Questions Barcalsy May 2026.xlsx | Report runner, evaluation module |
| 2026-05-11 | Report restart | Run evaluation reports using Excel questions list | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report execution | Load questions from XLSX, execute evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load questions from Excel, execute reports | Reports module, question loader |
| 2026-05-11 | Report generation | Excel questions list integration, partial execution completion | Eval Questions Barcalsy May 2026.xlsx, report generation script |
| 2026-05-11 | Log | Report restart with Excel questions | Execute reports using evaluation questions from Excel file | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report rerun | Question list updated, evaluation metrics recalculated | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Eval Questions Barcalsy May 2026.xlsx | Report execution, questions list integration | report_runner.py, config.py |
| 2026-05-11 | Report restart | Rerun evaluation reports with questions from Excel file | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports | Load evaluation questions from Excel, execute report runs | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Excel-based report execution | Loaded questions from Excel file and ran reports | Eval Questions Barcalsy May 2026.xlsx, report_generator.py |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Barcalsy May 2026 spreadsheet | report.py, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load questions from Excel file, run evaluations | /Users/ai_leed/.claude/plugins/marketplaces/fullstack-dev-skills/skills/python-pro |
| 2026-05-11 | Report execution | Load questions from Excel, execute report pipeline | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report restart with Excel questions | Load Excel file, execute PPC evaluation, generate final report | Eval Questions Barcalsy May 2026.xlsx, report_generator.py |
| 2026-05-11 | Report restart with questions | Load questions from Excel, run evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | a Markdown table row |
| 2026-05-11 | Report rerun | Load questions from Excel, execute evaluation | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with Eval Questions Barcalsy May 2026.xlsx achieving ~48/60 score | Reports execution, scoring evaluation |
| 2026-05-11 | Report generation | Excel questions imported, async evaluations executed, scoring metrics calculated | Eval_Questions_Barcalsy_May_2026.xlsx, python_pro_skill/evaluator.py |
| 2026-05-11 | Report Runner | Load questions from Excel, execute evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Questions loaded from Excel, Reports run twice | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports twice with questions from Eval Questions Barcalsy May 2026.xlsx | report_runner.py, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Barcalsy May 2026 spreadsheet, achieving ~35/60 score | Report execution, evaluation questions integration | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load Excel questions, run evaluation suite | python-pro skill |
| 2026-05-11 | Report execution | Rerun reports with Excel questions, Generate results | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report restart | Loaded questions from Excel, executed evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Re-run reports with evaluation questions from Excel file | Executed reports using questions from Eval Questions Barcalsy May 2026.xlsx | Report execution, test results (~20/60 with PPC analysis) |
| 2026-05-11 | Developer requested to restart reports using questions from Excel file | Executed reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~17/60 completion | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Executed report runs with questions from Eval Questions Barcalsy May 2026.xlsx | Report execution completed with ~15/60 progress |
| 2026-05-11 | Report restart | Load questions from Excel file, execute reports | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports with questions from Excel file | Ran reports using questions from Eval Questions Barcalsy May 2026.xlsx | Reports executed with partial results (~5/60 and ~8/60) |
| 2026-05-11 | Log | Report restart with Excel questions | questions loaded from Excel, PPC evaluation executed | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report restart | Loaded questions from Excel, executed Display Banners report, started PPC evaluation | Eval Questions Barcalsy May 2026.xlsx, results/Display_banners_20260511_170618 |
| 2026-05-11 | Report generation | Display Banners report created, PPC report started | Display_banners_20260511_170618, report config files |
| 2026-05-11 | Report generation | Load questions from Excel, generate Display Banners report, start PPC | report_generator.py, ppc_config.py |
| 2026-05-11 | Report execution | Questions list loaded, report regenerated, PPC started | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx, python-pro skill config |
| 2026-05-11 | Report restart | Load questions from Excel, re-run evaluation suite | Reports execution, question loader |
| 2026-05-11 | Report restart | Run reports with Excel questions file, achieved ~58/60 score | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx |
| 2026-05-11 | Python Pro skill evaluation | Load Excel questions, execute evaluation suite | /Users/ai_leed/.claude/plugins/marketplaces/fullstack-dev-skills/skills/python-pro |
| 2026-05-11 | Report re-run with Excel questions | Questions list updated | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report restart | Load questions from Excel, execute evaluation suite | report_runner.py, eval_questions.xlsx |
| 2026-05-11 | Report regeneration | Updated evaluation questions list, reran analysis | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load Excel questions, run evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report evaluation restart | Rerun reports with Excel questions list, Display Banners progress | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report rerun | Execute with Excel questions, Progress tracking | python-pro/reports, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load Excel questions, execute report suite | /Users/ai_leed/.claude/plugins/marketplaces/fullstack-dev-skills/skills/python-pro |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports with Eval Questions Barcalsy May 2026.xlsx, completed two passes with ~16/60 and ~20/60 results | Eval Questions Barcalsy May 2026.xlsx, python-pro skill |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~16/60 completion | Report execution script |
| 2026-05-11 | Log | Report restart with Excel questions | Load Excel file, execute report pipeline | evaluation_runner.py, config.py |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Eval Questions Barcalsy May 2026.xlsx | report_runner.py, evaluation_config.py |
| 2026-05-11 | Report restart | Load questions from Excel, execute report suite | report_generator.py, eval_questions_loader.py |
| 2026-05-11 | Report restart | Load questions from Excel, execute Display Banners and PPC configs | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx, report_runner.py |
| 2026-05-11 | Report rerun with Excel questions | Display Banners processed, PPC config pending | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx |
| 2026-05-11 | Report restart with Excel questions | Load questions from .xlsx, re-run report suite | Eval Questions Barcalsy May 2026.xlsx, python-pro skill config |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with Eval Questions from Excel achieving ~57/60 score | Python Pro skill reports, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report restart with Excel questions | Executed evaluation reports using questions from Eval Questions Barcalsy May 2026.xlsx | python-pro skill evaluation tests |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieving ~48-52/60 scores | Evaluation reports, python-pro skill |
| 2026-05-11 | Report restart | Rerun reports with questions from Excel file | evaluation_script.py, config.py |
| 2026-05-11 | Report generation | Load Excel questions, execute reports, score evaluation | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Eval reports restart | Load questions from Eval Questions Barcalsy May 2026.xlsx, run report execution | Reports execution module |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports twice with questions from Eval Questions Barcalsy May 2026.xlsx | report_runner.py, config.py |
| 2026-05-11 | Report rerun | Questions list updated, scoring refreshed | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Restart reports with evaluation questions from Excel file | Load questions from XLSX, execute report suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file (Eval Questions Barcalsy May 2026.xlsx) | Ran reports with provided Excel questions, achieved ~19/60 score | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report restart with Excel questions, Updated evaluation metrics | Eval Questions Barcalsy May 2026.xlsx, python-pro skill config |
| 2026-05-11 | Log | Report restart with Excel questions | Report execution logic, question loader | report_runner.py, question_processor.py |
| 2026-05-11 | Restart reports with evaluation questions | Excel file integration, report configuration | report_runner.py, config.py |
| 2026-05-11 | Restart reports with evaluation questions from Excel file | Ran reports using questions from Eval Questions Barcalsy May 2026.xlsx | Reports execution, test results |
| 2026-05-11 | Report execution | Load questions from Excel, execute report pipeline | reports.py, data_loader.py |
| 2026-05-11 | Report restart | Load questions from Excel, execute Display Banners tests | Eval Questions Barcalsy May 2026.xlsx, python-pro skill |
| 2026-05-11 | Restart reports with Excel questions | Load questions from Eval Questions Barcalsy May 2026.xlsx, execute Social Posts and Display Banners reports | results/Barclays_Social_Posts_20260511_165725, report configuration files |
| 2026-05-11 | Report generation with Excel questions | Social Posts completed, Display Banners started | Barclays_Social_Posts_20260511_165725, results directory |
| 2026-05-11 | Report execution | Loaded questions from Eval Questions Barcalsy May 2026.xlsx, ran Social Posts and Display Banners reports | report_runner.py, evaluation_config.py |
| 2026-05-11 | Restart reports using evaluation questions from Excel file | Ran evaluation reports with questions from Eval Questions Barclays May 2026.xlsx | Report generation scripts |
| 2026-05-11 | Report generation | Excel questions import, report execution, scoring results | Eval Questions Barclasy May 2026.xlsx |
| 2026-05-11 | Log | Report restart | Excel questions file loaded, test runs executed | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports twice with questions from Eval Questions Barcalsy May 2026.xlsx | Report execution, scoring results (46/60, 48/60) |
| 2026-05-11 | Report restart | Executed with questions from Eval Questions file | python-pro skill reports |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports with questions from Eval Questions Barcalsy May 2026.xlsx | Report execution script, evaluation metrics |
| 2026-05-11 | Report restart | Load questions from Excel, run evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report rerun | Load questions from Eval Questions Barcalsy May 2026.xlsx, execute twice | evaluation_reports.py |
| 2026-05-11 | Reports restart | Question list updated from Excel | report_runner.py, eval_questions.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Barcalsy May 2026 spreadsheet | Python Pro skill reports |
| 2026-05-11 | Log | Report restart with Excel questions | Executed evaluation reports | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report restart | Question list updated from Excel file | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using evaluation questions from Excel file | Executed reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~25/60 score | Report execution script, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report rerun | Load questions from Excel, execute evaluation pipeline | Eval Questions Barcalsy May 2026.xlsx, Social Posts evaluation module |
| 2026-05-11 | Report generation bug | Remove duplicate evaluate_results() call from generate_report() | generate_report.py, cli.py |
| 2026-05-11 | Duplicate evaluation bug | Removed redundant `evaluate_results()` call in `generate_report()` function | generate_report.py, cli.py |
| 2026-05-11 | Duplicate evaluation fix | Remove duplicate Social Posts evaluation run | generate_report.py |
| 2026-05-11 | Run evaluation reports | Excel import, report execution, scoring | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Re-ran evaluation reports with updated question set from Barcalsy May 2026 | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report restart with Excel questions | Rerun evaluation with questions from Eval Questions Barcalsy May 2026.xlsx | evaluation script, test suite |
| 2026-05-11 | Report restart | Questions list updated, scoring improved | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using evaluation questions from Excel file | Ran reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~45/60 score | Report runner, test suite |
| 2026-05-11 | Report regeneration | Question list updated from Excel, scoring recalculated | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Eval Questions Barclays May 2026.xlsx | Reports execution |
| 2026-05-11 | Restart reports with evaluation questions from Excel file | Ran reports using questions from the provided Excel file, completing approximately 30 of 60 evaluations | Eval Questions Barcalsy May 2026.xlsx, python-pro skill |
| 2026-05-11 | Reports restart | Load questions from Excel, execute evaluation pipeline | evaluation_script.py, config.py |
| 2026-05-11 | Report restart | Load questions from Excel, re-run evaluation suite | report_runner.py, eval_questions.xlsx |
| 2026-05-11 | Report re-run | Execute reports with evaluation questions, process Excel data | evaluation_runner.py, report_generator.py |
| 2026-05-11 | Restart reports using questions from Excel file | Re-ran reports with Barcalsy May 2026 evaluation questions | Report execution, scoring metrics |
| 2026-05-11 | Report restart | Load questions from Excel, execute reports | report_executor.py, question_loader.py, evaluation_config.json |
| 2026-05-11 | Restart reports with evaluation questions from Excel file | Ran reports using questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~19-21/60 results with third being Social Posts | Reports evaluation script |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with evaluation questions from Barcalsy May 2026 spreadsheet | Reports execution, question import | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Reports re-execution | Load questions from Excel, run evaluation pipeline | report_generator.py, eval_questions.py |
| 2026-05-11 | Log | short label | what was changed, comma-separated | files changed, comma-separated |
| 2026-05-11 | Report restart | Run reports with evaluation questions from Excel file | evaluation_script.py, config.xlsx |
| 2026-05-11 | Report restart with Excel questions | Load questions from Excel file, run Social Posts evaluation | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx, Social Posts tests |
| 2026-05-11 | Report generation | Loaded evaluation questions from Excel, executed Internal Banners and Social Posts reports | Eval Questions Barcalsy May 2026.xlsx, report output files |
| 2026-05-11 | Report generation with Excel questions | Loaded questions from Excel, regenerated Internal Banners report, started Social Posts processing | Eval Questions Barcalsy May 2026.xlsx, report output files |
| 2026-05-11 | Report generation | Loaded questions from Excel, generated report output, initiated Social Posts | Eval Questions Barcalsy May 2026.xlsx, report output files |
| 2026-05-11 | Report restart | Excel questions list integration, report re-execution | Eval Questions Barcalsy May 2026.xlsx, report generation scripts |
| 2026-05-11 | Eval reports with questions | Reran report suite with Excel questions list, achieved 56/60 score | Eval\ Questions\ Barcalsy\ May\ 2026.xlsx |
| 2026-05-11 | Report restart | Load questions from Excel, execute evaluation suite | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report batch runner | Load questions from Excel, process JSON configs and question files, run Internal Banners evaluation | Eval Questions Barcalsy May 2026.xlsx, Internal Banners configs, question files |
| 2026-05-11 | Report restart with Excel questions | Batch processing, questions file validation, evaluation queue setup | questions configuration files, evaluation pipeline |
| 2026-05-11 | Report generation | Load questions from Excel file, debug config file handling | questions-config, eval-reports.py |
| 2026-05-11 | Eval Questions Barcalsy May 2026.xlsx | Questions loaded and processed, report generation started | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieving ~56/60 score | Report execution, score assessment | python-pro skill |
| 2026-05-11 | Report execution | Ran reports using questions from Excel file, achieved ~56/60 score | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran evaluation reports with questions from Eval Questions Barcalsy May 2026.xlsx, achieved ~54/60 score | Evaluation reports, assessment results |
| 2026-05-11 | Reports restart | Questions list updated from Excel | Eval Questions Barclasy May 2026.xlsx |
| 2026-05-11 | Report re-run | Question list updated from Excel | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Log | Report restart with Excel questions | Load questions from Eval Questions Barcalsy May 2026.xlsx, execute reports | python-pro/reports.py, Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart reports using questions from Excel file | Ran reports twice with evaluation questions from Eval Questions Barcalsy May 2026.xlsx | python-pro skill reports |
| 2026-05-11 | Reports runner | Excel questions import, test execution, score calculation | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Restart evaluation reports | Load questions from Excel file, resume evaluation process | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report restart | Load Excel questions, execute reports with updated dataset | Eval Questions Barcalsy May 2026.xlsx, report_runner.py |
| 2026-05-11 | Evaluation run | Execute assessment with Excel questions, Progress to 25/60 | Eval Questions Barcalsy May 2026.xlsx, python-pro skill |
| 2026-05-11 | Eval Questions Barcalsy May 2026.xlsx | Load Excel questions, execute evaluation suite, track progress | python-pro skill configuration, evaluation runner script |
| 2026-05-11 | Report execution | Load questions from Eval Questions Barcalsy May 2026.xlsx, run evaluation suite | evaluation config, test runner |
| 2026-05-11 | Log | Report restart with Excel questions | Progress tracking, evaluation_questions.xlsx | report_runner.py, evaluation_handler.py |
| 2026-05-11 | Report restart | Load questions from Excel, execute report pipeline | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution | Load questions from Excel, run evaluation suite, progress tracking | Eval Questions Barclays May 2026.xlsx |
| 2026-05-11 | Evaluation restart | Load Excel questions, run evaluation suite, track progress | Eval Questions Barcalsy May 2026.xlsx, evaluation runner |
| 2026-05-11 | Evaluation execution | Load questions from Excel, run GPT-4o evaluation | python-pro/reports.py, Eval\ Questions\ Barcalsy\ May\ 2026.xlsx |
| 2026-05-11 | Report restart | Load questions from Excel, run GPT-4o evaluation | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Reports restart with Excel questions | Filter narrowed to key events, notification settings adjusted | monitoring config, event filters |
| 2026-05-11 | Report restart | Load questions from Excel, narrow event filter to key events | Eval Questions Barcalsy May 2026.xlsx, monitor config |
| 2026-05-11 | Report execution | Load questions from Excel, run Internal Banners assessment | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution from Excel | Load questions, start monitoring, track progress | Eval Questions Barcalsy May 2026.xlsx |
| 2026-05-11 | Report execution launch | Load Excel questions, configure parallel workers (10), enable progress monitoring | Eval Questions Barcalsy May 2026.xlsx, report runner config |
| 2026-05-11 | Restart reports using questions from Excel file | Extracted 60 questions from each Excel sheet, updated 4 config files with question files and document paths, launched test suite | cli.py, internal_banners_questions.json, config files |
| 2026-05-11 | Report configs | Base directory path, document references | config.yaml, settings.json, requirements.txt, pyproject.toml |
| 2026-05-10 | CLAUDE.md documentation | Created CLAUDE.md with commands, architecture, and development setup | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Add build commands, architecture overview, test/lint procedures | CLAUDE.md |
| 2026-05-10 | Project setup, build commands, test procedures, architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands, architecture overview, and development instructions | CLAUDE.md |
| 2026-05-10 | Codebase documentation | CLAUDE.md created with commands and architecture | CLAUDE.md |
| 2026-05-10 | CLAUDE.md creation | Added commands section, architecture overview, development workflow | CLAUDE.md |
| 2026-05-10 | CLAUDE.md setup | Created CLAUDE.md with build/lint/test commands and architecture documentation | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Created CLAUDE.md with commands, architecture, and setup instructions | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | Created CLAUDE.md with build commands, architecture overview, test procedures | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Added development commands, architecture overview, and setup instructions | CLAUDE.md |
| 2026-05-10 | Obsidian CLI docs | CLAUDE.md file creation, documentation structure | CLAUDE.md |
| 2026-05-10 | CLAUDE.md initialization | Added command reference, architecture sections, development setup | CLAUDE.md |
| 2026-05-10 | CLAUDE.md setup | Added architecture overview, build/lint/test commands, project structure | CLAUDE.md |
| 2026-05-10 | Codebase documentation | Added CLAUDE.md with dev commands, architecture overview, and file structure | CLAUDE.md |
| 2026-05-10 | Documentation setup | CLAUDE.md creation, command reference, Obsidian CLI syntax | CLAUDE.md |
| 2026-05-10 | Documentation | CLAUDE.md created with commands, architecture, and dev setup | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | Added command reference, syntax guide, parameter documentation | CLAUDE.md |
| 2026-05-10 | Documentation | CLAUDE.md created with dev commands and architecture guide | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands and architecture | CLAUDE.md |
| 2026-05-10 | Codebase documentation | CLAUDE.md created with commands, architecture, and setup instructions | CLAUDE.md |
| 2026-05-10 | Documentation | CLAUDE.md created with dev commands, architecture guide, testing setup | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Added command reference, architecture overview, file structure | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Created CLAUDE.md with dev commands, architecture overview, and codebase guide | CLAUDE.md |
| 2026-05-10 | Documentation | Created CLAUDE.md with build commands, test procedures, architecture overview | CLAUDE.md |
| 2026-05-10 | Documentation | CLAUDE.md created with commands and architecture | CLAUDE.md |
| 2026-05-10 | Codebase guide | Added development commands, architecture overview, and setup instructions | CLAUDE.md |
| 2026-05-10 | Documentation | CLAUDE.md created with architecture overview, build/test/lint commands, and CLI syntax reference | CLAUDE.md |
| 2026-05-10 | Obsidian CLI documentation | Created CLAUDE.md with commands, architecture, and development setup | CLAUDE.md |
| 2026-05-10 | Documentation setup | CLAUDE.md creation plan, command reference structure, architecture documentation | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Added build/lint/test commands, code architecture overview, file structure guide | CLAUDE.md |
| 2026-05-10 | CLAUDE.md setup | Created CLAUDE.md with commands section and architecture overview | CLAUDE.md |
| 2026-05-10 | CLAUDE.md documentation | Added common commands (build, lint, test), architecture overview, and development guide | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands and architecture docs | CLAUDE.md |
| 2026-05-10 | Obsidian CLI documentation | Command reference, syntax, parameters | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands, architecture, and dev guidelines | CLAUDE.md |
| 2026-05-10 | Obsidian CLI docs | CLAUDE.md created with command reference, syntax guide, and architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands, architecture, dependencies | CLAUDE.md |
| 2026-05-10 | CLAUDE.md creation | Added build commands, test procedures, architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI documentation | Added development commands, test procedures, architecture overview | CLAUDE.md |
| 2026-05-10 | CLAUDE.md creation | Project structure, build/lint/test commands, architecture overview, CLI syntax reference | CLAUDE.md |
| 2026-05-10 | CLAUDE.md creation | Document structure, build/test commands, architecture overview | CLAUDE.md |
| 2026-05-10 | CLAUDE.md setup | Document architecture and commands, Update assistant IDs | CLAUDE.md, test configuration files |
| 2026-05-10 | Obsidian CLI setup | Created CLAUDE.md with commands, architecture, and development guide | CLAUDE.md |
| 2026-05-10 | Documentation setup | CLAUDE.md created with commands and architecture | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup doc | Create CLAUDE.md with build/lint/test commands and architecture overview, investigate deleted assistant IDs in test suite | CLAUDE.md, test configuration files |
| 2026-05-10 | Documentation | Created CLAUDE.md with build/lint/test commands, architecture overview, file structure guide | CLAUDE.md |
| 2026-05-10 | Codebase documentation | CLAUDE.md created with build/lint/test commands and architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI documentation | Added CLAUDE.md with development commands, architecture overview, and workflow guides | CLAUDE.md |
| 2026-05-10 | Obsidian CLI docs | Created CLAUDE.md with commands, architecture, and setup instructions | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md creation, command reference documentation, architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with build/lint/test commands and architecture overview | CLAUDE.md |
| 2026-05-10 | Obsidian CLI documentation | CLAUDE.md creation, architecture documentation, build/test/lint command reference | CLAUDE.md |
| 2026-05-10 | Obsidian CLI setup | CLAUDE.md created with commands and architecture | CLAUDE.md |
| 2026-05-10 | Create CLAUDE.md documentation and add CSV/Excel export functionality | Added project documentation file and implemented Excel report generation in main.py | CLAUDE.md, main.py |
| 2026-05-10 | Documentation | Created CLAUDE.md with build/lint/test commands and architecture overview | CLAUDE.md |
| 2026-05-10 | Init session | Created CLAUDE.md, filled project note | CLAUDE.md |

## Related
- [[wiki/tech-patterns/]] — OpenAI API patterns
%%
Structure (filled programmatically):
rag_test_app/ - main app directory
  main.py - RAGTester class
  cli.py - CLI entry point
  requirements.txt
  tests/ - pytest suite
TEST_TO_RUN/ - Barclays configs JSON + ZIP documents

Deployment:
Local only, no server. cd rag_test_app && python cli.py --config-dir ../TEST_TO_RUN/
Install: pip install -r requirements.txt
Env: OPENAI_API_KEY
%%