diff --git a/99 Daily/2026-04-27.md b/99 Daily/2026-04-27.md index deed053..44c7c06 100644 --- a/99 Daily/2026-04-27.md +++ b/99 Daily/2026-04-27.md @@ -413,3 +413,6 @@ tags: [daily] - 18:01 | `aimpress` - **Asked:** Asked | Configured ntfy homelab alerts and investigated database offset columns | ntfy config, homelab wiki - **Done:** Done | Set up ntfy alert routing and resolved NULL x_offset/y_offset database issue | Database schema, homelab configuration +- 18:04 (<1min) | `memory-compiler` + - **Asked:** Generate a structured weekly knowledge report for week 2026-W18. + - **Done:** Created weekly report documenting AI Cost Tracker project development, wiki documentation, and repository scaffolding progress. diff --git a/wiki/_master-index.md b/wiki/_master-index.md index 70475e2..9a3a560 100644 --- a/wiki/_master-index.md +++ b/wiki/_master-index.md @@ -19,7 +19,7 @@ This 3-hop pattern works for hundreds of articles without vector search. | Topic | Description | Articles | |-------|-------------|----------| | [[wiki/obsidian-rag/_index\|obsidian-rag/]] | Karpathy's LLM wiki method — Obsidian RAG, setup, vs true RAG | 3 | -| [[wiki/projects-overview/_index\|projects-overview/]] | All 37 Oliver Agency projects — navigable table with stack + server | 1 | +| [[wiki/projects-overview/_index\|projects-overview/]] | All 42 Oliver Agency projects — grouped by server (optical-web-1, optical-dev, baic, box-cli) | 1 | | [[wiki/tech-patterns/_index\|tech-patterns/]] | Recurring tech stacks: FastAPI, React/Vite, Next.js, Azure AD, AI, Box, One2Edit, Redis/Celery, cost-tracker | 13 | | [[wiki/architecture/_index\|architecture/]] | Cross-cutting architectural patterns: Docker Compose, multi-agent AI, GCP timeout, RAG, hotfolder, optical-dev deploy, cost-tracker | 7 | | [[wiki/client-knowledge/_index\|client-knowledge/]] | Per-client notes for Ford, H&M, L'Oréal (2+ projects each) | 3 | @@ -32,7 +32,7 @@ This 3-hop pattern works for hundreds of articles without vector search. | [[wiki/agent-sdk/_index\|agent-sdk/]] | Claude Agent SDK (formerly Claude Code SDK) — build autonomous AI agents in Python and TypeScript | 30 | | [[wiki/llm-models/_index\|llm-models/]] | LLM model catalogs — OpenAI and Claude/Anthropic models, IDs, context, pricing | 2 | | [[wiki/claude-code/_index\|claude-code/]] | Claude Code product docs — install, capabilities, surfaces, MCP, hooks, scheduling, multi-agent, plugins, skills, channels, error recovery | 12 | -| [[wiki/reports/_index\|reports/]] | Weekly and monthly knowledge base summaries | 0 | +| [[wiki/reports/_index\|reports/]] | Weekly and monthly summaries — generate: `uv run python scripts/report-generator.py --weekly` | 1 | | [[wiki/infrastructure/_index\|infrastructure/]] | Server inventory: all 10 SSH hosts — optical, optical-dev, optical-prod, baic, librechat, modocmms, box-cli, aimpress, pve | 10 | diff --git a/wiki/reports/_index.md b/wiki/reports/_index.md index 745dddf..7319f3f 100644 --- a/wiki/reports/_index.md +++ b/wiki/reports/_index.md @@ -1,29 +1,26 @@ --- -title: "Reports" -description: "Weekly and monthly knowledge base summaries" tags: [reports, index] -updated: 2026-04-24 +updated: 2026-04-27 --- -# Reports +# Reports — Index -Automatically generated weekly and monthly summaries of Claude Code sessions and wiki growth. - -## How to Generate +Weekly and monthly knowledge base summaries. Generated via `report-generator.py`. +**Generate:** ```bash -# Weekly report for current week cd ~/.claude/memory-compiler && uv run python scripts/report-generator.py --weekly - -# Monthly report for current month cd ~/.claude/memory-compiler && uv run python scripts/report-generator.py --monthly - -# Force regenerate -cd ~/.claude/memory-compiler && uv run python scripts/report-generator.py --weekly --force ``` -Reports are scheduled automatically every Monday (weekly) and first of month (monthly). +## Weekly Reports -## All Reports +| Report | Period | Key Topics | +|--------|--------|------------| +| [[wiki/reports/weekly-2026-W18\|Week 18 (2026-04-27)]] | 2026-04-21 – 2026-04-27 | ai-cost-tracker launch, video-accessibility RBAC, NotebookLM cost tracking | -*(No reports yet — first report will generate automatically)* +## Monthly Reports + +| Report | Period | Key Topics | +|--------|--------|------------| +| — | — | — | diff --git a/wiki/reports/weekly-2026-W18.md b/wiki/reports/weekly-2026-W18.md new file mode 100644 index 0000000..75b4758 --- /dev/null +++ b/wiki/reports/weekly-2026-W18.md @@ -0,0 +1,85 @@ +--- +title: "Weekly Report — 2026-W18" +type: report +period: 2026-W18 +generated: 2026-04-27 +tags: [report, weekly] +--- + +# Weekly Report — 2026-W18 + +## Summary +Week 18 was dominated by building and integrating the **AI Cost Tracker** — a new shared Oliver platform service for centralised AI spend tracking. The tracker was scaffolded, deployed to optical-dev, and successfully connected to two consumer projects (video-accessibility, sandbox-notebookllamalm-nextjs) in a single day. Significant parallel work was done on video-accessibility: a Pydantic v2 serialisation bug was fixed, and a multi-phase RBAC/invitation system with SaaS-style tenant isolation was architected and partially implemented. + +## Projects Worked On + +| Project | Days Active | Key Work Done | +|---|---|---| +| **ai-cost-tracker** | 1 | Built FastAPI+MongoDB+Celery service; fixed rollup field names; switched analytics from nightly rollups to live `usage_events`; redesigned dashboard (dynamic dropdowns, sub-cent cost display, `source_app` pivot) | +| **video-accessibility** | 1 | Fixed Pydantic v2 `Field(alias="_id")` bug causing `undefined` client IDs; added `PATCH /jobs/{id}` endpoint + Cost Tracker Project ID UI field; scaffolded Client→Team→Project RBAC + PM role; MS SSO provisioning fix; multi-phase invitation system with Redis membership cache; Mailgun emailer | +| **sandbox-notebookllamalm-nextjs** | 1 | Integrated cost tracker (preflight + record); fixed ContextVar propagation through `asyncio.wait_for`; verified smoke test end-to-end | + +## Key Decisions Made + +- **`source_app` as project identifier** — `project_id` is never populated in `usage_events`; all analytics now pivot on `source_app` string +- **Drop Python `ContextVar` for user context** — unreliable across `asyncio.wait_for` task boundaries; always pass `user_external_id` as explicit function parameter +- **`preflight()` is fail-open** (`fail_open=True`) — cost-tracker outage must never block AI pipelines; returns `allow=true` when service unreachable +- **`record()` never raises** — SQLite outbox with 30s flusher + 10 retries + `dead` status; background error, not pipeline error +- **Alias mapping in pricing_engine** — keep `"google"` as provider string in client code; `pricing_engine.py` maps it to `vertex_ai-language-models` internally +- **Analytics reads live `usage_events`** — switched from nightly `usage_rollups` (invisible until 01:00 UTC) to real-time collection reads +- **`dev` branch for SaaS refactor** — video-accessibility tenant architecture (Org→members→teams→projects) goes to `dev`, not `main` +- **Redis membership cache** for RBAC (`cached_memberships.py`) — cache bumped on every membership write, avoiding per-request DB queries +- **Mailgun over SendGrid** — reuses existing `httpx` dependency; identical method signatures + +## Technical Insights + +- **`asyncio.wait_for` does NOT propagate Python ContextVars** — any context set before `wait_for` is invisible inside the nested coroutine; always pass values explicitly +- **`docker compose restart` does NOT re-read `env_file`** — must use `docker compose up -d --force-recreate` after `.env` changes +- **`.env` files on optical can be silently wiped** on `git pull` or Docker rebuild; verify after every deploy: `grep COST_TRACKER .env` + `docker inspect ` +- **Pydantic v2 `Field(alias="_id")` gotcha** — serialises the JSON key as `_id`, not `id`; frontend receives `undefined` on `.id` access; fix with explicit `_from_doc` helpers that rename `_id → id` and remove the alias from the response model +- **LiteLLM pricing key for Gemini is `vertex_ai-language-models`**, not `google` — always verify exact provider key in `model_prices` collection after a LiteLLM sync +- **`toFixed(2)` silently zeroes sub-cent costs** — use dynamic precision for cost display (e.g. `toPrecision(4)` or threshold-based formatting) +- **ElevenLabs returns no usage metadata** — character count must be captured via `len(text)` before the API call +- **Google TTS bills stripped chars for SSML** — use `len(synthesis_input.text)`, not `len(ssml)`, to avoid over-counting +- **When making FastAPI helpers async**, audit every call site — there were 10 spread across multiple route files in video-accessibility +- **`docker compose build --no-cache`** required when source code is baked into the image (not volume-mounted) +- **`git pull --rebase` fails with unstaged changes** — `git stash` first + +## Knowledge Base Growth + +10 new wiki articles created: + +| Article | Location | Topic | +|---|---|---| +| Homelab Services Map | `homelab/homelab-services-map.md` | Full network/container reference | +| AI Cost Tracker — Pricing Sources | `tech-patterns/cost-tracker-pricing-sources.md` | 3-layer pricing pipeline (override → yaml → litellm) | +| AI Cost Tracker — Billing Units per Provider | `tech-patterns/cost-tracker-providers.md` | Gemini/ElevenLabs/GCS TTS extraction patterns | +| AI Cost Tracker — Integration Guide | `tech-patterns/cost-tracker-integration.md` | Step-by-step project onboarding (v3, ContextVar pitfall, docker inspect) | +| AI Cost Tracker — Project Card | `projects-overview/ai-cost-tracker.md` | Repo, stack, status, consumers | +| AI Cost Tracker — Architecture | `architecture/ai-cost-tracker.md` | Service design, live URLs, data model | +| Sync HTTP + SQLite Outbox Pattern | `concepts/sync-with-outbox.md` | Resilient record() without blocking pipelines | +| LiteLLM as Pricing Source | `concepts/litellm-pricing-source.md` | Why LiteLLM, sync schedule, provider key format | +| Preflight + Record Pattern | `concepts/preflight-record-pattern.md` | Universal 3-step pattern for all AI calls | +| Lazy User Mirror | `concepts/lazy-user-mirror.md` | On-first-use user provisioning in cost-tracker | + +## Action Items Carried Forward + +| Item | Project | Priority | +|---|---|---| +| Push `ai-cost-tracker` to Bitbucket: `cd /tmp/ai-cost-tracker && git push origin main` | ai-cost-tracker | High | +| Monitor dashboard — confirm video-accessibility events appearing after today's fixes | ai-cost-tracker / video-accessibility | High | +| Run migration on optical-web-1: `docker compose exec -T api python migrate.py up` (adds `project_manager` enum + `pm_client_ids`) | video-accessibility | High | +| Complete + commit Phase 4 frontend reorg (new routes, Settings IA, org switcher in Sidebar) | video-accessibility | Medium | +| Commit `frontend/src/hooks/useAccessibleVideoEdit.ts` unstaged changes | video-accessibility | Medium | +| Add `**/__pycache__/` to `.gitignore` | video-accessibility | Low | +| Add outbox depth metric to monitoring alerts for `dead` status rows | ai-cost-tracker | Low | +| Phase 2: track Whisper `audio_duration_seconds` and approximate from Cloud Run billing | ai-cost-tracker | Backlog | +| Check if video-accessibility MongoDB has historical AI call logs for back-fill | ai-cost-tracker | Backlog | +| Add deploy checklist to wiki: verify .env → docker inspect → smoke test | homelab/ops | Low | +| Grafana: change admin password from default `admin:admin` | homelab | Security | + +## Stats +- Sessions: 9 +- Daily logs: 1 day (2026-04-27) +- Wiki articles updated/created: 10 +- Flush errors: 6 (FLUSH_ERROR throughout the day — investigate memory compiler reliability) \ No newline at end of file