From 26eeb7deac21bd582425d414dc28af195ca77bf8 Mon Sep 17 00:00:00 2001 From: Vadym Samoilenko Date: Mon, 18 May 2026 18:46:52 +0100 Subject: [PATCH] vault backup: 2026-05-18 18:46:52 --- .../baic_dashboard/DEVELOPER_MANUAL.md | 141 ++++++++++++++ 01 Projects/baic_dashboard/USER_MANUAL.md | 82 +++++++++ 01 Projects/homepage/DEVELOPER_MANUAL.md | 11 ++ 01 Projects/homepage/USER_MANUAL.md | 16 +- .../lusa-back-planner/DEVELOPER_MANUAL.md | 172 ++++++++++++++++++ 01 Projects/lusa-back-planner/USER_MANUAL.md | 81 +++++++++ .../obsidian-vault/DEVELOPER_MANUAL.md | 111 +++++++++++ 01 Projects/obsidian-vault/USER_MANUAL.md | 105 +++++++++++ .../DEVELOPER_MANUAL.md | 146 +++++++++++++++ .../USER_MANUAL.md | 79 ++++++++ .../social-reporting-tool/DEVELOPER_MANUAL.md | 163 +++++++++++++++++ .../social-reporting-tool/USER_MANUAL.md | 106 +++++++++++ 99 Daily/2026-05-18.md | 2 + 13 files changed, 1214 insertions(+), 1 deletion(-) create mode 100644 01 Projects/baic_dashboard/DEVELOPER_MANUAL.md create mode 100644 01 Projects/baic_dashboard/USER_MANUAL.md create mode 100644 01 Projects/lusa-back-planner/DEVELOPER_MANUAL.md create mode 100644 01 Projects/lusa-back-planner/USER_MANUAL.md create mode 100644 01 Projects/obsidian-vault/DEVELOPER_MANUAL.md create mode 100644 01 Projects/obsidian-vault/USER_MANUAL.md create mode 100644 01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md create mode 100644 01 Projects/sandbox-notebookllamalm-nextjs/USER_MANUAL.md create mode 100644 01 Projects/social-reporting-tool/DEVELOPER_MANUAL.md create mode 100644 01 Projects/social-reporting-tool/USER_MANUAL.md diff --git a/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md b/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..a0f68fe --- /dev/null +++ b/01 Projects/baic_dashboard/DEVELOPER_MANUAL.md @@ -0,0 +1,141 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- client/oliver +- status/active +- tech/azure-ad +- tech/flask +- tech/python +- tech/react +- tech/typescript +- type/sop +type: sop +--- + +# Oliver Agency Reporting Module - Developer Manual + +## Architecture Overview +The application is a **Full-Stack Web App** with: +- **Frontend**: React 18 + Vite, styled with CSS, using Azure MSAL for authentication. +- **Backend**: Python Flask API running on a separate port (default 5001). +- **Data Source**: External Make.com Webhook providing JSON data. +- **Deployment**: Configurable base path and domain via environment variables. + +The frontend communicates with the backend, which likely fetches or proxies data from the Make.com webhook. Authentication is handled via Azure AD (MSAL). + +## Tech Stack +- **Frontend**: React, Vite, `@azure/msal-react`, `@azure/msal-browser`, Recharts (for graphs), date-fns. +- **Backend**: Python 3, Flask. +- **Authentication**: Azure Active Directory (OAuth2/OIDC). +- **Data Processing**: Client-side aggregation using `userAggregation.js`. + +## Local Setup + +### Prerequisites +- Node.js (v18+) +- Python 3.8+ +- Make.com Account (for webhook data) +- Azure AD App Registration + +### 1. Clone and Configure +```bash +git clone +cd Oliver-Agency-Reporting-Module +cp .env.example .env +``` + +Edit `.env` with your: +- `DOMAIN` +- `BASE_PATH` +- `AZURE_TENANT_ID` +- `AZURE_CLIENT_ID` +- `MAKE_WEBHOOK_URL` + +### 2. Backend Setup +```bash +cd backend +python3 -m venv venv +source venv/bin/activate +pip install -r requirements.txt +# Load env vars and start server +set -a && source ../.env && set +a +python app.py +``` +Backend runs on `http://localhost:5001`. + +### 3. Frontend Setup +```bash +cd frontend +npm install +npm run dev +``` +Frontend runs on `http://localhost:5173`. + +### 4. Quick Start (Alternative) +Run `./run.sh` from the root directory to start both servers simultaneously. + +## Environment Variables +| Variable | Description | Required | +|----------|-------------|----------| +| `DOMAIN` | Deployment domain (e.g., `example.com`) | Yes | +| `BASE_PATH` | App base path (e.g., `/dashboard/`) | Yes | +| `AZURE_TENANT_ID` | Azure AD Tenant ID | Yes | +| `AZURE_CLIENT_ID` | Azure AD Client ID | Yes | +| `BACKEND_PORT` | Flask server port | No (default: 5001) | +| `MAKE_WEBHOOK_URL` | Source data webhook URL | Yes | +| `VITE_*` | Frontend build-time variables mirroring above | Yes | +| `VITE_REDIRECT_URI` | Azure AD redirect URI (Production) | Yes | +| `VITE_REDIRECT_URI_DEV` | Azure AD redirect URI (Dev) | Yes | + +## Key Services & Entry Points + +### Frontend Entry: `frontend/src/main.jsx` +- Initializes MSAL instance. +- Handles redirect promise for auth flow. +- Renders `` wrapping ``. + +### Auth Config: `frontend/src/authConfig.ts` +- Defines `msalConfig` (clientId, tenantId, scopes). +- Defines `loginRequest` scopes (e.g., `User.Read`). + +### Dashboard Logic: `frontend/src/components/Dashboard.jsx` +- Manages state for filters (organization, user, assistant, dateRange). +- Fetches data (implied via props or API calls not fully shown in snippet, but likely from backend). +- Filters `conversations` and `messages` arrays. + +### Data Aggregation: `frontend/src/utils/userAggregation.js` +- `aggregateMessagesByUser(filteredMessages, processedUserData)`: Aggregates message counts per user. + +### Authentication Components +- `LoginComponent.jsx`: Handles popup login via `instance.loginPopup()`. +- `AuthenticatedApp.jsx`: Renders main dashboard after auth. + +## API Reference +*Note: Specific API endpoints are not fully visible in the provided snippets, but the backend (`app.py`) likely exposes: +- `GET /api/data`: Fetches conversation/message data from Make.com webhook. +- `POST /api/...`: If any write operations exist. + +The frontend likely calls an internal proxy endpoint (e.g., `/api/data`) which forwards to `MAKE_WEBHOOK_URL`.* + +## Deployment +1. **Build Frontend**: `npm run build`. +2. **Configure Web Server**: Ensure your web server (Nginx, Apache, or static host) serves the `dist` folder. +3. **Base Path**: Ensure the web server routes all requests to `index.html` for SPA routing (e.g., `/dashboard/...`). +4. **Azure AD**: Ensure `VITE_REDIRECT_URI` matches the Azure Portal redirect URI exactly. +5. **Backend**: Deploy Flask app (e.g., via Gunicorn) on the specified `BACKEND_PORT`. + +## Known Gotchas +1. **Azure AD Redirect URIs**: Must match exactly. Dev uses `http://localhost:5173`, Prod uses `https://yourdomain.com/dashboard/`. +2. **Date Formatting**: Use `date-fns` for consistent ISO formatting. Avoid native `Date` parsing for complex ranges. +3. **CORS**: Backend must allow CORS from the frontend domain. +4. **Webhook Latency**: Make.com webhooks can have delay. Cache data appropriately if performance is an issue. +5. **CSV Export**: Large datasets may cause browser memory issues during client-side CSV generation. Consider server-side export for large volumes. +6. **Path Trailing Slashes**: Ensure `BASE_PATH` ends with `/` as per config requirements. + +## Related +- [[01 Projects/ac-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/baic_dashboard/USER_MANUAL.md b/01 Projects/baic_dashboard/USER_MANUAL.md new file mode 100644 index 0000000..83772eb --- /dev/null +++ b/01 Projects/baic_dashboard/USER_MANUAL.md @@ -0,0 +1,82 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: User_Manual +status: active +tags: +- client/oliver +- domain/analytics +- tech/azure-ad +- tech/fastapi +- tech/postgresql +- tech/react +- tech/typescript +- type/dashboard +- type/sop +type: dashboard +--- + +# Oliver Agency Reporting Module - User Manual + +## What This Tool Does +The Oliver Agency Reporting Module is a web-based analytics dashboard designed to visualize and export usage data for conversations and messages. It provides insights into message volume by user, allowing agencies to monitor engagement, track performance, and generate reports for stakeholders. + +## Who Uses It +- **Marketing/Analytics Teams**: To track conversation volumes and user engagement metrics. +- **Agency Managers**: To monitor assistant performance and brand interaction trends. +- **Support Leads**: To analyze message distribution and user activity over specific timeframes. + +## How to Access +1. Navigate to the deployed URL (e.g., `https://yourdomain.com/dashboard/`). +2. You will see a "Sign In with Microsoft" screen. +3. Click the **Sign In** button. +4. Authenticate using your organization's Microsoft Azure Active Directory credentials. +5. Once authenticated, the Dashboard will load automatically. + +## Main Workflows + +### 1. Filtering Data +The dashboard allows you to narrow down data by: +- **Date Range**: Select "All time", "Last 30 days", "Last week", "Today", or a custom range. +- **Organization/Brand**: Filter by specific organizations or brands if the dataset contains multiple. +- **User/Assistant**: Filter by specific users or AI assistants. + +**Steps:** +1. Locate the **Filter Panel** at the top of the dashboard. +2. Adjust the **Date Picker** to your desired timeframe. +3. Use dropdown menus (if available in the rendered UI) to select specific Organizations, Users, or Assistants. +4. The charts and tables below will update in real-time to reflect the selected filters. + +### 2. Viewing Analytics +The dashboard displays two main views: +- **Volume Graph**: A bar chart showing message counts by user. +- **User Report**: A table listing specific users and their message counts. + +**Steps:** +1. Ensure the **Volume** tab is active (default). +2. Review the bar chart for high-level trends. +3. Scroll down to the **User Report** table for detailed numbers. +4. Use the **Export User Report** button to download the current view as a CSV file. + +### 3. Exporting Reports +To share data with stakeholders: +1. Apply your desired filters (Date, Organization, etc.). +2. Click the **Export User Report** button. +3. A CSV file named `user_message_counts_{organization}_{date_range}.csv` will be downloaded. +4. Open the file in Excel, Google Sheets, or any spreadsheet application. + +## FAQ + +**Q: Why is my data not showing up?** +A: Ensure your selected date range matches the data available in the source (Make.com webhook). If no data exists for the selected period, the chart will be empty. + +**Q: Can I filter by specific messages?** +A: Currently, the dashboard aggregates data by user and time. You cannot filter by specific message content or ID directly in the UI. + +**Q: Who do I contact if the dashboard fails to load?** +A: Contact your IT administrator to check Azure AD app registration settings or the backend server status. Ensure your browser is up to date. + +**Q: Is my data secure?** +A: Yes, the application uses Azure Active Directory for authentication. Data is retrieved via secure webhook endpoints and processed client-side. Do not share your login credentials. \ No newline at end of file diff --git a/01 Projects/homepage/DEVELOPER_MANUAL.md b/01 Projects/homepage/DEVELOPER_MANUAL.md index 7b900c6..a05e5aa 100644 --- a/01 Projects/homepage/DEVELOPER_MANUAL.md +++ b/01 Projects/homepage/DEVELOPER_MANUAL.md @@ -1,7 +1,18 @@ --- auto_generated: true +created: 2026-05-18 manual_updated_at: 2026-05-18 modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- domain/devops +- tech/docker +- tech/nextjs +- tech/react +- tech/tailwind +- tech/typescript +type: reference --- # Homepage Dashboard Developer Manual diff --git a/01 Projects/homepage/USER_MANUAL.md b/01 Projects/homepage/USER_MANUAL.md index 4ce952d..c486f09 100644 --- a/01 Projects/homepage/USER_MANUAL.md +++ b/01 Projects/homepage/USER_MANUAL.md @@ -1,7 +1,18 @@ --- auto_generated: true +created: 2026-05-18 manual_updated_at: 2026-05-18 modified: 2026-05-18 +name: User_Manual +status: active +tags: +- domain/homelab +- domain/security +- tech/docker +- tech/nextjs +- type/dashboard +- type/sop +type: sop --- # Homepage Dashboard User Manual @@ -90,4 +101,7 @@ A: Homepage updates itself when the underlying Docker image is updated. Check th A: Yes, it is a Next.js application. However, the provided setup relies on Docker for the proxy and environment configuration. **Q: How is security handled?** -A: API requests are proxied through the backend, preventing direct exposure of credentials. Always use HTTPS in production via a reverse proxy. \ No newline at end of file +A: API requests are proxied through the backend, preventing direct exposure of credentials. Always use HTTPS in production via a reverse proxy. + +## Related +- [[01 Projects/homepage/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/lusa-back-planner/DEVELOPER_MANUAL.md b/01 Projects/lusa-back-planner/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..84faf00 --- /dev/null +++ b/01 Projects/lusa-back-planner/DEVELOPER_MANUAL.md @@ -0,0 +1,172 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- domain/devops +- tech/azure-ad +- tech/react +- tech/typescript +type: reference +--- + +# LUSA Back Planner - Developer Manual + +## 1. Architecture Overview +This is a client-side React SPA built with Vite. It does **not** have a backend API. All logic (SLA calculation, holiday mapping, PDF generation) runs in the browser. + +* **Entry Point:** `src/main.tsx` (implied, via Vite) +* **State Management:** React Context (for Toasts) and Local State (for form inputs). +* **Data Layer:** Static JSON/TS files (`src/lib/sla-data.ts`). + +## 2. Tech Stack +* **Framework:** React 18+ with TypeScript +* **Build Tool:** Vite +* **Styling:** Tailwind CSS + shadcn/ui components +* **PDF Parsing:** `pdfjs-dist` (for reading uploaded briefs) +* **PDF Export:** `jsPDF` + `html2canvas` +* **Testing:** Vitest + React Testing Library +* **Linting:** ESLint with `@typescript-eslint` and `react-refresh` + +## 3. Local Setup + +### Prerequisites +* Node.js 18+ +* npm or pnpm + +### Installation +```bash +npm install +``` + +### Running Development Server +```bash +npm run dev +``` +* Server runs at `http://localhost:3000` (configured in `vite.config.ts`). +* Note: The `vite.config.ts` sets `base` to `/lusa-Back-Planner/` in production mode, but `/` in dev. Ensure your proxy/Apache config aligns if testing production builds locally. + +### Running Tests +```bash +npm run test +``` +* Tests are located in `src/**/*.test.ts`. +* Environment is `jsdom`. +* Setup file: `src/test/setup.ts` (mocks `matchMedia` and includes `@testing-library/jest-dom`). + +## 4. Environment Variables +All environment variables must use the `VITE_` prefix to be exposed to the browser. + +Create a `.env` file based on `.env.example`: + +| Variable | Description | Example | +| :--- | :--- | :--- | +| `VITE_AZURE_TENANT_ID` | Azure AD Tenant for Auth | `e519c2e6-bc6d-4fdf-8d9c-923c2f002385` | +| `VITE_AZURE_CLIENT_ID` | Azure AD Application Client ID | `9079054c-9620-4757-a256-23413042f1ef` | +| `VITE_AZURE_REDIRECT_URI` | OAuth Callback URL | `https://ai-sandbox.oliver.solutions/lusa-Back-Planner` | + +**Note:** For local dev, you may need different Azure credentials (`15c0c4e2-bac0-4564-a3a6-c2717f00a6d9`) and a redirect URI pointing to `http://localhost:8888/lusa-Back-Planner`. + +## 5. Key Services & Entry Points + +### `src/lib/sla-data.ts` +Contains the core business logic data: +* `SlaEntry`: Interface defining the SLA structure. +* `slaData`: Array of objects mapping Complexity + Category + AssetType to Workday ranges. +* `getContentCategoriesForComplexity()`: Helper to filter categories dynamically. + +### `src/lib/workdays.ts` +Contains holiday and date calculation logic: +* `nthWeekday()`: Helper for finding specific weekdays. +* `lastMonday()`: Helper for week boundaries. +* `observed()`: Handles holiday observation rules. +* *Note: The file was truncated in the codebase. Ensure full holiday lists are defined.* + +### `src/hooks/use-toast.ts` +Custom toast notification hook based on Radix UI patterns: +* Manages a stack of toasts (`TOAST_LIMIT = 1`). +* Uses a reducer pattern (`addToRemoveQueue`, `reducer`, `dispatch`). + +### `src/components/ui/` +Contains shadcn/ui components. Import via `@/components/ui/...` (mapped via `vite.config.ts` alias `@`). + +## 6. API Reference (Internal) + +### `SlaEntry` Interface +```typescript +interface SlaEntry { + complexity: string; + contentCategory: string; + assetType: string; + workdaysMin: number; + workdaysMax: number; + roundsOfAmends: number; + steps: { + copyCreation: string; + translation: string; + claimValidation: string; + marketValidation: string; + visual1stDraft: string; + productionRevisions: string; + uploadToOpera: string; + }; +} +``` + +## 7. Deployment + +### Build +```bash +npm run build +``` +Produces static files in `dist/`. + +### Apache Deployment (Ubuntu) +1. Copy `dist/` to `/var/www/backplanner`. +2. Configure VirtualHost (`/etc/apache2/sites-available/backplanner.conf`): + ```apache + + ServerName your-domain.com + DocumentRoot /var/www/backplanner + + Options -Indexes + AllowOverride None + Require all granted + FallbackResource /index.html + + + Header set Cache-Control "public, max-age=31536000, immutable" + + + ``` +3. Enable modules and site: + ```bash + sudo a2enmod headers + sudo a2ensite backplanner.conf + sudo systemctl reload apache2 + ``` +4. (Optional) HTTPS via Certbot: + ```bash + sudo certbot --apache -d your-domain.com + ``` + +### Subdirectory Deployment +If deploying under `your-server.com/backplanner/`: +1. Update `vite.config.ts`: + ```typescript + export default defineConfig({ + base: "/backplanner/", + // ... rest of config + }); + ``` +2. Use Apache Alias instead of VirtualHost. + +## 8. Known Gotchas +1. **Azure Auth:** Ensure `VITE_AZURE_REDIRECT_URI` matches exactly the configured redirect in Azure Portal, including trailing slashes. +2. **Date Calculations:** The `workdays.ts` logic assumes European/local holiday calendars. Hardcode holidays for specific regions if international support is needed. +3. **PDF Export:** Client-side PDF generation can be memory-intensive for large DOM trees. Test with complex timelines. +4. **Vite Base Path:** Changing `base` in `vite.config.ts` requires a full rebuild (`npm run build`) before deployment. It does not update dynamically in dev without restarting. +5. **Shadcn UI:** Ensure `tailwind.config.ts` extends are correctly mapped to CSS variables in `src/index.css` (not shown but implied by Tailwind config). \ No newline at end of file diff --git a/01 Projects/lusa-back-planner/USER_MANUAL.md b/01 Projects/lusa-back-planner/USER_MANUAL.md new file mode 100644 index 0000000..0dee8ab --- /dev/null +++ b/01 Projects/lusa-back-planner/USER_MANUAL.md @@ -0,0 +1,81 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: User_Manual +status: active +tags: +- client/lusa +- tech/azure-ad +- tech/docker +- tech/react +- tech/typescript +- type/sop +type: sop +--- + +# LUSA Back Planner - User Manual + +## 1. What This Tool Does +LUSA Back Planner is a production timeline generator designed for teams managing digital content assets (PDPs, Landing Pages, Creative, etc.). It calculates estimated production timelines based on: + +* **Project Complexity:** From "Creation" to "Launch Complex". +* **Content Category:** Such as PDP, Landing Page, CRM, or Launch assets. +* **Asset Types:** Specifying if the deliverable includes text only, static visuals, or BTF (Behind The Front). +* **SLA Data:** Utilizing predefined Service Level Agreement (SLA) steps like Copy Creation, Translation, Market Validation, and Visual Drafts. + +The tool instantly generates a Gantt-style timeline, accounts for holidays and weekends (workdays), and supports PDF export for stakeholder briefs. + +## 2. Who Uses It +* **Production Managers:** To plan resources and set deadlines. +* **Content Creators:** To understand their specific task timelines and handoff dates. +* **Stakeholders:** To view exported PDF briefs for approval. + +## 3. How to Access +The application is a Single Page Application (SPA) deployed on a web server. +* **Production URL:** Visit the deployed domain (e.g., `https://your-domain.com/lusa-Back-Planner/`). +* **Authentication:** Ensure your browser is logged into the corporate identity provider (Azure AD) if single sign-on is configured. The app uses the `VITE_AZURE_TENANT_ID` and `VITE_AZURE_CLIENT_ID` for authentication flows. + +## 4. Main Workflows + +### Workflow A: Generate a Timeline +1. **Set Deadline:** Input your target launch date. +2. **Select Complexity:** Choose the complexity level (e.g., "Medium Adaptation"). +3. **Select Content Category:** Choose the type of content (e.g., "PDP"). +4. **Select Asset Type:** Specify what needs to be created (e.g., "PDP Texts + Static visuals"). +5. **Click Generate:** The system calculates the start date and intermediate milestones based on the `sla-data.ts` matrix. +6. **Review Timeline:** View the visual timeline in the browser, highlighting: + * Copy Creation + * Translation + * Claim/Market Validation + * Visual Drafts & Revisions + * Upload to Opera + +### Workflow B: Manage Change Requests +1. If the deadline or complexity changes, use the **Change Request** feature. +2. The timeline will recalculate. +3. Use the **Toast Notifications** (top-right corner) to confirm updates or errors. + +### Workflow C: Export Brief to PDF +1. Click the **Export PDF** button in the timeline view. +2. The tool uses `jsPDF` and `html2canvas` to render the current view. +3. A PDF file is downloaded to your local machine, suitable for email or archival. + +## 5. FAQ + +**Q: Why is my timeline longer than expected?** +A: The calculator uses "Workdays," excluding weekends and public holidays. If a holiday falls within the calculated range, the duration in calendar days may appear longer. + +**Q: Can I customize the SLA steps?** +A: No. The steps (Copy, Translation, Validation, etc.) are hardcoded in `src/lib/sla-data.ts` based on company policy. Contact the dev team if new steps are required. + +**Q: Does the app save my data?** +A: No. This is a stateless client-side tool. Refreshing the page will reset the form. Export your timeline to PDF if you need a record. + +**Q: What browsers are supported?** +A: Modern browsers (Chrome, Firefox, Safari, Edge) with JavaScript enabled. IE11 is not supported. + +--- + +*For technical support or deployment issues, please refer to the Developer Manual.* \ No newline at end of file diff --git a/01 Projects/obsidian-vault/DEVELOPER_MANUAL.md b/01 Projects/obsidian-vault/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..b9f229c --- /dev/null +++ b/01 Projects/obsidian-vault/DEVELOPER_MANUAL.md @@ -0,0 +1,111 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- domain/devops +- status/active +- tech/typescript +- type/reference +- type/sop +type: sop +--- + +# Obsidian Second Brain: Developer Manual + +## Architecture Overview +This project is a **static content repository** representing an Obsidian Vault. It is not a traditional software application with a runtime server or database. Instead, it is a collection of Markdown files and a `.obsidian` configuration directory that defines the behavior of the Obsidian app. + +### Structure +- **Root Directory:** Contains the main Markdown notes (`*.md`). +- **`.obsidian/`:** Configuration directory for Obsidian settings. + - **`.obsidian/plugins/`:** Installed community plugins. + - Each plugin folder contains `main.js` (bundled code), `manifest.json`, and potentially `styles.css`. +- **`assets/` (implied):** Directory for local images and media files managed by `obsidian-local-images-plus`. + +## Tech Stack +- **Core Application:** Obsidian (Electron-based Markdown editor). +- **Frontend/Logic:** JavaScript/TypeScript (bundled into the plugin `.js` files). +- **Data Format:** Markdown (`.md`) with YAML frontmatter for metadata. +- **Bundlers Used in Plugins:** + - **Esbuild:** Used for `mermaid-tools` and `obsidian-tasks-plugin`. + - **Rollup:** Used for `obsidian-local-images-plus`. + - **TypeScript Compiler:** Used for `nldates-obsidian` and `calendar` (indicated by `require$$0` patterns and `__awaiter` helpers). + +## Local Setup +1. **Clone/Download:** Retrieve the vault folder. +2. **Install Obsidian:** Download from [obsidian.md](https://obsidian.md). +3. **Install Plugins:** + - Open Obsidian. + - Go to Settings > Community Plugins. + - Turn off "Secure Mode" if necessary. + - Install the following from the community store: + - `obsidian-tasks-plugin` + - `calendar` + - `nldates-obsidian` + - `mermaid-tools` + - `obsidian-local-images-plus` +4. **Sync:** If collaborating, configure a git client or file sync service (Git, Dropbox, iCloud Drive). + +## Environment Variables +There are **no environment variables** in this project. Configuration is stored in: +- **`vault.json`:** Obsidian core settings. +- **Plugin Settings:** Stored in `.obsidian/plugins//data.json` within the Obsidian app context. + +## Key Services & Entry Points + +### 1. Plugin Initialization +Each plugin hooks into Obsidian's lifecycle: +- **`main.js`:** The entry point for each plugin. It exports a class extending `obsidian.Plugin`. + - Example (`mermaid-tools/main.js`): + ```javascript + module.exports = __toCommonJS(main_exports); + // ... exports default: () => MermaidPlugin + ``` + +### 2. Date & Time Handling (`nldates-obsidian`) +- **Purpose:** Parses and generates date strings for periodic notes. +- **Key Constants:** + - `DEFAULT_DAILY_NOTE_FORMAT = "YYYY-MM-DD"` + - `DEFAULT_WEEKLY_NOTE_FORMAT = "gggg-[W]ww"` +- **Logic:** Checks for `periodic-notes` plugin settings. If present, it defers to those settings; otherwise, it uses defaults. + +### 3. Task Parsing (`obsidian-tasks-plugin`) +- **Purpose:** Parses markdown task lists (`- [ ]`) into structured data objects. +- **Library:** Uses `rrule.js` for recurrence rule parsing (MIT license). +- **Processing:** Scans note content for task patterns and updates the task cache. + +### 4. Visualization (`mermaid-tools`) +- **Purpose:** Renders Mermaid diagrams. +- **Entry:** Exports `MermaidPlugin` class. +- **Features:** Provides sample diagrams for testing (ER, Class, Flowchart, Gantt, etc.). + +### 5. Calendar Integration (`calendar`) +- **Purpose:** Provides a visual calendar view and note creation triggers. +- **Key Functions:** + - `getDailyNoteSettings()`: Retrieves format/folder/template from `periodic-notes` or `internalPlugins`. + - `getWeeklyNoteSettings()`: Similar retrieval for weekly notes. +- **Constants:** `VIEW_TYPE_CALENDAR = "calendar"`. + +## API Reference (Plugin Hooks) +Developers interacting with this vault programmatically would use the Obsidian API: +- **`app.plugins.getPlugin("plugin-name")`**: Access plugin instances. +- **`app.metadataCache`**: Retrieve note metadata. +- **`app.workspace.getActiveFile()`**: Access current note. +- **`editor`**: Manipulate note content programmatically. + +## Deployment +This "deployment" is simply **version control**: +1. **Git:** Initialize git in the vault root. Add `.gitignore` for temporary files if needed. +2. **Push:** Push to GitHub/GitLab. +3. **Sync:** Clone onto other devices. + +## Known Gotchas +1. **Plugin Conflicts:** The `calendar` and `nldates` plugins both check for `periodic-notes`. If `periodic-notes` is disabled, `calendar` will fall back to legacy `internalPlugins` settings. Ensure `periodic-notes` is enabled for best behavior. +2. **Asset Paths:** Local images require absolute paths relative to the vault root or `assets/` folder. Moving files manually may break links. Use Obsidian's built-in drag-and-drop. +3. **Bundled Code:** The `main.js` files are minified/bundled. Do not edit them directly. Changes must be made in source repositories (GitHub) and rebuilt. +4. **Markdown Compatibility:** Ensure your markdown parser supports the specific Mermaid syntax variants (e.g., `flowchart LR` vs `graph TD`). +5. **Task Plugin Dependencies:** The `obsidian-tasks-plugin` is heavy and may impact performance in very large vaults (>10k files). Monitor memory usage. \ No newline at end of file diff --git a/01 Projects/obsidian-vault/USER_MANUAL.md b/01 Projects/obsidian-vault/USER_MANUAL.md new file mode 100644 index 0000000..1eacaec --- /dev/null +++ b/01 Projects/obsidian-vault/USER_MANUAL.md @@ -0,0 +1,105 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: User_Manual +status: active +tags: +- domain/ai +- tech/obsidian +- type/resource +- type/sop +type: resource +--- + +# Obsidian Second Brain: User Manual + +## What This Tool Does +This project is not a single software application but a configured **Obsidian Vault** designed to function as a "Second Brain." It leverages the Obsidian note-taking platform enhanced by a suite of community plugins to manage knowledge, tasks, schedules, and visual data. + +### Core Capabilities +- **Knowledge Management:** Centralize notes, documents, and references. +- **Task Management:** Track to-dos, recurring tasks, and deadlines using the Obsidian Tasks plugin. +- **Temporal Organization:** Automatically organize notes by day, week, month, and year using Calendar and N-L-Dates plugins. +- **Visualization:** Create diagrams (flowcharts, mind maps, ER diagrams) and Gantt charts using Mermaid Tools. +- **Asset Management:** Handle local images and media efficiently. + +## Who Uses It +- **Personal Knowledge Managers (PKM):** Individuals who want to link ideas and create a personal wiki. +- **Project Managers:** Users who need to track tasks and visualize project timelines. +- **Researchers & Writers:** People who need to organize research notes and structure complex information. +- **Students:** Users managing coursework, schedules, and study materials. + +## How to Access +1. **Prerequisites:** + - Install [Obsidian](https://obsidian.md) on your desktop or mobile device. + - Ensure you have the following plugins installed via the Community Plugins store: + - `obsidian-tasks-plugin` + - `calendar` + - `nldates-obsidian` + - `mermaid-tools` + - `obsidian-local-images-plus` + +2. **Opening the Vault:** + - Open Obsidian. + - Click "Open folder as vault." + - Select the root directory of this project folder. + +## Main Workflows + +### 1. Scheduling Notes (Calendar & Periodic Notes) +This setup integrates with Obsidian's periodic notes to auto-create daily, weekly, and monthly notes. + +- **Daily Notes:** + - Press `Ctrl+P` (or `Cmd+P` on Mac) to open the command palette. + - Select `Calendar: Open Calendar View`. + - Click on a date in the calendar grid. + - If the daily note doesn't exist, it will be created automatically using the format `YYYY-MM-DD`. + +- **Weekly/Monthly Notes:** + - In the Calendar view, long-press (or right-click) on a date to see options for creating weekly/monthly notes. + - The format defaults to `gggg-[W]ww` for weeks and `YYYY-MM` for months. + +### 2. Managing Tasks +- **Creating Tasks:** + - Type any line starting with `- [ ]` in a note. + - Add metadata if needed: `- [ ] Task text #due/2023-10-27 #priority/A` +- **Viewing Tasks:** + - Use the Tasks plugin query view to filter tasks by date, tag, or project. + - Example query: `#task is:not done due:after today` + +### 3. Creating Diagrams (Mermaid) +- **Flowcharts & Diagrams:** + - Insert a code block with `mermaid` as the language: + ```mermaid + graph TD + A[Start] --> B{Is it working?} + B -- Yes --> C[Great!] + B -- No --> D[Debug] + ``` +- **Supported Types:** + - Flowcharts, Class Diagrams, ER Diagrams, Gantt Charts, GitGraphs, Pie Charts, Sequence Diagrams, State Diagrams, User Journeys, Mindmaps. + +### 4. Image Handling +- **Local Images:** + - Drag and drop images directly into a note. + - The `obsidian-local-images-plus` plugin ensures images are stored locally relative to the vault or in a designated `assets` folder. + - Use the standard markdown syntax `![alt text](path/to/image.png)` to embed. + +## FAQ + +**Q: What if the calendar doesn't create notes for me?** +A: Ensure the `periodic-notes` plugin is installed and enabled in Obsidian's core settings. The `calendar` and `nldates` plugins rely on `periodic-notes` for configuration if available. + +**Q: How do I reset my task list?** +A: Tasks are stored as plain text in markdown files. You can clear completed tasks manually or use the Tasks plugin's "Reset all tasks" command from the command palette. + +**Q: Can I use this on mobile?** +A: Yes. Ensure you have synced the vault using Obsidian Sync or a git-based sync tool compatible with mobile (like Git, Dropbox, or iCloud). All plugins listed have mobile counterparts. + +**Q: How are dates formatted?** +A: The system defaults to `YYYY-MM-DD` for daily notes. This can be customized in the plugin settings for `nldates-obsidian` or `calendar`. + +## Related +- [[01 Projects/obsidian-vault/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md b/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..822e5a7 --- /dev/null +++ b/01 Projects/sandbox-notebookllamalm-nextjs/DEVELOPER_MANUAL.md @@ -0,0 +1,146 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- client/oliver +- domain/ai +- domain/devops +- tech/docker +- tech/fastapi +- tech/llamaindex +- tech/nextjs +- tech/postgresql +- tech/python +- tech/redis +- tech/typescript +- type/sop +type: sop +--- + +# 🦙 Sandbox NotebookLM Developer Manual + +## Architecture Overview +The application is a monorepo split into two main containers: +1. **Frontend**: A Next.js application serving the UI. +2. **Backend**: A Python FastAPI/Streamlit hybrid application handling business logic, AI integration, and database operations. + +It uses **PostgreSQL** for persistent storage and **Redis** for caching/sessions. All services are containerized via Docker Compose. + +## Tech Stack +- **Frontend**: Next.js (React), TypeScript. +- **Backend**: Python 3, FastAPI, Streamlit (for some components), SQLAlchemy, LlamaIndex. +- **Database**: PostgreSQL 18. +- **Cache/Broker**: Redis 7. +- **AI/ML**: LlamaIndex, OpenTelemetry, various LLM providers (OpenAI, Anthropic, Groq, etc.). +- **Infrastructure**: Docker, Docker Compose. + +## Local Setup + +### Prerequisites +- Git +- Docker & Docker Compose +- Python 3.10+ (if running backend locally without Docker) + +### Steps +1. **Clone Repository**: + ```bash + git clone git@bitbucket.org:zlalani/sandbox-notebookllamalm-nextjs.git + cd sandbox-notebookllamalm-nextjs + ``` + +2. **Configure Environment Variables**: + - Create `backend/.env` with the following keys: + ```env + OPENAI_API_KEY=sk-... + LLAMACLOUD_API_KEY=llx-... + pgql_user=postgres + pgql_psw=admin + pgql_db=postgres_nextjs + # Optional: AZURE_CLIENT_ID, AZURE_AUTHORITY, etc. for SSO + ``` + - Create `frontend/.env.local` for local dev: + ```env + NEXT_PUBLIC_API_URL=http://localhost:9000 + NEXT_PUBLIC_WS_URL=ws://localhost:9000 + ``` + +3. **Build and Start Services**: + ```bash + docker compose up -d --build + ``` + This starts: + - Backend: Port 9000 + - Frontend: Port 4000 + - PostgreSQL: Port 5433 + - Redis: Port 6380 + +4. **Initialize Database**: + ```bash + docker compose exec backend /app/.venv/bin/python -c \ + "import sys; sys.path.insert(0, '/app/src/notebookllama'); from database import init_db; init_db(); print('Done')" + ``` + +## Environment Variables + +### Backend +- `OPENAI_API_KEY`: Key for OpenAI models. +- `LLAMACLOUD_API_KEY`: Key for LlamaIndex Cloud services. +- `pgql_user`, `pgql_psw`, `pgql_db`: PostgreSQL connection details. +- `ELEVENLABS_API_KEY`: (Optional) For podcast generation. +- `AZURE_CLIENT_ID`, `AZURE_AUTHORITY`, `AZURE_REDIRECT_URI`: (Optional) For Microsoft SSO. + +### Frontend +- `NEXT_PUBLIC_API_URL`: Base URL for backend API calls. +- `NEXT_PUBLIC_WS_URL`: WebSocket URL for real-time updates. +- `NEXT_PUBLIC_AZURE_*`: Azure SSO configuration. + +## Key Services & Entry Points + +### Backend (`backend/src/notebookllama/`) +- `server.py`: Main FastMCP server defining tools (`process_file_tool`, `get_mind_map_tool`, `query_index_tool`). +- `auth.py`: Handles local password authentication (bcrypt) and SSO user creation/update. +- `instrumentation_init.py`: Initializes OpenTelemetry tracing for LlamaIndex observability. +- `instrumentation.py`: Custom SQL engine for exporting OpenTelemetry traces to PostgreSQL. +- `cost_tracker.py`: Lightweight HTTP client for tracking AI usage costs. + +### Frontend (`frontend/`) +- Next.js App Router structure. +- Environment variables baked into the Docker build via `frontend/.env.production`. + +## API Reference + +### Health Check +- **GET** `/api/health` +- Returns 200 if backend is healthy. + +### MCP Tools (Internal) +The backend exposes three main tools via FastMCP: +1. `process_file_tool(filename: str)`: Processes uploaded files, returns summary and text. +2. `get_mind_map_tool(summary: str, highlights: list[str])`: Generates mind map data. +3. `query_index_tool(question: str)`: Queries the LlamaIndex vector store. + +## Deployment +1. Ensure all required `OPENAI_API_KEY` and `LLAMACLOUD_API_KEY` are set. +2. Build the Docker images: + ```bash + docker compose build + ``` +3. Run in production mode: + ```bash + docker compose up -d + ``` +4. Configure reverse proxy (e.g., Nginx) to point to ports 9000 (backend) and 4000 (frontend) if not using the provided domain directly. + +## Known Gotchas +- **Database Initialization**: The database tables are **not** auto-created. You must run the `init_db` command after the first run. +- **SSO Merging**: If a user logs in via SSO after registering locally, the system merges the accounts (updates provider to SSO, clears password). Ensure your email addresses match exactly. +- **Service Availability**: The backend checks for `LLAMACLOUD_API_KEY` and `OPENAI_API_KEY` at startup. If missing, processing tools will return errors. Check logs for `Missing required environment variables`. +- **Health Checks**: The backend health check waits 30 seconds (`start_period`) before checking. Do not rely on immediate health check success on startup. +- **Port Mapping**: PostgreSQL is mapped to host port 5433, not 5432. Redis to 6380. Adjust local database clients accordingly. + +## Related +- [[01 Projects/ppt-tool/DEVELOPER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/sandbox-notebookllamalm-nextjs/USER_MANUAL.md b/01 Projects/sandbox-notebookllamalm-nextjs/USER_MANUAL.md new file mode 100644 index 0000000..0b4cc22 --- /dev/null +++ b/01 Projects/sandbox-notebookllamalm-nextjs/USER_MANUAL.md @@ -0,0 +1,79 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: User_Manual +status: active +tags: +- domain/ai +- domain/security +- tech/azure-ad +- tech/docker +- tech/elevenlabs +- tech/llamaindex +- tech/postgresql +- tech/redis +type: reference +--- + +# 🦙 Sandbox NotebookLM User Manual + +## What This Tool Does +Sandbox NotebookLM is an enterprise-ready alternative to Google NotebookLM. It allows users to upload documents, generate AI-powered summaries, create mind maps, and query their document content using Large Language Models (LLMs). It supports multi-user access, Single Sign-On (SSO) via Microsoft Azure, and real-time podcast generation. + +## Who Uses It +- **Students & Researchers**: For summarizing long papers and generating study aids. +- **Enterprise Teams**: For internal document knowledge bases with secure authentication. +- **Content Creators**: For generating transcripts and highlights from uploaded media. + +## How to Access +- **Live Demo**: https://ai-sandbox.oliver.solutions/notebookllama/ +- **Local Deployment**: Run the project locally using Docker Compose (see Developer Manual for setup). + +## Main Workflows + +### 1. Creating a Notebook +1. Log in using your email (local auth) or Microsoft SSO account. +2. Navigate to the "Create New Notebook" section. +3. Give your notebook a title and description. + +### 2. Uploading Documents +1. Open a notebook. +2. Click the **Upload** button. +3. Select supported file types (PDFs, TXT, DOCX, etc.). +4. Wait for the file to be processed. The backend will extract text and generate summaries/highlights. + +### 3. Querying Your Documents +1. In the notebook interface, use the chat/query sidebar. +2. Type your question regarding the uploaded content. +3. The AI will retrieve relevant information from your documents and provide an answer. + +### 4. Generating Podcasts (Optional) +1. After uploading files, look for the **Podcast** generation option. +2. Click to generate an audio discussion about your documents. +3. Requires an ElevenLabs API key configured by the administrator. + +### 5. Mind Map Creation +1. After processing files, select the **Mind Map** view. +2. View the hierarchical structure of key concepts extracted from your documents. + +## FAQ + +**Q: What file formats are supported?** +A: Primarily PDF, TXT, and DOCX. The backend processes these via LlamaIndex. + +**Q: Is my data private?** +A: Yes. Data is stored in a local PostgreSQL database and Redis cache. No third-party storage is used except for AI model API calls. + +**Q: Can I use this without an internet connection?** +A: No. It requires an internet connection to communicate with LLM providers (OpenAI, Anthropic, Groq, etc.) via their APIs. + +**Q: Why is my file not processing?** +A: Check that the file size is within limits and the format is supported. Also, ensure the backend services are running and healthy. + +**Q: How do I log out?** +A: Use the account menu in the top right corner of the frontend interface. + +## Related +- [[01 Projects/ppt-tool/USER_MANUAL.md]] \ No newline at end of file diff --git a/01 Projects/social-reporting-tool/DEVELOPER_MANUAL.md b/01 Projects/social-reporting-tool/DEVELOPER_MANUAL.md new file mode 100644 index 0000000..72efa69 --- /dev/null +++ b/01 Projects/social-reporting-tool/DEVELOPER_MANUAL.md @@ -0,0 +1,163 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: Developer_Manual +status: active +tags: +- domain/ai +- domain/devops +- tech/claude +- tech/docker +- tech/node +- tech/postgresql +- tech/typescript +type: sop +--- + +# Social Listening Pipeline: Developer Manual + +## Architecture Overview + +The project is a monorepo-style application consisting of: + +- **Frontend**: Static HTML/JS served by Apache, handling UI and SSE connections. +- **Backend (Node.js)**: `agents/social-listening/dashboard/` handles HTTP API, pipeline orchestration, and database interactions. +- **Pipeline**: `agents/social-listening/pipeline-v2.ts` orchestrates 8 stages of data processing. +- **Database**: PostgreSQL (`db` service) stores run history, briefs, and cost logs. +- **Infrastructure**: Docker Compose manages dependencies (Postgres, App). + +## Tech Stack + +- **Runtime**: Node.js 20+ +- **Database**: PostgreSQL 16 +- **AI**: Anthropic Claude API (via `claude-cli.ts`) +- **Scraping**: Apify SDK +- **Frontend**: Vanilla JS + MSAL (Microsoft Authentication Library) +- **Deployment**: Docker & Docker Compose +- **Web Server**: Apache (for static frontend and reverse proxy) + +## Local Setup + +1. **Prerequisites**: + - Docker & Docker Compose + - Node.js 20+ + - Apify API Token + - Anthropic API Key + +2. **Configuration**: + Copy `.env.example` to `.env` and fill in your keys: + ```env + APIFY_TOKEN=your_apify_token + ANTHROPIC_API_KEY=your_anthropic_key + APIFY_LIVE_APPROVED=true + APIFY_COST_LIMIT=5 + TEST_MODE=false + DASH_USER=admin + DASH_PASS=changeme + ``` + +3. **Run Services**: + ```bash + docker compose up -d + ``` + +4. **Local Development (Optional)**: + For live reload of the backend, run the Node.js server directly: + ```bash + cd agents/social-listening/dashboard + npm install + npm start + ``` + Ensure `ALLOWED_ORIGIN` in `.env` permits `http://localhost:3456`. + +## Environment Variables + +| Variable | Description | Default | +|----------|-------------|---------| +| `APIFY_TOKEN` | Apify API token | - | +| `ANTHROPIC_API_KEY` | Anthropic API key | - | +| `APIFY_LIVE_APPROVED` | Enable live Apify actors | `false` | +| `APIFY_COST_LIMIT` | Max spend for Apify | `5` (USD) | +| `TEST_MODE` | Enable mock data | `false` | +| `DASHBOARD_PORT` | Port for the dashboard | `3456` | +| `DB_PORT` | Port for local DB access | `5436` | +| `DASH_USER` | Dashboard username | `admin` | +| `DASH_PASS` | Dashboard password | `changeme` | +| `ALLOWED_ORIGIN` | CORS allowed origin | Empty | +| `SESSION_SECRET` | Secret for session signing | - | +| `AZURE_TENANT_ID` | Azure AD Tenant ID | - | +| `AZURE_CLIENT_ID` | Azure AD Client ID | - | + +## Key Services & Entry Points + +### 1. Pipeline Orchestrator +- **File**: `agents/social-listening/pipeline-v2.ts` +- **Function**: `runPipeline(rawBrief, onProgress, onCost)` +- **Description**: Coordinates the 8-stage pipeline. Emits progress and cost updates via callbacks. +- **Stages**: + 1. `stage1-brief.js`: Validates input. + 2. `stage2-strategy-review.js`: AI strategy review. + 3. `stage3-discovery-scrape.js`: Apify scraping. + 4. `stage4-data-review.js`: AI trend analysis. + 5. `stage5-enrichment-scrape.js`: Fetches transcripts/metadata. + 6. `stage6-pre-report-review.js`: AI refinement. + 7. `stage7-desk-research.js`: Web search (if implemented). + 8. `stage8-report.js`: Generates final HTML. + +### 2. Claude AI Client +- **File**: `agents/social-listening/claude-cli.ts` +- **Function**: `callClaude(messages, options)` +- **Description**: Wraps Anthropic API calls, handling token tracking and cost calculation. Uses `claude-opus-4-6` by default. + +### 3. Database Layer +- **File**: `agents/social-listening/db.js` +- **Functions**: `createRun`, `logCostEvent`, `finishRun`, `getRunTotals` +- **Schema**: See `db/init.sql` for table definitions. + +### 4. Frontend Configuration +- **File**: `frontend/config.js` +- **Variables**: + - `window.__API_BASE`: Backend API path (`/social-reports`). + - `window.__MSAL_CONFIG`: Azure AD SSO configuration. + +## API Reference + +The backend exposes REST endpoints under `/social-reports`: + +- `POST /social-reports/run`: Start a new pipeline run. +- `GET /social-reports/runs`: List past runs. +- `GET /social-reports/runs/:id`: Get run details. +- `POST /social-reports/briefs`: Save a brief. +- `GET /social-reports/briefs`: List saved briefs. +- `DELETE /social-reports/briefs/:id`: Delete a brief. + +*Note: Specific endpoint paths may vary; consult `dashboard/server.js` or equivalent for the exact router definitions.* + +## Deployment + +### Production Deployment +1. Use `docker-compose.prod.yml`: + ```bash + docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d + ``` +2. Set `NODE_ENV=production` and a secure `SESSION_SECRET` in `.env`. +3. Ensure `ALLOWED_ORIGIN` is set to your domain. + +### Azure AD SSO +1. Update `frontend/config.js` with your MSAL client ID and authority. +2. Set `AZURE_TENANT_ID` and `AZURE_CLIENT_ID` in environment variables. +3. Ensure the redirect URI is whitelisted in your Azure App Registration. + +## Known Gotchas + +- **Apify Actor Limits**: Ensure your Apify account has sufficient memory hours for scraping. +- **API Rate Limits**: Anthropic API has strict rate limits. Implement exponential backoff if needed. +- **CORS Issues**: If running frontend and backend separately, ensure `ALLOWED_ORIGIN` matches your frontend origin exactly. +- **Docker Volumes**: The `pgdata` volume persists DB data. Remove it to reset the database. +- **Unicode Handling**: The `claude-cli.ts` includes sanitization for unpaired surrogates to prevent JSON errors. + +## Related +- [[agents/social-listening/pipeline-v2]] +- [[agents/social-listening/dashboard]] \ No newline at end of file diff --git a/01 Projects/social-reporting-tool/USER_MANUAL.md b/01 Projects/social-reporting-tool/USER_MANUAL.md new file mode 100644 index 0000000..e316910 --- /dev/null +++ b/01 Projects/social-reporting-tool/USER_MANUAL.md @@ -0,0 +1,106 @@ +--- +auto_generated: true +created: 2026-05-18 +manual_updated_at: 2026-05-18 +modified: 2026-05-18 +name: User_Manual +status: active +tags: +- domain/analytics +- status/active +- tech/azure-ad +- tech/claude +- tech/docker +- tech/node +- type/sop +type: sop +--- + +# Social Listening Pipeline: User Manual + +## What This Tool Does + +The **Social Listening Pipeline** is an automated research tool that scrapes public data from TikTok, Instagram, and YouTube, analyzes the content using Claude AI, and generates professional, client-ready HTML reports. It helps marketing teams and researchers identify trends, opportunities, and competitive intelligence quickly. + +## Who Uses It + +- **Marketing Managers**: To track brand sentiment and competitor activity. +- **Social Media Strategists**: To discover trending topics and hashtags. +- **Client Consultants**: To produce white-label reports for stakeholders. + +## How to Access + +### Local Development +1. Ensure you have **Docker** and **Node.js 20+** installed. +2. Clone the repository. +3. Create a `.env` file based on `.env.example` (see Developer Manual for details). +4. Start the services: + ```bash + docker compose up -d + ``` +5. Access the dashboard at `http://localhost:3456` (or the port defined in your `.env` as `DASHBOARD_PORT`). + +### Production +Deploy using the provided `docker-compose.prod.yml` file alongside the base `docker-compose.yml`. + +## Main Workflows + +### 1. Creating a New Report + +1. **Log in** to the dashboard. + - If Azure AD SSO is enabled, you will be redirected to your organization's login. + - Otherwise, use the local credentials (`DASH_USER` and `DASH_PASS` in `.env`). +2. **Navigate to the "New Report" tab.** +3. **Fill in the Client Brief:** + - **Client Name**: Name of the client. + - **Category**: Industry or topic (e.g., "Fitness", "Tech"). + - **Platforms**: Select TikTok, Instagram, YouTube. + - **Hashtags**: Enter target hashtags. + - **Apify Budget**: Set a maximum spend limit for scraping costs (optional). +4. **Click "Run Pipeline"**. +5. **Monitor Progress:** + - The dashboard updates in real-time via Server-Sent Events (SSE). + - Watch the progress bar and live cost tracking. + - The pipeline runs 8 stages: Brief Validation, Strategy Review, Discovery Scrape, Data Review, Enrichment Scrape, Pre-Report Review, Desk Research, and Report Generation. +6. **Download the Report:** + - Once complete, the final HTML report is available for download. + - It includes video embeds (YouTube/Instagram) and trend analysis. + +### 2. Saving and Loading Briefs + +1. Go to the "Saved Briefs" tab. +2. Click "Save Brief" to store your current configuration server-side. +3. Select a saved brief from the list and click "Load" to populate the form. + +### 3. Viewing Run History + +1. Go to the "Run History" tab. +2. View past runs with details including: + - Date and time + - Cost breakdown (Claude AI and Apify) + - Status (Success/Error) +3. **Download** previous reports or **Delete** old entries. + +## FAQ + +**Q: What are the required API keys?** +A: You need an **Apify API Token** for scraping and an **Anthropic API Key** for AI analysis. Both must be set in your `.env` file. + +**Q: How much does a run cost?** +A: Costs depend on the length of the analysis and scraping scope. You can set an `APIFY_COST_LIMIT` in `.env` to cap scraping expenses. AI costs are tracked in real-time but not hard-capped by default. + +**Q: Can I use this with my company's Azure AD?** +A: Yes. Set `__SSO_ENABLED = true` in `frontend/config.js` and provide your `AZURE_TENANT_ID` and `AZURE_CLIENT_ID` in the environment variables. + +**Q: What happens if the Apify scrape fails?** +A: The pipeline will log an error and stop. Check the dashboard logs for details. Ensure your Apify API token is valid and has sufficient credits. + +**Q: How do I reset my local password?** +A: Update `DASH_PASS` in your `.env` file and restart the `social-listening` container: + ```bash + docker compose restart social-listening + ``` + +## Related +- [[05 Aimpress LTD/Projects/Social Listening Pipeline]] +- [[03 Resources/Developer Manual]] \ No newline at end of file diff --git a/99 Daily/2026-05-18.md b/99 Daily/2026-05-18.md index 3016117..7d00536 100644 --- a/99 Daily/2026-05-18.md +++ b/99 Daily/2026-05-18.md @@ -68,3 +68,5 @@ tags: [daily] - 18:30 — session ended | `.vault-agent` - 18:32 — session ended | `.vault-agent` - 18:37 (1min) — session ended | `.vault-agent` +- 18:43 — session ended | `pimco-charts` +- 18:45 — session ended | `.vault-agent`