f3a63d5d54
- Removed instructions exclusion from export pipeline so system prompts flow through - Added system_prompt field to registration payload for compliance audits - Added tool_resources and actions to metadata - Created README.md and CLAUDE.md for project documentation Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
96 lines
3.3 KiB
Markdown
96 lines
3.3 KiB
Markdown
# Agent Sync
|
|
|
|
Syncs agent metadata from LibreChat (MongoDB) to the Agent Tracker API. Runs as a daily cron job on the LibreChat web server (`optical-librechat`).
|
|
|
|
## Pipeline
|
|
|
|
```
|
|
LibreChat MongoDB --> export_shared_agents.js --> shared_agents.json --> register_agents.py --> Agent Tracker API
|
|
(aclentries, (mongosh script) (intermediate) (Python script) (POST /agents)
|
|
agents, users,
|
|
conversations,
|
|
messages,
|
|
transactions)
|
|
```
|
|
|
|
## Files
|
|
|
|
| File | Description |
|
|
|------|-------------|
|
|
| `export_shared_agents.js` | MongoDB aggregation pipeline that exports shared agents with usage stats, token usage, and system prompts |
|
|
| `register_agents.py` | Python script that maps exported agent data to the Agent Tracker API payload and POSTs it |
|
|
| `weekly_agent_sync.sh` | Shell wrapper that runs the full export-then-register pipeline |
|
|
| `shared_agents.json` | Intermediate export output (generated, not committed) |
|
|
| `registration_results.json` | Log of API registration results from the last run |
|
|
| `requirements.txt` | Python dependencies (`requests`, `urllib3`) |
|
|
|
|
## Server Deployment
|
|
|
|
- **Server:** `optical-librechat` (GCP)
|
|
- **Install path:** `/opt/agent-sync/`
|
|
- **LibreChat install:** `/home/michael_clervi/LibreChat/` (Docker Compose)
|
|
- **LibreChat Docker containers:** `LibreChat-API`, `LibreChat-NGINX`, `chat-mongodb`
|
|
- **Cron schedule:** Daily at midnight (`0 0 * * *`), runs under root's crontab
|
|
- **Logs:** `/opt/agent-sync/log/agent_sync.log`
|
|
- **Python venv:** `/opt/agent-sync/venv/`
|
|
|
|
## Setup (on the server)
|
|
|
|
```bash
|
|
cd /opt/agent-sync
|
|
python3 -m venv venv
|
|
venv/bin/pip install -r requirements.txt
|
|
```
|
|
|
|
## Usage
|
|
|
|
### Manual run
|
|
|
|
```bash
|
|
sudo /opt/agent-sync/weekly_agent_sync.sh
|
|
```
|
|
|
|
### Dry run (preview payloads without sending)
|
|
|
|
```bash
|
|
# Export first
|
|
docker exec -i chat-mongodb env MONGOSH_NO_RC=1 mongosh --norc --quiet LibreChat --file /dev/stdin < /opt/agent-sync/export_shared_agents.js > /opt/agent-sync/shared_agents.json
|
|
|
|
# Preview
|
|
/opt/agent-sync/venv/bin/python /opt/agent-sync/register_agents.py --input /opt/agent-sync/shared_agents.json --dry-run
|
|
```
|
|
|
|
### Environment variables (optional)
|
|
|
|
| Variable | Default | Description |
|
|
|----------|---------|-------------|
|
|
| `AGENT_REG_URL` | `https://ai-sandbox.oliver.solutions/agent_collector/agents` | Agent Tracker API endpoint |
|
|
| `AGENT_REG_KEY` | (built-in static key) | API key for authentication |
|
|
|
|
## What gets synced
|
|
|
|
For each shared agent (agents used by more than one user):
|
|
|
|
- **Core fields:** name, description, purpose, model/provider, author email
|
|
- **System prompt:** the agent's `instructions` field (for compliance audit)
|
|
- **Usage timeline:** daily message counts per agent
|
|
- **Usage summary:** conversation count, unique users, total messages, first/last used dates
|
|
- **Token usage:** prompt tokens, completion tokens, total tokens (from LibreChat transactions)
|
|
- **Metadata:** tool resources, actions, avatar, category, project IDs
|
|
- **Agent URL:** direct link to the agent in LibreChat
|
|
|
|
## Updating
|
|
|
|
To deploy changes:
|
|
|
|
1. Edit files locally and commit
|
|
2. Copy the changed files to `/opt/agent-sync/` on the server
|
|
3. No restart needed — scripts run on-demand via cron
|
|
|
|
## Rollback
|
|
|
|
To stop syncing system prompts, re-add this line to the `$project` stage in `export_shared_agents.js`:
|
|
|
|
```js
|
|
"agentDetails.instructions": 0,
|
|
```
|