agent-sync/PLAN-prompt-audit.md

Agent-Sync: Prompt Audit — Implementation Plan

Context

The agent-sync tool currently syncs agent metadata from LibreChat to Agent Tracker, but explicitly excludes the instructions field (system prompt) from the export. The Agent Tracker team needs these system prompts to perform automated compliance audits using AI analysis.

This plan adds system prompt syncing through the existing pipeline — a minimal change to two files.

What Changes

The instructions field from LibreChat's agents collection needs to flow through the existing pipeline:

LibreChat MongoDB → export_shared_agents.js → shared_agents.json → register_agents.py → POST /agents API
                    (currently excludes instructions)    (doesn't send system_prompt)

After changes:

LibreChat MongoDB → export_shared_agents.js → shared_agents.json → register_agents.py → POST /agents API
                    (now INCLUDES instructions)  (sends system_prompt field)

Files to Modify

1. export_shared_agents.js — Remove instructions exclusion (line 201)

Current code (lines 199-206):

{ $project: {
    "agentDetails.versions": 0,
    "agentDetails.instructions": 0,    // ← DELETE THIS LINE
    "authorDetails": 0,
    "usageTimeline": 0,
    "usageSummary": 0,
    "tokenUsage": 0
  }
}

After change — just remove line 201:

{ $project: {
    "agentDetails.versions": 0,
    "authorDetails": 0,
    "usageTimeline": 0,
    "usageSummary": 0,
    "tokenUsage": 0
  }
}

This means the instructions field will now be included in the exported shared_agents.json.

2. register_agents.py — Add system_prompt to payload (in build_payload())

Add to the payload dict (around line 170, alongside the other optional fields):

# System prompt for audit analysis:
"system_prompt": agent.get("instructions") or None,

Also add tool detail to metadata (in the metadata dict, around line 129-140):

"tool_resources": agent.get("tool_resources"),
"actions": agent.get("actions"),

The final prune at line 184 will automatically remove these if they're None/empty.

Verification

1. Run the export on the LibreChat server:

   docker exec chat-mongodb mongosh LibreChat --quiet --file /opt/agent-sync/export_shared_agents.js > /opt/agent-sync/shared_agents.json
   

2. Check that instructions appear in the JSON:

   python3 -c "import json; data=json.load(open('shared_agents.json')); print(any(d.get('agentDetails',{}).get('instructions') for d in data))"
   # Should print: True
   

3. Dry-run the registration to confirm system_prompt is in the payload:

   python register_agents.py --input shared_agents.json --dry-run 2>&1 | grep system_prompt
   

4. Once Agent Tracker has been updated to accept the system_prompt field in the collector API, run a full sync:

   ./weekly_agent_sync.sh
   

Dependencies

• Agent Tracker must be updated first (or simultaneously) to accept the new system_prompt field in the AgentCollectorCreate model. Without this, the field will be silently ignored by the API (Pydantic strips unknown fields).
• No new Python packages needed.
• No environment variable changes needed.

Rollback

If needed, simply re-add the exclusion line to export_shared_agents.js:

"agentDetails.instructions": 0,

The system prompt data already stored in Agent Tracker will remain but won't be updated on future syncs.