---
auto_generated: true
created: 2026-05-19
manual_updated_at: 2026-05-19
modified: 2026-05-19
name: User_Manual
status: active
tags:
- client/barclays
- domain/ai
- status/active
- tech/gemini
- tech/openai
- tech/python
- tech/rag
- type/reference
type: reference
---

# RAG Test App User Manual

## What This Tool Does
The RAG Test App is a comprehensive tool designed to evaluate the performance of OpenAI Assistants with Retrieval-Augmented Generation (RAG) capabilities. It allows users to:
- Test assistant responses against specific documents
- Generate questions based on different strategies (task-based, content-based, scenario-based)
- Evaluate assistant accuracy and performance
- Generate detailed reports and visualizations

## Who Uses It
- AI Engineers testing RAG implementations
- Data Scientists evaluating model performance
- QA Engineers validating assistant behavior
- Researchers conducting AI experiment analysis

## How to Access

### Installation
```bash
pip install rag-test-app
```

### Configuration
Create a JSON configuration file (e.g., `config.json`):
```json
{
  "api_key": "your-openai-api-key",
  "assistant_id": "asst_xxxxx",
  "document": "path/to/document.txt",
  "output_dir": "results",
  "num_questions": 20,
  "iterations": 3,
  "model": "gpt-4o",
  "prompt_type": "task-based",
  "parallel": 5
}
```

## Main Workflows

### 1. Basic Evaluation
```bash
rag-test \
  --config config.json \
  --api-key sk-xxxxx \
  --assistant-id asst_xxxxx \
  --document docs/report.pdf
```

### 2. Generate Questions Only
```bash
rag-test \
  --config config.json \
  --generate-only
```

### 3. Parallel Testing
```bash
rag-test \
  --config config.json \
  --parallel 10
```

### 4. Custom Model
```bash
rag-test \
  --config config.json \
  --model gpt-4-turbo
```

## Step-by-Step Example

1. **Setup Configuration**
   ```json
   {
     "api_key": "sk-your-key",
     "assistant_id": "asst_123",
     "document": "./company_policy.pdf",
     "num_questions": 50,
     "model": "gpt-4o"
   }
   ```

2. **Run Test**
   ```bash
   rag-test --config my_config.json
   ```

3. **View Results**
   - Check `results/` directory
   - View `report.md` for summary
   - Check `visualizations/` for charts

## FAQ

**Q: What formats are supported?**
A: TXT, PDF, DOCX, and Markdown files.

**Q: How are results measured?**
A: Through accuracy scores, response times, and semantic similarity metrics.

**Q: Can I test multiple documents?**
A: Yes, use the `documents` key in config with a list of paths.

**Q: How do I run tests in parallel?**
A: Use the `--parallel` flag or set `parallel` in config.

**Q: Where are the results stored?**
A: In a timestamped directory within the output_dir (default: `results/`)

## Related
- [[01 Projects/barclays-rag-report/DEVELOPER_MANUAL]]
- [[01 Projects/barclays-rag-report/DEVELOPER_MANUAL.md]]
- [[01 Projects/barclays-rag-report/USER_MANUAL.md]]