# Contract Analysis Tool v2.0

A modern, production-ready Retrieval-Augmented Generation (RAG) application for intelligent contract analysis and document Q&A. Built with FastAPI backend and React frontend with advanced features including SSO integration, context-aware chat, and comprehensive document processing.

![Architecture](https://img.shields.io/badge/Backend-FastAPI-009688)
![Frontend](https://img.shields.io/badge/Frontend-React-61DAFB)
![Database](https://img.shields.io/badge/Database-MongoDB-47A248)
![Cache](https://img.shields.io/badge/Cache-Redis-DC382D)
![Vector Store](https://img.shields.io/badge/VectorDB-ChromaDB-FF6B35)

## 🚀 Key Features

### Core Functionality
- **Modern Architecture**: FastAPI + React + MongoDB + Redis + ChromaDB
- **AI-Powered Analysis**: OpenAI GPT-4 integration with contract summarization
- **Advanced RAG System**: Context-aware document Q&A with source citations
- **Document Processing**: Multi-format support (PDF, DOCX, DOC, TXT, CSV, JSON, HTML, MD, RTF)
- **Vector Search**: ChromaDB for semantic similarity search

### Authentication & Security
- **Dual Authentication**: Local JWT + Azure AD/SSO integration
- **Role-Based Access Control**: Admin/User permissions
- **JWT Token Management**: Automatic refresh with 3-hour expiration
- **Secure File Upload**: Validation, sanitization, and size limits

### Advanced Chat System
- **Context-Aware Conversations**: 24-hour rolling context window (max 10 messages)
- **Smart Caching**: Context-dependent responses aren't cached, simple queries are
- **Real-time Statistics**: Response times, cache hit rates, message counts
- **Proper Message Ordering**: Chronological display with accurate timestamps
- **Source Citations**: Direct references to document chunks in responses

### Document Management
- **Batch Processing**: Multiple document uploads with progress tracking
- **Index Organization**: Create themed document collections
- **Processing Pipeline**: PDF parsing → chunking → embedding → vector storage
- **Status Tracking**: Real-time processing and embedding status
- **Contract Summaries**: Automated contract analysis and key point extraction

### Admin Features
- **System Statistics**: Monitor usage, performance, and system health
- **User Management**: Create, edit, and manage user accounts
- **Document Reprocessing**: Retry failed documents
- **Index Management**: Create and manage document indices
- **Advanced RAG Interface**: Admin-specific query tools

## 🏗️ Architecture

```
React Frontend (Vite + Tailwind) → FastAPI Backend → MongoDB + ChromaDB → OpenAI API
                                        ↓
                                   Redis Cache
                                        ↓
                              Azure AD/SSO (Optional)
```

**Data Flow:**
1. Documents uploaded through React frontend
2. FastAPI processes with LlamaIndex (chunking, parsing)
3. OpenAI embeddings stored in ChromaDB
4. Metadata and user data in MongoDB
5. RAG queries combine vector search + GPT-4 generation
6. Redis caches responses for performance

## 📋 Prerequisites

- **Python 3.11+** (Backend)
- **Node.js 18+** (Frontend)
- **MongoDB 7+** (Document metadata)
- **Redis 7+** (Caching - optional)
- **OpenAI API Key** (Required)
- **LlamaParse API Key** (Optional - enhanced PDF processing)

## 🛠️ Quick Start

### Option 1: Docker (Recommended)

```bash
# Clone repository
git clone <repository-url>
cd llama-contracts-master

# Configure environment
cp backend/.env.example backend/.env
# Edit backend/.env with your API keys

# Start backend services
cd backend
docker-compose up -d

# Start frontend
cd ../frontend
npm install
npm run dev
```

### Option 2: Manual Development Setup

#### Backend Setup
```bash
cd backend
python3 -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate
pip install -r requirements.txt

# Configure environment
cp .env.example .env
# Edit .env file with your settings

# Start services (macOS with Homebrew)
brew services start mongodb-community
brew services start redis

# Initialize database and start server
uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
```

#### Frontend Setup
```bash
cd frontend
npm install

# Configure environment  
cp .env.example .env
# Edit frontend/.env

# Start development server
npm run dev
```

## ⚙️ Configuration

### Backend Environment Variables (.env)
```env
# Database Configuration
MONGODB_URL=mongodb://localhost:27017
DATABASE_NAME=contract_analysis

# Redis Cache (Optional)
REDIS_URL=redis://localhost:6379

# Authentication
JWT_SECRET_KEY=your-super-secret-jwt-key-change-in-production
JWT_ALGORITHM=HS256
JWT_EXPIRE_MINUTES=180

# Azure AD/SSO (Optional)
AZURE_CLIENT_ID=your-azure-client-id
AZURE_TENANT_ID=your-azure-tenant-id
AZURE_REDIRECT_URI=http://localhost:3000/auth/callback
SSO_ENABLED=false
ALLOW_LOCAL_ADMIN=true

# AI Services (Required)
OPENAI_API_KEY=your-openai-api-key
LLAMAPARSE_API_KEY=your-llamaparse-api-key

# Application Settings
DEBUG=true
CORS_ORIGINS=["http://localhost:3000","http://localhost:3002"]
UPLOAD_DIR=./uploads
INDICES_DIR=./indices

# Performance Settings
CACHE_ENABLED=false  # Disabled for development
CACHE_TTL=3600
MAX_DOCUMENT_CHARS=1000000
MAX_SUMMARY_CHARS=100000
```

### Frontend Environment Variables (.env)
```env
VITE_API_URL=http://localhost:8000
VITE_APP_NAME=Contract Analysis Tool
```

## 🚀 Getting Started

### 1. Initialize System
```bash
# Create default users (first time only)
curl -X POST http://localhost:8000/api/v1/auth/init-users
```

### 2. Default Credentials
- **Admin**: `admin@oliver.agency` / `admin123`
- **User**: `user@oliver.agency` / `user123`

### 3. Basic Workflow
1. **Login** at http://localhost:3000/login
2. **Create Index** for your document collection
3. **Upload Documents** (supports drag-and-drop)
4. **Wait for Processing** (documents → chunks → embeddings)
5. **Start Chatting** with natural language queries
6. **Review Sources** cited in AI responses

## 📚 API Documentation

### Development URLs
- **Application**: http://localhost:3000
- **API Docs**: http://localhost:8000/docs
- **ReDoc**: http://localhost:8000/redoc
- **Health Check**: http://localhost:8000/health

### Core API Endpoints

#### Authentication
- `POST /api/v1/auth/login` - Local login
- `POST /api/v1/auth/register` - User registration
- `GET /api/v1/auth/me` - Current user info
- `POST /api/v1/auth/refresh` - Token refresh
- `POST /api/v1/auth/sso/validate` - SSO login
- `GET /api/v1/auth/sso/config` - SSO configuration

#### Document Management
- `POST /api/v1/documents/upload` - Upload documents to index
- `GET /api/v1/documents/index/{index_id}` - List documents in index
- `GET /api/v1/documents/{document_id}` - Document details
- `DELETE /api/v1/documents/{document_id}` - Delete document
- `GET /api/v1/documents/{document_id}/summary` - Contract summary
- `POST /api/v1/documents/{document_id}/summary/reprocess` - Regenerate summary

#### Index Management
- `POST /api/v1/indices/create` - Create document index
- `GET /api/v1/indices/` - List user indices

#### Chat System
- `POST /api/v1/chat/query` - RAG query with context
- `GET /api/v1/chat/history/{index_id}` - Conversation history
- `DELETE /api/v1/chat/history/{index_id}` - Clear chat history

#### Admin Operations
- `GET /api/v1/admin/stats` - System statistics
- `POST /api/v1/admin/documents/upload-single` - Single document upload
- `POST /api/v1/admin/documents/upload-multiple` - Batch upload
- `POST /api/v1/admin/documents/{document_id}/reprocess` - Reprocess document
- `GET /api/v1/admin/indices` - All system indices
- `POST /api/v1/admin/chat/query` - Admin RAG interface

## 🔧 Advanced Features

### Context-Aware Chat System
- **Conversation Memory**: AI remembers last 10 messages within 24 hours
- **Smart Context Usage**: Follow-up questions reference previous conversation
- **Context Indicators**: UI shows when AI uses conversation history
- **Session Statistics**: Track response times and context usage

### Document Processing Pipeline
1. **Upload**: Drag-and-drop or browse files
2. **Validation**: File type, size, and content checks
3. **Processing**: LlamaIndex parsing and chunking
4. **Embedding**: OpenAI embeddings generation
5. **Storage**: ChromaDB vector storage + MongoDB metadata
6. **Indexing**: Ready for semantic search

### SSO Integration (Optional)
- **Azure AD Support**: Enterprise authentication
- **Local Fallback**: Admin accounts always available
- **Role Mapping**: Automatic role assignment from SSO claims
- **Session Management**: Unified token handling

### Advanced Caching Strategy
- **Smart Cache Logic**: Context-dependent queries bypass cache
- **Simple Query Cache**: Repeated questions served from Redis
- **TTL Management**: Configurable cache expiration
- **Cache Statistics**: Monitor hit rates and performance

## 🏗️ Project Structure

### Backend (`/backend`)
```
app/
├── main.py              # FastAPI application entry point
├── config/
│   ├── settings.py      # Environment configuration
│   └── database.py      # Database connections
├── api/v1/              # API endpoints
│   ├── auth.py         # Authentication routes
│   ├── documents.py    # Document management
│   ├── indices.py      # Index operations
│   ├── chat.py         # Chat/RAG system
│   └── admin.py        # Admin operations
├── models/              # Data models
│   ├── user.py         # User models
│   ├── document.py     # Document models
│   ├── index.py        # Index models
│   ├── chat.py         # Chat models
│   └── contract_summary.py # Summary models
├── services/            # Business logic
│   ├── document_processor.py    # Document processing
│   ├── rag_service.py          # RAG implementation
│   ├── chat_context_service.py # Context management
│   ├── contract_summary_service.py # Summarization
│   └── sso_service.py          # SSO integration
├── core/                # Core utilities
│   ├── auth.py         # Authentication logic
│   ├── security.py     # Security utilities
│   ├── cache.py        # Caching logic
│   └── chroma_client.py # ChromaDB client
└── utils/               # Helper utilities
    └── file_utils.py    # File operations
```

### Frontend (`/frontend`)
```
src/
├── App.jsx             # Main application with routing
├── pages/              # Page components
│   ├── HomePage.jsx    # Landing page
│   ├── Dashboard.jsx   # Main dashboard
│   ├── DocumentManager.jsx # Document management
│   ├── ChatInterface.jsx   # Chat interface
│   └── AdminPanel.jsx      # Admin interface
├── components/         # Reusable components
│   ├── auth/          # Authentication components
│   ├── admin/         # Admin-specific components
│   ├── chat/          # Chat components
│   ├── documents/     # Document components
│   ├── indices/       # Index components
│   └── common/        # Shared components
├── services/          # API service layer
│   ├── authService.js    # Authentication API
│   ├── documentService.js # Document API
│   ├── chatService.js    # Chat API
│   ├── indexService.js   # Index API
│   └── adminService.js   # Admin API
├── context/           # React context providers
│   └── AuthContext.jsx  # Authentication context
└── utils/             # Frontend utilities
    └── constants.js      # Application constants
```

## 🧪 Development & Testing

### Backend Testing
```bash
cd backend
source venv/bin/activate

# API testing via Swagger
# Visit http://localhost:8000/docs

# Manual testing
python test_chat_fixes.py
```

### Frontend Development
```bash
cd frontend

# Development server with hot reload
npm run dev

# Build for production
npm run build

# Preview production build
npm run preview

# Lint code
npm run lint
```

### Database Management
```bash
# View MongoDB collections
mongo contract_analysis

# View Redis cache
redis-cli
> KEYS *

# Clear ChromaDB indices
rm -rf backend/indices/chroma_db/
```

## 📊 Monitoring & Health Checks

### System Health
```bash
# Backend health
curl http://localhost:8000/health

# Database connectivity
curl http://localhost:8000/api/v1/admin/stats \
  -H "Authorization: Bearer <admin-token>"
```

### Performance Monitoring
- **Response Times**: Tracked per request with `X-Process-Time` header
- **Cache Hit Rates**: Monitor Redis performance
- **Document Processing**: Track success/failure rates
- **Vector Search**: Monitor ChromaDB query performance

## 🐛 Troubleshooting

### Common Issues

**1. MongoDB Connection Issues**
```bash
# Check if MongoDB is running
brew services list | grep mongodb
brew services start mongodb-community

# Check connection string in .env
MONGODB_URL=mongodb://localhost:27017
```

**2. Redis Connection Issues**
```bash
# Redis is optional - app continues without caching
brew services start redis

# Test Redis connection
redis-cli ping
```

**3. Document Processing Failures**
- Check OpenAI API key validity
- Verify file format support
- Review file size limits (50MB default)
- Check upload directory permissions

**4. ChromaDB Issues**
```bash
# Clear and reinitialize indices
rm -rf backend/indices/chroma_db/
# Restart backend to recreate
```

**5. Frontend Build Issues**
```bash
cd frontend
rm -rf node_modules package-lock.json
npm install
npm run dev
```

### Log Analysis
- **Backend**: Console output from uvicorn
- **Frontend**: Browser developer console
- **Database**: MongoDB logs in system logs
- **Processing**: Check document status in MongoDB

## 🚀 Production Deployment

### Production Checklist
- [ ] Set strong JWT secret key
- [ ] Configure production database URLs
- [ ] Enable HTTPS with SSL certificates
- [ ] Set up reverse proxy (Nginx/Apache)
- [ ] Configure monitoring and logging
- [ ] Set up regular MongoDB backups
- [ ] Disable debug mode
- [ ] Configure proper CORS origins
- [ ] Set up log rotation

### Docker Production
```bash
# Use production compose file
docker-compose -f docker-compose.prod.yml up -d

# Or build custom images
docker build -t contract-analysis-backend ./backend
docker build -t contract-analysis-frontend ./frontend
```

### Environment Variables for Production
```env
DEBUG=false
JWT_SECRET_KEY=your-ultra-secure-secret-key-here
CORS_ORIGINS=["https://yourdomain.com"]
CACHE_ENABLED=true
```

## 🔐 Security Considerations

### Implemented Security
- **JWT Authentication** with configurable expiration
- **Role-based Authorization** (Admin/User)
- **Input Validation** with Pydantic schemas
- **File Upload Validation** (type, size, content)
- **CORS Protection** with configurable origins
- **Environment Variable Protection**
- **SQL Injection Prevention** (NoSQL with validation)
- **XSS Prevention** (React built-in protection)

### Best Practices
- Regularly rotate JWT secret keys
- Use HTTPS in production
- Keep dependencies updated
- Monitor for security vulnerabilities
- Implement rate limiting for production
- Regular security audits

## 🤝 Contributing

1. **Fork** the repository
2. **Create** a feature branch (`git checkout -b feature/amazing-feature`)
3. **Commit** changes (`git commit -m 'Add amazing feature'`)
4. **Push** to branch (`git push origin feature/amazing-feature`)
5. **Open** a Pull Request

### Development Guidelines
- Follow existing code style and conventions
- Add tests for new features
- Update documentation as needed
- Ensure all tests pass before submitting PR

## 📄 License

This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

## 🙏 Acknowledgments

- **[OpenAI](https://openai.com/)** - GPT-4 and embedding models
- **[LlamaIndex](https://www.llamaindex.ai/)** - RAG framework and document processing
- **[ChromaDB](https://www.trychroma.com/)** - Vector database for semantic search
- **[FastAPI](https://fastapi.tiangolo.com/)** - Modern Python web framework
- **[React](https://react.dev/)** - Frontend framework
- **[Tailwind CSS](https://tailwindcss.com/)** - Utility-first CSS framework
- **[Vite](https://vitejs.dev/)** - Fast frontend build tool

---

**Built with ❤️ for intelligent contract analysis and document Q&A**

*For detailed migration information from v1.0, see [MIGRATION_PLAN.md](MIGRATION_PLAN.md)*
*For API testing guidance, see [API_TESTING_GUIDE.md](API_TESTING_GUIDE.md)*