---
name: "Ford QC System"
client: Ford
status: active
tech: [Python, Box SDK, HTML/Bootstrap, JSON, ZIP/Pillow, systemd]
local_path: /Users/ai_leed/Documents/Projects/Oliver/ford_qc
deploy: sudo systemctl start ford-qc-hotfolder.service
url: Box folder 332861865120
server: box-cli
tags: [ford, qc, quality-control, box, bnp, systemd, hotfolder]
created: 2026-04-14
last_commit: 2026-04-29
commits: 23
service: ford-qc-hotfolder.service
---

## Overview

Ford QC is an automated quality control system that validates Ford Build & Price (BnP) asset packs—ZIP files containing automotive images, metadata, and configuration—against 20 specialized validation checks. It monitors a Box cloud folder for incoming asset packs, processes them through the QC engine, generates detailed HTML reports, and stores results back to Box. The system is production-deployed as a systemd daemon on the `box-cli` host and handles both MEC (Market Enabled Configuration) and BAU (Business as Usual) asset types.

## Tech Stack
- **Frontend:** HTML/Bootstrap (static report generation)
- **Backend:** Python 3, Box SDK, Flask-adjacent (CLI + daemon pattern)
- **Database:** N/A (stateless processing; Box as storage)
- **Infrastructure:** systemd service, Box cloud integration, Python venv
- **AI/ML:** N/A
- **Key libraries:** `boxsdk` (3.13.0), `pillow` (11.1.0), `python-dotenv` (1.0.1), `requests` (2.32.3)

## Architecture

The system comprises three primary execution paths:

1. **CLI Entry Point (`qc_engine.py`)**: Loads a JSON profile, executes checks in sequence, generates JSON results, and optionally produces HTML reports. Used for manual/ad-hoc testing.

2. **QC Module (`qc_module.py`)**: Reusable Python module exposing `run_qc_checks()` and `run_qc_profile()` functions for programmatic check execution with enhanced error handling.

3. **Box Hotfolder Daemon (`ford_qc_box_hotfolder_process.py`)**: Production service monitoring Box folder `332861865120` for incoming `*_GPAS.zip` files. Downloads, processes through QC, uploads JSON/HTML reports to folder `332864939558`, and moves processed files to folder `332861653811` with MD5 stamps. Implements file locking, graceful shutdown, and systemd watchdog integration.

**Check Modules** (13 in `checks/` directory):
- `zip_filename_check`: Enforces `_GPAS.zip` suffix (Ford 3rd-party requirement)
- `unzip_and_verify_check`: Extracts ZIP and validates `linkingrecord.json` presence
- `colour_existence_check`: Verifies color chip files exist and are accessible
- `missing_images_check`: Cross-references metadata vs. actual files
- `special_requirements_mec_bau`: Auto-determines asset type (MEC/BAU) and validates type-specific rules
- `mec_powertrain_validation`, `lifestyle_inventory_validation`, `image_resolution_check`, `image_format_check`, `exterior_interior_pairing_check`, `base_assets_check`, `carousel_images_check`, `linkingrecord_header_validation`: Specialized validations per asset type
- `html_reporter.py`: Converts JSON results to Bootstrap-styled HTML success reports
- `html_error_reporter.py`: Converts error data to HTML error reports

**Data Flow**:
```
Box Input Folder (332861865120)
           ↓
[Hotfolder daemon polls every 60s]
           ↓
Download ZIP to /working_dir
           ↓
Extract + Run 13 QC checks → JSON results
           ↓
Generate HTML reports (success & error)
           ↓
Upload JSON/HTML to Box Report Folder (332864939558)
           ↓
Move processed ZIP to Box Processed Folder (332861653811) + MD5 stamp
```

## Dev Commands

```bash
# Install dependencies
pip install -r requirements.txt

# Run QC on a local ZIP file
python qc_engine.py profiles/ford_bnp.json --input_file path/to/asset_pack.zip --reports_dir reports

# Generate HTML report from existing JSON results
python -m checks.html_reporter results.json output_directory/

# Generate HTML error report
python -m checks.html_error_reporter <error_type> <filename> output_directory/

# Test a single check module
python -c "from checks.image_format_check import run_check; print(run_check({'working_dir': 'working'}))"

# Run all tests
python -m pytest tests/

# Start the Box hotfolder daemon (locally)
python ford_qc_box_hotfolder_process.py

# Verify systemd service status (production)
sudo systemctl status ford-qc-hotfolder.service

# View live production logs
journalctl -u ford-qc-hotfolder.service -f

# Restart production service
sudo systemctl restart ford-qc-hotfolder.service
```

## Deployment
- **Server:** box-cli
- **Deploy:** `sudo systemctl start ford-qc-hotfolder.service` (or `restart`)
- **URL:** Box folder ID `332861865120` (input), `332864939558` (reports output)
- **Port:** N/A (daemon; no HTTP port)
- **Service:** `ford-qc-hotfolder.service` (systemd)
- **Local path:** /Users/ai_leed/Documents/Projects/Oliver/ford_qc
- **Production path:** `/home/box-cli/FORD_SCRIPTS/ford_qc_git_prod/ford_qc`

**Setup**:
1. Copy `ford-qc-hotfolder.service` to `/etc/systemd/system/`
2. Run `sudo systemctl daemon-reload && sudo systemctl enable ford-qc-hotfolder.service`
3. Configure `.env.prod` with Box credentials and folder IDs in the script directory
4. Start: `sudo systemctl start ford-qc-hotfolder.service`

## Environment Variables

All env vars are loaded from `.env.prod` (production) or `.env.dev` (dev). Key variables:

- `FORD_QC_ENV` — Environment name (`production` or `development`)
- `BOX_SOURCE_FOLDER_ID` — Folder monitored for incoming ZIPs (e.g., `332861865120`)
- `BOX_REPORT_FOLDER_ID` — Folder where JSON/HTML reports are uploaded (e.g., `332864939558`)
- `BOX_PROCESSED_FOLDER_ID` — Folder where successfully processed files are moved (e.g., `332861653811`)
- `BOX_CONFIG_FILE` — Path to Box JWT JSON credentials file (default: `ford_box_config.json`)
- `FORD_QC_SCRIPT_DIR` — Base directory for QC system (auto-detected if not set)
- `DOWNLOAD_PATH` — Temp directory for Box downloads (default: `download_tmp`)
- `WORKING_DIR` — Directory for ZIP extraction and processing (default: `working`)
- `QC_PROFILE_PATH` — QC profile file (default: `profiles/ford_bnp.json`)
- `BOX_CONNECTION_TIMEOUT` — Timeout for Box API connection (seconds; default: 30)
- `BOX_READ_TIMEOUT` — Timeout for Box API read (seconds; default: 90)
- `MAX_RETRIES` — Max retry attempts for failed operations (default: 3)
- `RETRY_BACKOFF_BASE` — Base for exponential backoff (seconds; default: 2)
- `WATCHDOG_INTERVAL` — Systemd watchdog notification interval (seconds; default: 30)
- `LOG_LEVEL` — Log verbosity (`DEBUG`, `INFO`, `WARNING`, `ERROR`, `CRITICAL`; default: `INFO`)

## Known Issues

- **No SSH to production without explicit instruction** — security constraint
- **File locking enforced** — do not run multiple hotfolder instances simultaneously
- **ZIP filename enforcement** — files must end with `_GPAS.zip`; enforced by `zip_filename_check.py`
- **Box folder IDs hardcoded** — in `ford_qc_box_hotfolder_process.py`; changes require careful review
- **Python venv required** — production runs from `venv/bin/python`; system Python not supported
- **MEC vs BAU auto-detection** — logic in `special_requirements_mec_bau.py` determines asset type; ensure linkingrecord.json structure is valid
- **Ranger ptvl pattern configurable** — via profile; see recent git commits for changes
- **Exterior/interior pairing validation** — uses variant-signature approach; angle 21 support added recently

## Git

- **Remote:** git@bitbucket.org:zlalani/ford_qc.git
- **Last 20 commits include**: exterior/interior pairing, CV ABM support, linkingrecord header validation, asset count summary, GPAS naming enforcement, carousel image checks, Box config debugging, environment configuration, systemd watchdog improvements, and boxsdk upgrade to 3.13.0

## Timeline / Git History
| Date | Change |
|------|--------|
| 2026-04-29 | Add linkingrecord header validation check |
| 2026-04-17 | Add asset count summary to QC HTML reports |
| 2026-04-16 | Add zip filename check for GPAS naming convention |
| 2026-03-16 | Make Ranger ptvl pattern configurable via profile |

## Sessions
### 2026-05-01 – Check QC tool configuration and reboot
**Asked:** Check QC tool configuration and reboot the server to diagnose the issue.
**Done:** Identified that DEV environment has an invalid Box config with incorrect JWT key while PROD config is valid, causing the QC tool failure.

### 2026-05-01 – Asked | Done | Log
**Asked:** Asked | Done | Log
**Done:** ---|---|---

### 2026-05-01 – Can you restart the QC tool
**Asked:** Can you restart the QC tool and investigate the reported errors?
**Done:** Identified that the DEV service fails with Box API authentication error due to mismatched client credentials between DEV and PROD environments.

### 2026-05-01 – Check QC tool issues and investigate
**Asked:** Check QC tool issues and investigate DEV service status on the box-cli server.
**Done:** Connected to box-cli server and reviewed service logs to diagnose QC tool failures.

### 2026-04-29 – Verify series image counts using SE#
**Asked:** Verify series image counts using SE# and ABM codes, accounting for SIPS variations.
**Done:** Removed incorrect images and links for ambl2se#r1, verified all 20/20 tests pass, committed, pushed, and deployed to dev.

### 2026-04-29 – Check series image accounting using SE#
**Asked:** Check series image accounting using SE# and ABM WERS codes to verify all series are covered.
**Done:** Identified and removed duplicate ambl2se#r1 image linking while retaining series image; restarted ford-qc-hotfolder.service dev environment.

### 2026-04-29 – Implement a check to validate series
**Asked:** Implement a check to validate series images using SE# and ABM codes to ensure all series are accounted for.
**Done:** Rewrote check_series_permutations.py with autodetection logic to identify orphan series carousel entries lacking backing images.

### 2026-04-29 – Check that all series images are
**Asked:** Check that all series images are accounted for using SE# and ABM WERS codes combination.
**Done:** Implemented series permutation check that detects orphan carousel entries missing backing images and validates SE# and ABM code combinations.

### 2026-04-29 – Check which commit contains the Ford
**Asked:** Check which commit contains the Ford image pack naming feature and deploy it to production.
**Done:** Deployed zip filename validation check to production and identified service is running from different path requiring file sync.

### 2026-04-29 – Asked | Rename image pack files
**Asked:** Asked | Rename image pack files to end with _GPAS.zip instead of _image.zip to support Ford's 3rd party checker | ford_bnp.json
**Done:** Done | Updated ford_bnp.json to add zip format validation requiring _GPAS.zip suffix | ford_bnp.json

### 2026-04-29 – Identify which commit introduced the image
**Asked:** Identify which commit introduced the image pack naming requirement change and apply it to production.
**Done:** Located two files to deploy from commit 16bbd98 — the new zip filename validation check and updated Ford profile configuration.

### 2026-04-29 – Check which commit contains the image
**Asked:** Check which commit contains the image pack naming feature and deploy it to production.
**Done:** Identified commit 16bbd98 on origin/main with deployment command ready for the production server.

### 2026-04-29 – Find the commit implementing the GPAS
**Asked:** Find the commit implementing the GPAS zip filename naming convention and prepare it for production deployment.
**Done:** Located commit `16bbd98` with the GPAS naming check; clarified deployment path requires server-side git pull and service restart on `box-cli-01`.

### 2026-04-29 – Asked | Done | Log
**Asked:** Asked | Done | Log
**Done:** Add validation checks for linking record top section fields | Implemented format validation for market (3 alphabetic chars), model (dynamic list), year (YYY format), and header fields | LinkingRecord.ts, ValidationRules.ts

### 2026-04-29 – Add validation checks for linking record
**Asked:** Add validation checks for linking record data format fields | Implemented format validation for market (3 letters), year (YYY), and model fields with allowlist checking | LinkingRecord.ts, ValidationSchema.ts
**Done:** Pushed code to git and deployed to dev server | Changes committed, pushed to repository, and deployed to development server | deployment scripts executed

### 2026-04-29 – Add validation checks for linking record
**Asked:** Add validation checks for linking record top section fields (market, model, year, fullLoad, header).
**Done:** Implemented format validation for market (3-letter codes), model (dynamic list), and year (YYY format) with smoke tests confirming module imports and runs without errors.

### 2026-04-17 – Where can we find the report
**Asked:** Where can we find the report and why isn't the previously done task working?
**Done:** Generated report opened in browser using linkingrecord.json and QC results, added Asset Count card showing link and asset counts by section.

### 2026-04-17 – Asked | Done | Log
**Asked:** Asked | Done | Log
**Done:** Feature deployment | Pushed changes and restarted dev service | ford-qc-hotfolder.service

### 2026-04-17 – Where is the report and why
**Asked:** Where is the report and why isn't the implementation working?
**Done:** Updated implementation to display total linking records and unique assets for overall and per-section views by modifying the call site to pass four values.

### 2026-04-17 – Asked | Where can the new
**Asked:** Asked | Where can the new report be found after implementation? | Completed
**Done:** Done | Added two new static methods before `_build_head` and verified changes | Code updated with new static methods

### 2026-04-16 – Changed zip file naming convention from
**Asked:** Changed zip file naming convention from "_image.zip" to "_GPAS.zip" for image pack validation.
**Done:** Updated image pack naming format and added validation check to ensure all packs end with "_GPAS.zip" suffix.

### 2026-04-16 – Update zip file naming convention from
**Asked:** Update zip file naming convention from "_image.zip" to "_GPAS.zip" for image pack validation.
**Done:** Implemented zip filename check to enforce "_GPAS.zip" suffix and deployed to production with proper validation messaging.

### 2026-04-16 – Change image pack naming convention from
**Asked:** Change image pack naming convention from "_image.zip" to "_GPAS.zip" suffix.
**Done:** Updated naming validation logic and pushed changes to git, resolved merge conflict with .gitignore and .env configuration.

### 2026-04-16 – Update image pack naming convention from
**Asked:** Update image pack naming convention from "_image.zip" to "_GPAS.zip" suffix.
**Done:** Modified naming validation logic to enforce "_GPAS.zip" format for image pack files.

### 2026-04-16 – Change image pack naming convention from
**Asked:** Change image pack naming convention from "_image.zip" to "_GPAS.zip" suffix.
**Done:** Updated naming validation to enforce _GPAS.zip suffix for all image packs and committed changes to git.

### 2026-04-16 – Update image pack naming from "_image.zip"
**Asked:** Update image pack naming from "_image.zip" to "_GPAS.zip" suffix for Ford third-party checker compatibility.
**Done:** Modified image pack validation logic and pushed changes to git, but encountered merge conflict in .gitignore during pull.

### 2026-04-16 – Update image pack naming from "_image.zip"
**Asked:** Update image pack naming from "_image.zip" to "_GPAS.zip" suffix requirement.
**Done:** Modified validation logic to enforce _GPAS.zip naming convention and deployed changes via git pull with service restart.

### 2026-04-16 – Update image pack naming from "_image.zip"
**Asked:** Update image pack naming from "_image.zip" to "_GPAS.zip" suffix with validation check.
**Done:** Modified naming convention and validation logic, pushed to Git and restarted the ford-qc-hotfolder service.

### 2026-04-16 – Change image pack naming from "_image.zip"
**Asked:** Change image pack naming from "_image.zip" to "_GPAS.zip" suffix in the validation check.
**Done:** Updated zip file naming validation logic to require "_GPAS.zip" suffix instead of "_image.zip", and pushed changes to git after resolving merge conflicts.

### 2026-04-16 – Change image pack naming from "_image.zip"
**Asked:** Change image pack naming from "_image.zip" to "_GPAS.zip" suffix for Ford QC validation.
**Done:** Updated naming convention and resolved git merge conflict by stashing local changes before pulling updates.

### 2026-04-16 – Update image pack naming convention from
**Asked:** Update image pack naming convention from _image.zip to _GPAS.zip | Changed zip file naming pattern validation and .gitignore file | .gitignore, image_pack_validator.py
**Done:** —

### 2026-04-16 – Update image pack naming validation to
**Asked:** Update image pack naming validation to require "_GPAS.zip" suffix instead of "_image.zip"
**Done:** Created zip filename check module and integrated it into Ford QC validation pipeline

### 2026-04-16 – Update zip file naming requirement from
**Asked:** Update zip file naming requirement from "_image.zip" to "_GPAS.zip" suffix.
**Done:** Created filename validation check, integrated into Ford BNP profile, and verified all test scenarios pass.

### 2026-04-16 – Update image pack naming convention to
**Asked:** Update image pack naming convention to use "_GPAS.zip" suffix instead of "_image.zip".
**Done:** Modified naming validation logic and updated all image pack references to enforce the new "_GPAS.zip" suffix format.

### 2026-04-14 – Project catalogued
**Done:** Added to Obsidian second brain with full details.

---
## Change Log
| Date | Requested | Changed | Files |
|------|-----------|---------|-------|
| 2026-05-01 | QC tool config fix | Restore valid JWT key in ford_box_config.json, align DEV config with PROD | ford_box_config.json, .env |
| 2026-05-01 | Check QC tool server issues and attempt reboot | Verified Box folder configurations for DEV environment and checked PROD .env folder ID settings | Server config, .env |
| 2026-05-01 | QC tool debugging | Box API 400 error, clientID/publicKeyID mismatch between DEV and PROD | ford-qc-hotfolder.service config |
| 2026-05-01 | QC tool investigation | Server connection established, service logs reviewed | box-cli server logs |
| 2026-04-29 | Obsidian project docs enrichment | Added session tracking, deployment details, stack info, URLs, and change history to project notes | project-notes.ts, obsidian-sync.ts, project-enrichment.ts |
| 2026-04-29 | Series image verification | Removed images/links for ambl2se#r1, verified test results, committed and deployed | Image assets, test configuration, deployment config |
| 2026-04-29 | Series image validation | Removed duplicate ambl2se#r1 linking, restarted service | Image linking config, ford-qc-hotfolder.service |
| 2026-04-29 | Series image validation | Added _autodetect_mode() function, orphan detection logic | checks/check_series_permutations.py |
| 2026-04-29 | Series validation check | Added orphan detection for carousel entries, implemented SE#/ABM mode autodetection | checks/check_series_permutations.py |
| 2026-04-29 | Image pack naming validation | Added zip_filename_check to Ford profile, copied check script to prod | checks/zip_filename_check.py, profiles/ford_bnp.json |
| 2026-04-29 | Log | Image pack naming convention | Added _GPAS.zip validation check, reordered validation rules | ford_bnp.json |
| 2026-04-29 | Image pack naming update | Added _GPAS.zip validation check, Updated Ford profile with new validation rule | checks/zip_filename_check.py, profiles/ford_bnp.json |
| 2026-04-29 | Image pack naming | Updated naming format from _image.zip to _GPAS.zip, added validation check | FORD_ASSET_PACK_QC_DEV service |
| 2026-04-29 | GPAS filename validation | Added _GPAS.zip naming check, removed _image.zip legacy support | Image pack validator, deployment config |
| 2026-04-29 | Pushed code to Git and deployed to dev server | Committed 3 files to main branch (5abd481) and restarted ford-qc-hotfolder service on dev with successful git pull | ford_qc_git_dev/ford_qc, service config |
| 2026-04-29 | Linking record validation | market length/alphabet check, year format check, model allowlist validation | LinkingRecord.ts, ValidationSchema.ts |
| 2026-04-29 | Validation checks | Added market/model/year format validation, removed obsolete function | unzip_and_verify_check.py, linking_record_validator.py |
| 2026-04-17 | Report generation and deployment | Generated report with Asset Count card, linked QC results to summary | linkingrecord.json, test_reports, box-cli dev deployment |
| 2026-04-17 | Report location | Code pushed to git and deployed to box-cli dev folder | dev deployment |
| 2026-04-17 | Report display implementation | Import validation, function signature updated to accept four parameters | call site, implementation file |
| 2026-04-17 | New static methods | Added two static methods before _build_head, verified implementation | Source file |
| 2026-04-16 | Image pack naming | Zip suffix changed from _image to _GPAS, validation check added | .gitignore, validation script |
| 2026-04-16 | Zip naming validation | zip_filename_check implementation, validation logic, error messaging | .gitignore, validation module files |
| 2026-04-16 | Zip naming update | Validation logic for _GPAS.zip suffix, .env configuration setup | .gitignore, utils/config.py, .env.example |
| 2026-04-16 | Zip naming convention | Filename suffix validation logic, file pattern matching | validation.py, config.py |
| 2026-04-16 | Zip naming convention | Updated suffix validation from _image.zip to _GPAS.zip | .gitignore, image pack validator |
| 2026-04-16 | Image pack naming | Suffix pattern changed from _image.zip to _GPAS.zip, validation logic updated | .gitignore, validation module files |
| 2026-04-16 | Image pack naming validation | Suffix check from _image.zip to _GPAS.zip | .gitignore, ford_qc validation module |
| 2026-04-16 | Image pack naming | Zip suffix changed from _image.zip to _GPAS.zip, validation check added | .gitignore, ford_qc service configuration |
| 2026-04-16 | Zip naming validation | Suffix changed from _image.zip to _GPAS.zip | Image pack validator, .gitignore, ford_box_config.json |
| 2026-04-16 | Zip naming convention | Suffix changed from _image.zip to _GPAS.zip | .gitignore, ford_box_config.json |
| 2026-04-16 | zip naming validation | New check module creation, profile integration, documentation | checks/zip_filename_check.py, profiles/ford_bnp.json, CLAUDE.md |
| 2026-04-16 | Zip naming validation | New check module creation, profile integration, test validation | checks/zip_filename_check.py, profiles/ford_bnp.json, CLAUDE.md |
| 2026-04-16 | Image pack naming | Zip suffix validation, pack name references | validation.ts, config.ts, constants.ts |
| 2026-03-16 | Configurable pattern | Ranger ptvl pattern via profile JSON | qc_engine.py |

## Related
- [[ford-gechub-sftp/Ford SFTP Transfer]]