Oliver/solventum-image-metadata
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
9 commits 2 branches 0 tags 780 KiB
Commit graph

6 commits

Author SHA1 Message Date
SamoilenkoVadym
03079080d8 Phase 2.4: Metadata import from external files (CSV, Excel, JSON) 
Created comprehensive metadata_importer.py module:
- CSV import with multiple encoding support (UTF-8, Latin1, ISO-8859-1, CP1252)
- Excel import (.xlsx, .xls) with sheet selection
- JSON import (object and array formats)
- Intelligent column detection for filename, title, subject, keywords
- Fuzzy column matching (case-insensitive, multiple aliases)
- Metadata normalization to standard format
- Import validation with statistics
- File lookup by filename stem (case-insensitive)

Web interface enhancements:
- /import-metadata endpoint for file uploads
- Import section UI (appears when Import source selected)
- Real-time import statistics display (records, title/subject/keywords counts)
- Import session management with unique session IDs
- Visual feedback (active state, success/error messages)
- Validation: requires import file before processing with import source

Import workflow:
1. User selects "Import from File" metadata source
2. Import section appears with file chooser
3. User uploads CSV/Excel/JSON with metadata
4. System validates and shows statistics
5. User uploads files to process
6. System matches files to imported metadata by filename

Supported import formats:
- CSV: filename, title, subject/description, keywords columns
- Excel: Any sheet with filename and metadata columns
- JSON: {filename: {metadata}} or [{filename, metadata}] formats

Technical features:
- Pandas DataFrame parsing for CSV/Excel
- Flexible column name detection (10+ aliases per field)
- NaN/null value handling
- List/array keyword support
- Unicode filename support

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 15:39:27 +00:00
SamoilenkoVadym
1bf2483f2d Phase 2.3: AI metadata generation with production-ready features 
Enhanced metadata_analyzer.py with production-ready capabilities:
- Token counting with tiktoken for accurate OpenAI usage tracking
- Exponential backoff retry logic with tenacity library
- Intelligent content truncation based on token limits (not characters)
- Configurable timeout and max retries from Config
- Graceful fallback when tiktoken/tenacity unavailable
- Enhanced error reporting with _ai_error and _tokens_used metadata

Integrated AI generation in web interface:
- AI analyzer lazy initialization in web_app.py
- Real content extraction and AI analysis in upload endpoint
- Error handling for insufficient content or API failures
- Token usage logging for monitoring and optimization

UI improvements for AI experience:
- Special loading message for AI processing (10-30s per file)
- Display token usage for AI-generated metadata
- Show AI errors prominently with helpful messages
- Filter internal metadata fields (_tokens_used, _ai_error) from forms

Dependencies leveraged:
- tiktoken: Proper OpenAI token counting (10x more accurate)
- tenacity: Exponential backoff retry (3 attempts, 2-10s delays)
- openai: Production timeout support (30s default)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 15:36:48 +00:00
SamoilenkoVadym
fa2b4da2f7 Phase 2.1 & 2.2: Manual metadata editing and multiple sources 
Implemented manual metadata editing UI:
- Added editable input fields for title (200 chars), subject (300 chars), keywords (500 chars)
- Character counters with warning/danger indicators at 90%/100%
- Real-time validation with visual feedback
- Save and Reset buttons for each file
- Individual file metadata updates via /update-manual endpoint

Implemented multiple metadata sources:
- Added metadata source selector dropdown (Excel, Manual, AI, Import)
- Modified /upload endpoint to handle different metadata sources
- Excel lookup: existing functionality (fastest)
- Manual entry: empty fields for user input
- AI generation: placeholder for Phase 2.3
- Import: placeholder for Phase 2.4

Technical improvements:
- Session-based metadata storage for persistence
- Graceful success/error feedback with visual indicators
- Sanitized metadata input with length limits
- Backup creation before updates

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 15:34:05 +00:00
SamoilenkoVadym
f4e1017964 Phase 1.3: Improve startup error handling and dependency checks 
Added comprehensive startup checks in web_app.py:
- Check for Excel file existence with helpful error message
- Validate OpenAI API key availability (optional)
- Check ExifTool installation (optional)
- Display available metadata sources based on configuration
- Updated branding in startup messages

Benefits:
- Users see clear error messages for missing dependencies
- Easy troubleshooting of configuration issues
- Graceful degradation when optional features unavailable

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 15:17:03 +00:00
SamoilenkoVadym
7db62e06da Phase 1.1: Rebrand to Oliver Metadata Tool v3.0 
- Updated application name to "Oliver Metadata Tool"
- Updated version to 3.0.0
- Added App Info constants to config.py (APP_NAME, APP_VERSION, APP_DESCRIPTION)
- Updated web interface (title, header, footer)
- Updated README with new branding and description
- Added AI configuration settings to config.py
- Added ExifTool check method to config.py

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 15:15:26 +00:00
SamoilenkoVadym
2082ea7ce7 Initial commit: Universal metadata tool with Excel-based lookup 
- Added Flask web interface for batch metadata processing
- Added Excel-based metadata lookup (Celum ID mapping)
- Dual-sheet support: DSB (primary) and Medsurg (fallback)
- Unicode/hieroglyph support for CGA region (Chinese, Japanese, Korean)
- Multi-format support: PDF, images, Office docs, video
- OCR with multi-language support (Tesseract)
- Filename matching without extension (case-insensitive)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
 2026-01-25 14:23:42 +00:00