-
released this
2025-12-18 00:54:57 +10:00 | 8 commits to main since this releaseMajor Architectural Refactoring
This release introduces a complete refactoring of the scraping system to improve maintainability, configurability, and extensibility.
Key Changes
New Architecture:
- Workflow-based scraping system replacing monolithic service
- Discrete, testable scraping steps organized by category
- Configuration-driven selectors, JavaScript templates, and text patterns
- Step registry and workflow factory for extensibility
- PageState tracking and step dependencies
- Generic IWorkflow interface supporting different response types
New Components:
- Scraping steps: Navigation, Search, Map, Metadata, Capture
- BaseScrapingStep base class with common dependencies
- ScrapingContext for shared state between steps
- RadarScrapingWorkflow orchestrating step execution
- SelectorService for configurable element selection
- Configuration models for externalized settings
Performance & Monitoring:
- Step-level timing metrics with historical tracking
- Automatic warnings for slow steps/workflows (50%+ slower than average)
- Integrated metrics with CacheService
Configuration:
- All selectors externalized to appsettings.json
- JavaScript templates externalized to appsettings.json
- Text patterns externalized to appsettings.json
- Docker configuration support via mounted appsettings.json
- Reduced hardcoded waits, using configurable timeouts
Bug Fixes:
- Fixed DebugService collection modification exception
- Fixed ResetToFirstFrameStep click interception with JavaScript fallback
- Removed deprecated minutes ago fallback, using timestamp parsing only
- Updated timestamp parsing for new BOM website format
Documentation:
- Updated README with architecture details
- Added configuration section with Docker guidance
- Added development guidelines for extending scraping system
- Added performance monitoring documentation
Downloads
-
released this
2025-12-10 10:46:10 +10:00 | 15 commits to main since this releaseChanges
- Add automatic monitoring and restart of Xvfb if it crashes
- Move startup script to scripts/docker-start.sh for better organization
- Improve error handling and cleanup on container shutdown
- Prevents cache update failures when Xvfb process dies
Technical Details
The startup script now includes a background monitor that checks every 5 seconds if Xvfb is running and automatically restarts it if it crashes. This prevents the 'Missing X server or $DISPLAY' errors that were causing cache update failures.
Downloads
-
Version 0.0.2 Stable
released this
2025-12-09 03:16:34 +10:00 | 16 commits to main since this releaseAdd multi-architecture support (amd64 and arm64)
Downloads
-
Version 0.0.1 Stable
released this
2025-12-09 02:33:01 +10:00 | 17 commits to main since this releaseInitial release with Docker image publishing workflow
Downloads
mirror of
https://github.com/alexhopeoconnor/bom-local-service.git
synced 2026-10-03 21:21:59 +10:00