Case study
Global Search Worker
An event-driven npm library that keeps denormalized search data in sync without blocking the main application.
Overview
Global Search Worker is an asynchronous search-indexing utility built for applications that need reliable, near-real-time denormalized search data. It captures database changes with MongoDB Change Streams, then hands search updates to a Redis and BullMQ queue so the main application stays responsive.
Technical details
- 01
MongoDB Change Streams for real-time database events
- 02
Redis and BullMQ queue for asynchronous processing
- 03
Batch processing with a batch size of 100
- 04
Dead letter queues and automated retries for reliable delivery
- 05
Designed for low-resource environments and controlled infrastructure cost
Impact
Separates expensive search updates from the request path, improving responsiveness while keeping search data current and reliable.
- Search indexing
- Event-driven systems
- High-throughput backend services