AlertHandler
Publisher Platform Alert Processing Microservice
Service Type: Kubernetes Microservice Technology: Node.js (JavaScript) Runtime: Node.js Container Last Updated: 2025-07-01
π Overview
The AlertHandler microservice is a critical component of the Publisher platform that processes system events and triggers appropriate alerts based on configured rules. It acts as an intelligent notification system that monitors various system events and sends alerts via multiple channels (Email, HTTP, Slack) when specific conditions are met.
Key Features
- Event-Driven Architecture: Processes system events from Event Hub
- Multi-Channel Alerting: Supports Email, HTTP webhooks, and Slack notifications
- Rule-Based Matching: Intelligent alert matching based on system events
- Document Cache Integration: Retrieves alert configurations from document cache
- Batch Processing: Handles multiple events concurrently with configurable limits
- Error Handling: Robust error handling with retry mechanisms
π Technology Stack
| Component | Technology |
|---|---|
| Language | JavaScript (Node.js) |
| Runtime | Node.js Container |
| Framework | Express.js / HTTP Server |
| Database | Azure Cosmos DB |
| Messaging | Azure Event Hub |
| Build Tool | Webpack |
| Container | Docker |
| Orchestration | Kubernetes |
π Architecture
Service Architecture
- Deployment Pattern: Kubernetes Deployment with HTTP Service
- Event Source: Azure Event Hub (
publishersystemevents) - Consumer Group:
alerthandler - Output Binding: Event Hub for results
- Configuration: ConfigMap and environment variables
- Monitoring: Application Insights integration
System Flow
graph TD
A[Event Hub: publishersystemevents] --> B[AlertHandler Microservice]
B --> C[Message Validation]
C --> D[Document Cache Lookup]
D --> E[Alert Matching Engine]
E --> F{Matching Alerts Found?}
F -->|No| G[Stop Processing]
F -->|Yes| H[Prepare Alert Context]
H --> I{Alert Type}
I -->|Email| J[Email Alert Processor]
I -->|HTTP| K[HTTP Webhook Processor]
I -->|Slack| L[Slack Alert Processor]
J --> M[Send Email via AWS SES]
K --> N[Send HTTP Request]
L --> O[Send Slack Message]
M --> P[Event Hub: Results]
N --> P
O --> P
P --> Q[Self Healer on Error]
π Processing Pipeline
The AlertHandler follows a sophisticated 3-stage pipeline architecture:
Stage 1: Message Handling
graph LR
A[S1InitialSetup] --> B[S2GetDocumentCache]
B --> C[S3CheckForMatchingAlert]
C --> D[S4MatchPaths]
Stage 2: Context Preparation
graph LR
A[Alert Context] --> B{Alert Type}
B -->|Email| C[Email Context Prep]
B -->|HTTP| D[HTTP Context Prep]
B -->|Slack| E[Slack Context Prep]
Stage 3: Alert Delivery
graph LR
A[Prepared Context] --> B{Delivery Channel}
B -->|Email| C[AWS SES Integration]
B -->|HTTP| D[HTTP Webhook Call]
B -->|Slack| E[Slack API Integration]
βββ src/Helpers/
βββ src/SPipeline/
βββ src/SPipeline/S1HandleMessage/
βββ src/SPipeline/S2PrepareContext/
βββ src/SPipeline/S2PrepareContext/SType_Email/
βββ src/SPipeline/S2PrepareContext/SType_HTTP/
βββ ...
## π‘ API Specification
### HTTP API Endpoint
- **Method**: POST
- **Authentication**: Internal service authentication
- **Content-Type**: application/json
- **Port**: 8080 (configurable)
### Request Format
```json
{
"partitionKey": "string",
"systemEvents": ["EventType1", "EventType2"],
"additionalData": {
// Event-specific payload
}
}
Response Format
{
"processedMsgs": [
{
"status": "SUCCESS",
"response": {},
"errors": []
}
],
"failedEvents": []
}
π§ Business Logic
Alert Types
- Admin Alerts: Triggered by
SourceAddedsystem events - Campaign Alerts: Triggered by campaign-related system events
Alert Matching Logic
- Event Validation: Validates incoming system events array
- Partition Key Check: Ensures valid partition key for document lookup
- Document Cache Lookup: Retrieves alert configurations from cache
- Event Matching: Matches incoming events with configured alert rules
- Alert Preparation: Prepares alert context based on alert type and delivery method
Processing Flow
sequenceDiagram
participant EH as Event Hub
participant AH as AlertHandler
participant DC as Document Cache
participant AS as Alert Sender
participant OUT as Output Channel
EH->>AH: System Events Batch
AH->>AH: Validate Request
AH->>DC: Get Alert Configurations
DC-->>AH: Alert Rules
AH->>AH: Match Events to Alerts
AH->>AS: Prepare Alert Context
AS->>OUT: Send Alert (Email/HTTP/Slack)
OUT-->>AS: Delivery Confirmation
AS-->>AH: Processing Result
AH->>EH: Results/Errors
βΈοΈ Kubernetes Deployment
Kubernetes Resources
- Deployment: β Configured
- Service: β Configured
- ConfigMap: β Configured
- Secrets: β Configured
π¦ Dependencies
Main Dependencies
@azure/cosmosasyncaws-sdkdebugdirectory-treeflatglobidgenmomentpathtimer-nodeunflatten
Installation
npm install
βοΈ Configuration
Environment Variables
No environment variables detected.
π Build & Deploy
Local Development
# Clone the repository
git clone <repository-url>
cd AlertHandler
# Install dependencies
npm install
# Run locally
npm start
π‘ API Documentation
β οΈ No API documentation found.
Consider adding Swagger/OpenAPI documentation for better API discoverability.
ποΈ Database
No database configuration detected.
π Monitoring & Logging
Health Checks
# Health check endpoint
curl http://localhost:8080/health
# Readiness check
curl http://localhost:8080/ready
# Liveness check
curl http://localhost:8080/alive
Metrics
- Prometheus metrics: Check
/metricsendpoint - Application metrics: Monitor via Kubernetes dashboard
Logging
- Log Level: Configurable via environment variables
- Log Format: JSON structured logging recommended
- Log Aggregation: Logs are collected by Kubernetes logging infrastructure
π§ Troubleshooting
Common Issues
Service Not Starting
# Check pod status
kubectl get pods -l app=alerthandler
# Check pod logs
kubectl logs <pod-name> -f
# Describe pod for events
kubectl describe pod <pod-name>
Configuration Issues
- Verify environment variables are set correctly
- Check ConfigMap and Secret configurations
- Ensure database connectivity
Performance Issues
- Monitor resource usage:
kubectl top pods - Check application metrics at
/metricsendpoint - Review application logs for errors
π€ Contributing
Development Workflow
- Create feature branch from
main - Make changes and test locally
- Run tests:
npm test - Build and test Docker image
- Create pull request
Code Standards
- Follow established coding conventions
- Add appropriate tests for new features
- Update documentation as needed
- Ensure Kubernetes manifests are valid
Testing
# Run unit tests
npm test
# Run integration tests
npm run test:integration
This documentation was auto-generated based on codebase analysis. Please review and update as needed.