Skip to content

DocumentCRUD

Publisher Platform Document Management Microservice

Service Type: Kubernetes Microservice
Technology: Node.js (JavaScript) with Fastify
Runtime: Node.js Container
Last Updated: 2025-07-01

📋 Overview

The DocumentCRUD microservice is the central document management system for the Publisher platform. It provides comprehensive CRUD (Create, Read, Update, Delete) operations for all platform documents including campaigns, vendors, controls, and configurations. The service features role-based access control, multi-tenant organization support, and real-time document change tracking with audit integration.

Key Features

  • Universal CRUD Operations: Standardized Create, Read, Update, Delete for all document types
  • Role-Based Access Control: JWT-based authentication with granular permissions
  • Multi-Tenant Support: Organization-based document isolation and access control
  • Bulk Operations: Efficient batch processing for large document sets
  • Change Tracking: Automatic audit trail generation for all document modifications
  • Schema Validation: Configurable document validation and constraints
  • Event Integration: Document change events published to Event Hub

🛠 Technology Stack

Component Technology
Language JavaScript (Node.js)
Runtime Node.js Container
Framework Fastify (with Express.js compatibility)
Database Azure Cosmos DB
Authentication JWT with role-based access
Validation JSON Schema validation
Build Tool Webpack
Container Docker
Orchestration Kubernetes

🏗 Architecture

Service Architecture

  • Deployment Pattern: Kubernetes Deployment with HTTP Service
  • Port: 8080 (configurable)
  • Database: Azure Cosmos DB with partition key support
  • Authentication: JWT-based with configurable secret
  • Authorization: Role and organization-based access control
  • Event Publishing: Document change events to Event Hub

System Flow

graph TD
    A[Client Request] --> B[DocumentCRUD Service]
    B --> C[JWT Authentication]
    C --> D[Role Authorization]
    D --> E[Request Validation]
    E --> F[Cosmos DB Operation]
    F --> G[Change Event Generation]
    G --> H[Event Hub Publishing]
    H --> I[Response Generation]
    I --> A

    F --> J[Audit Trail]
    J --> K[AuditHandler Service]

🔄 CRUD Operations Flow

Document Lifecycle Management

graph LR
    A[Create] --> B[Validate Schema]
    B --> C[Check Permissions]
    C --> D[Generate ID]
    D --> E[Save to Cosmos DB]
    E --> F[Publish Change Event]

Multi-Operation Processing

sequenceDiagram
    participant Client
    participant CRUD as DocumentCRUD
    participant Auth as Authentication
    participant DB as Cosmos DB
    participant EventHub as Event Hub

    Client->>CRUD: CRUD Request
    CRUD->>Auth: Validate JWT Token
    Auth-->>CRUD: User Claims & Roles
    CRUD->>CRUD: Check Permissions
    CRUD->>DB: Execute Operation
    DB-->>CRUD: Operation Result
    CRUD->>EventHub: Publish Change Event
    CRUD-->>Client: Response

📡 API Specification

Authentication

All endpoints require JWT authentication via Authorization: Bearer <token> header.

Standard CRUD Endpoints

Create Document

  • Method: POST
  • Endpoint: /{collection}
  • Authorization: Create permissions for collection
{
  "data": {
    "friendlyName": "New Campaign",
    "organizationId": "vendor-123",
    "campaignId": "campaign-456"
  }
}

Read Document

  • Method: GET
  • Endpoint: /{collection}/{id}
  • Authorization: Read permissions for collection

Update Document

  • Method: PUT
  • Endpoint: /{collection}/{id}
  • Authorization: Update permissions for collection
{
  "data": {
    "friendlyName": "Updated Campaign",
    "status": "active"
  }
}

Delete Document

  • Method: DELETE
  • Endpoint: /{collection}/{id}
  • Authorization: Delete permissions for collection

List Documents

  • Method: GET
  • Endpoint: /{collection}
  • Query Parameters: Filtering and pagination support

Bulk Operations

Bulk Create/Update

  • Method: POST
  • Endpoint: /{collection}/bulk
  • Authorization: Bulk permissions for collection
{
  "operations": [
    {
      "operationType": "Create",
      "resourceBody": { "data": {...} }
    },
    {
      "operationType": "Upsert", 
      "resourceBody": { "data": {...} }
    }
  ]
}

Specialized Endpoints

Content Management

  • Endpoint: /content/*
  • Description: Content-specific CRUD operations

Hosted SFTP

  • Endpoint: /hostedSFTP/*
  • Description: SFTP configuration management

🔧 Business Logic

Document Collections

The service manages multiple document types: - campaigns: Campaign configurations and settings - vendors: Vendor/organization information - controls: Traffic control and throttling rules - sources: Data source configurations - content: Content management documents - hostedSFTP: SFTP server configurations

Partition Key Strategy

Documents are partitioned using configurable partition keys: 

// Example partition key configuration
partitionKey: ["organizationId", "campaignId"]

Role-Based Access Control

  • Organization Isolation: Users can only access documents within their organization
  • Role Permissions: Different roles have different CRUD permissions
  • Collection-Specific Access: Permissions can be granted per document collection

Change Event Generation

All document modifications generate change events: 

{
  "recordid": "unique-event-id",
  "ts": 1640995200000,
  "collection": "campaigns",
  "action": "update",
  "organizationId": "vendor-123",
  "auth": {
    "name": "user@example.com"
  },
  "record": { /* new document state */ },
  "oldRecord": { /* previous document state */ }
}

Validation & Constraints

  • Schema Validation: JSON schema validation for document structure
  • Business Rules: Custom validation rules per document type
  • Referential Integrity: Cross-document relationship validation

⚙️ Configuration

Environment Variables

  • JWT Configuration: Secret keys and token validation settings
  • Cosmos DB: Account, database, and connection settings
  • Authentication: Enable/disable authentication for development
  • Server Settings: Port, bind address, HTTP/2 support

Key Configuration Files

  • config/config.js: Main configuration
  • config/config.int.js: Integration environment
  • config/config.prod.js: Production environment

Service Configuration

{
  "port": 8080,
  "bindAddress": "0.0.0.0",
  "http2": false,
  "disableAuthentication": false,
  "cosmosdbAccount": "publisherdb",
  "cosmosdbDatabase": "publishers",
  "jwtSecret": "jwt-signing-secret"
}

🚀 Deployment

Kubernetes Deployment

# Apply Kubernetes manifests
kubectl apply -f k8s/

# Check deployment status
kubectl get pods -l app=documentcrud

# View logs
kubectl logs -l app=documentcrud -f

Local Development

# Install dependencies
npm install

# Run with authentication disabled
node src/index.js --disable-authentication

# Run with custom configuration
node src/index.js --port 3000 --jwt-secret mysecret

📊 Monitoring & Health Checks

Health Endpoints

  • Liveness: /live - Basic health check
  • Readiness: /ready - Service readiness check

Key Metrics to Monitor

  • CRUD operation success rates
  • Authentication/authorization failures
  • Cosmos DB operation latency
  • Document validation errors
  • Bulk operation performance
  • Event publishing success rates

Performance Metrics

  • Throughput: Operations per second by type
  • Latency: Response time percentiles
  • Error Rates: Failed operations by collection
  • Resource Usage: Memory and CPU utilization

🔧 Troubleshooting

Common Issues

Authentication Failures

  1. Check JWT token validity and expiration
  2. Verify JWT secret configuration
  3. Validate user roles and permissions
  4. Check organization membership

Database Connection Issues

# Check Cosmos DB connectivity
# Verify account and database configuration
# Monitor connection pool status

Permission Denied Errors

  1. Verify user has required role for operation
  2. Check organization-based access restrictions
  3. Validate collection-specific permissions
  4. Review JWT token claims

Validation Errors

  • Schema Validation: Check document structure against schema
  • Required Fields: Ensure all required fields are provided
  • Data Types: Validate field types and formats
  • Business Rules: Check custom validation rules

🧪 Testing

Unit Tests

npm test

Manual Testing

# Test authentication
curl -H "Authorization: Bearer <jwt-token>" \
  "http://localhost:8080/campaigns"

# Test document creation
curl -X POST -H "Content-Type: application/json" \
  -H "Authorization: Bearer <jwt-token>" \
  -d '{"data":{"friendlyName":"Test Campaign"}}' \
  "http://localhost:8080/campaigns"

# Test bulk operations
curl -X POST -H "Content-Type: application/json" \
  -H "Authorization: Bearer <jwt-token>" \
  -d '{"operations":[{"operationType":"Create","resourceBody":{"data":{}}}]}' \
  "http://localhost:8080/campaigns/bulk"

🤝 Dependencies

Internal Dependencies

  • AuthExchange Service: JWT token validation and user management
  • AuditHandler Service: Document change audit trail
  • Event Hub: Document change event publishing

External Dependencies

  • Azure Cosmos DB: Primary document storage
  • JWT Libraries: Token validation and claims extraction

NPM Dependencies

  • fastify: Web framework
  • @azure/cosmos: Cosmos DB client
  • jsonwebtoken: JWT handling
  • lodash: Utility functions
  • cors: Cross-origin resource sharing
  • http-errors: HTTP error handling

This documentation was created through manual code analysis of the DocumentCRUD microservice codebase.