Performance

Advanced performance tuning and scaling tips.

Rate Limiting

The API service includes built-in rate limiting to protect against overload and ensure fair usage across clients. The system uses a simplified policy structure based on operation types and resource intensity.

Rate Limiting Policies

The following rate limiting policies are applied per user/IP address:

Policy	Operation Types	Requests per Second	Queue Limit	Description
Global	All endpoints	1,000	200	Safety net across all APIs
Light Operations	GET single items, list operations	100	50	Low-impact read operations
Medium Operations	Query API, batch reads	50	25	Moderate resource usage
Heavy Operations	Create/Update/Delete operations	20	10	High-impact write operations
Admin Operations	Model management, job operations	50 per minute	25	Administrative functions

Endpoint Classifications

Light Operations: Digital twin GET, relationship GET, list operations
Medium Operations: Query API for graph traversals and searches
Heavy Operations: Digital twin CREATE/UPDATE/DELETE, relationship CREATE/UPDATE/DELETE
Admin Operations: Model management, import job management

Configuration

All settings can be customized through the Parameters section in appsettings.json:

{
  "Parameters": {
    // PostgreSQL Connection Settings
    "ConnectionIdleLifetime": 60,
    "ConnectionLifetime": 300,
    "MaxPoolSize": 100,
    "MinPoolSize": 0,
    "ConnectionTimeout": 15,
    "CommandTimeout": 30,
    
    // Rate Limiting Settings (per second unless noted)
    "GlobalPermitLimit": 1000,
    "GlobalWindowSeconds": 1,
    "LightOperationsPermitLimit": 100,
    "LightOperationsWindowSeconds": 1,
    "MediumOperationsPermitLimit": 50,
    "MediumOperationsWindowSeconds": 1,
    "HeavyOperationsPermitLimit": 20,
    "HeavyOperationsWindowSeconds": 1,
    "AdminOperationsPermitLimit": 50,
    "AdminOperationsWindowMinutes": 1,  // Note: per minute
    
    // Database Protection Settings
    "MaxConcurrentRequestsPerUser": 50,
    "MaxQueryComplexityPerWindow": 20000,
    "BaseQueryComplexity": 1
  }
}

Environment Variable Support

All parameters support environment variable overrides using the pattern Parameters__<SettingName>:

# PostgreSQL settings
Parameters__MaxPoolSize=200
Parameters__ConnectionTimeout=30

# Rate limiting settings  
Parameters__GlobalPermitLimit=2000
Parameters__LightOperationsPermitLimit=200
Parameters__HeavyOperationsPermitLimit=50

Error Response

When rate limits are exceeded, the API returns HTTP 429 with details:

{
  "error": "Rate limit exceeded",
  "message": "Too many requests. Please try again later.",
  "retryAfterSeconds": 1.0
}

PostgreSQL Performance Tuning

For optimal performance, especially when using Apache AGE with digital twins data, proper PostgreSQL configuration is crucial.

Cluster Configuration

When deploying PostgreSQL in a cluster (e.g., using CloudNativePG), consider these resource and storage configurations:

cluster:
  instances: 3
  resources:
    limits:
      cpu: "2"
      memory: 3Gi
    requests:
      cpu: 50m
      memory: 1000Mi
  storage:
    size: 64Gi
    pvcTemplate:
      storageClassName: managed-csi-premium
  walStorage:
    enabled: true
    size: 64Gi
    pvcTemplate:
      storageClassName: managed-csi-premium

PostgreSQL Parameters

Key parameters for performance optimization:

postgresql:
  parameters:
    # Connection Management
    max_connections: '200'          # Adjust based on expected concurrent users
    
    # Memory Configuration
    shared_buffers: 512MB           # 25% of available RAM for caching
    
    # WAL (Write-Ahead Logging) Configuration
    wal_compression: lz4            # Compress WAL files to save space
    max_wal_size: 2GB              # Maximum WAL size before checkpoint
    min_wal_size: 512MB            # Minimum WAL size to keep
    wal_keep_size: 512MB           # WAL files to keep for replication
    max_slot_wal_keep_size: 1GB    # WAL to keep for replication slots
    
    # Checkpoint Configuration
    checkpoint_flush_after: 2MB     # Flush dirty pages incrementally
    wal_writer_flush_after: 2MB     # WAL writer flush frequency
    checkpoint_completion_target: "0.9"  # Spread checkpoint I/O
    checkpoint_timeout: 5min        # Maximum time between checkpoints

Performance Recommendations

Memory Settings

shared_buffers: Set to 25% of available RAM (512MB for 2GB systems)
effective_cache_size: Set to 75% of total system memory
work_mem: Start with 4MB, increase for complex queries

Connection Pool Settings

max_connections: Match your application's connection pool size
Connection pooling: Use PgBouncer or similar for high-concurrency scenarios

WAL and Checkpointing

wal_compression: Use lz4 for good compression with minimal CPU overhead
checkpoint_completion_target: Set to 0.9 to spread checkpoint I/O over time
max_wal_size: Increase for write-heavy workloads to reduce checkpoint frequency

Storage Considerations

Use SSD storage for both data and WAL files
Separate WAL storage for better I/O performance
Use premium storage classes in cloud environments

Monitoring and Optimization

Monitor these key metrics:

Connection usage and pool efficiency
Buffer hit ratio (target: >95%)
Checkpoint frequency and duration
WAL generation rate
Query performance and slow query logs

Apache AGE Specific Tuning

For graph workloads with Apache AGE:

Increase work_mem for complex graph traversals
Monitor memory usage during large graph operations
Consider partitioning large graph datasets
Use appropriate indexes on graph node and edge properties