Resource Management Guide

This comprehensive guide covers everything you need to know about managing resources in KtrlPlane, from initial deployment to optimization and troubleshooting.

Creating Resources

Step 1: Resource Planning

Before creating a resource, consider:

Purpose and Requirements

• What functionality do you need?
• What are your performance requirements?
• Do you need real-time processing or batch operations?
• What are your data storage requirements?

Integration Needs

• Will this resource integrate with other resources in your project?
• Do you need external API connections?
• Are there specific data formats or protocols required?

Scale and Growth

• What's your expected initial usage?
• How might usage grow over time?
• Do you need auto-scaling capabilities?

Step 2: Resource Selection

Choose the appropriate resource type:

Konnektr.Graph

Choose when you need:

• Connected data storage and querying
• Relationship mapping and analysis
• Digital twin data models
• Complex data relationships

Konnektr.Flow

Choose when you need:

• Real-time event processing
• Data transformation and routing
• Workflow automation
• Stream processing

Konnektr.Assembler

Choose when you need:

• AI-powered model generation
• Automated data mapping
• DTDL model creation
• Intelligent schema inference

Step 3: Tier Selection

Select the appropriate tier based on your needs:

Step 4: Configuration

Configure your resource settings based on your requirements:

Graph Configuration Example

{
  "database_name": "production-twins",
  "enable_analytics": true,
  "backup_retention_days": 30,
  "query_timeout_seconds": 60,
  "max_concurrent_connections": 200,
  "enable_audit_logging": true,
  "performance_tier": "standard",
  "auto_scaling": {
    "enabled": true,
    "min_capacity": 2,
    "max_capacity": 10
  }
}

Flow Configuration Example

{
  "max_concurrent_flows": 25,
  "retention_days": 90,
  "enable_dead_letter_queue": true,
  "processing_mode": "streaming",
  "error_handling": "retry_with_backoff",
  "auto_scaling": {
    "enabled": true,
    "min_instances": 2,
    "max_instances": 20,
    "target_cpu_utilization": 70
  }
}

Assembler Configuration Example

{
  "model_complexity": "standard",
  "output_format": "dtdl_v2",
  "confidence_threshold": 0.8
}

Step 5: Deployment

1. Review your configuration
2. Click "Create Resource"
3. Monitor the deployment progress
4. Wait for the resource to become available (typically 2-5 minutes)

Managing Existing Resources

Resource Dashboard

Your resource dashboard provides:

Status Overview

• Current operational status
• Health and performance metrics
• Recent activity and events
• Resource utilization

Quick Actions

• Access resource endpoints
• View configuration
• Update settings
• Scale resource
• Access logs

Updating Resource Configuration

Most settings can be updated without recreating the resource:

1. Navigate to your resource
2. Click "Settings"
3. Modify the configuration
4. Click "Save Changes"
5. Monitor the update process

Scaling Resources

Manual Scaling

Adjust resource capacity based on current needs:

1. Go to resource settings
2. Find the scaling section
3. Adjust capacity limits or performance tiers
4. Apply changes

Auto-Scaling (Future Feature)

Automatic scaling based on resource metrics is planned for future releases. Currently, you can manually adjust resource configuration through the dashboard or API.

Resource Integration

Inter-Resource Communication

Resources within the same project can communicate using internal service discovery:

// Graph to Flow integration
const flowClient = new FlowClient({
  endpoint: process.env.FLOW_INTERNAL_ENDPOINT,
  authentication: 'service_account'
});

// Assembler to Graph integration  
const graphClient = new GraphClient({
  endpoint: process.env.GRAPH_INTERNAL_ENDPOINT,
  authentication: 'service_account'
});

External Integrations

Connect your resources to external services:

API Integration

// Configure external API endpoint in resource settings
{
  "external_apis": {
    "weather_service": {
      "endpoint": "https://api.weather.com/v1",
      "authentication": "api_key",
      "rate_limit": 1000
    }
  }
}

Webhook Configuration

{
  "webhooks": {
    "incoming": {
      "endpoint": "/webhooks/data-ingestion",
      "authentication": "hmac_sha256",
      "payload_validation": true
    },
    "outgoing": {
      "alerts": "https://your-system.com/alerts",
      "events": "https://your-system.com/events"
    }
  }
}

Security and Access Control

API Key Management

Each resource provides API keys for programmatic access:

1. Navigate to resource settings
2. Go to "API Keys" section
3. Click "Generate New Key"
4. Configure key permissions and expiration
5. Copy the key (it won't be shown again)

Resource-Level Permissions

Control who can access and modify resources:

• Viewer: Read-only access to resource data and configuration
• Editor: Can modify configuration and manage the resource
• Admin: Full access including security settings and key management

Network Security

Configure network access controls:

{
  "network_security": {
    "ip_whitelist": [
      "192.168.1.0/24",
      "10.0.0.0/8"
    ],
    "enable_private_endpoints": true,
    "require_tls": true,
    "min_tls_version": "1.2"
  }
}

Monitoring and Observability

Built-in Monitoring

KtrlPlane provides comprehensive monitoring for all resources:

Performance Metrics

• Response time percentiles (p50, p95, p99)
• Request throughput and error rates
• Resource utilization (CPU, memory, storage)
• Custom business metrics

Operational Metrics

• Service availability and uptime
• Deployment and configuration changes
• Error patterns and trends
• Cost and billing information

Custom Monitoring

Add custom metrics to your resources:

// Graph resource custom metrics
const metrics = {
  nodes_created_per_minute: nodeCreationRate,
  query_complexity_average: averageQueryComplexity,
  relationship_depth_max: maxRelationshipDepth
};

await graphResource.recordMetrics(metrics);

Alerting

Set up alerts for important events:

1. Go to resource monitoring
2. Click "Create Alert"
3. Choose alert conditions:
   • Performance thresholds
   • Error rate limits
   • Resource utilization
   • Cost thresholds
4. Configure notification channels (email, Slack, webhooks)
5. Save alert configuration

Troubleshooting

Common Issues and Solutions

High Response Times

Symptoms: Slow API responses, timeouts Solutions:

• Scale up resource capacity
• Optimize queries or processing logic
• Enable caching
• Check for resource bottlenecks

Resource Unavailability

Symptoms: Service unreachable, connection errors Solutions:

• Check resource status in dashboard
• Review recent configuration changes
• Verify network connectivity
• Check API key validity

High Costs

Symptoms: Unexpected billing increases Solutions:

• Review resource utilization metrics
• Optimize resource configuration
• Implement auto-scaling policies
• Consider tier adjustments

Diagnostic Tools

Resource Logs

Access detailed logs for troubleshooting:

# Using KtrlPlane CLI
ktrlplane logs --resource-id res-abc123 --follow

# Via API
curl -H "Authorization: Bearer $API_KEY" \
  https://ktrlplane.konnektr.io/v1/resources/res-abc123/logs

Health Checks

Monitor resource health:

{
  "health_check": {
    "endpoint": "/health",
    "interval_seconds": 30,
    "timeout_seconds": 10,
    "healthy_threshold": 2,
    "unhealthy_threshold": 3
  }
}

Best Practices

Configuration Management

• Use version control: Track configuration changes
• Environment consistency: Use similar configs across environments
• Gradual rollouts: Test configuration changes in staging first
• Backup configurations: Keep copies of working configurations

Performance Optimization

• Right-sizing: Choose appropriate resource sizes
• Caching strategies: Implement appropriate caching
• Connection pooling: Use connection pools for database resources
• Batch processing: Use batch operations where possible

Security Practices

• Rotate API keys: Regularly rotate authentication keys
• Principle of least privilege: Grant minimum required permissions
• Network segmentation: Use private networks where possible
• Audit logging: Enable comprehensive audit logs

Cost Management

• Resource monitoring: Regularly review resource usage
• Auto-scaling: Use auto-scaling to optimize costs
• Cleanup unused resources: Remove resources no longer needed
• Tier optimization: Regularly review and optimize tier selections

Advanced Features

Blue-Green Deployments

For critical resources, use blue-green deployment strategies:

1. Create a new resource with updated configuration
2. Test the new resource thoroughly
3. Gradually shift traffic from old to new resource
4. Remove the old resource once migration is complete

Multi-Region Deployment

Deploy resources across multiple regions for high availability:

{
  "deployment": {
    "strategy": "multi_region",
    "regions": ["us-east-1", "eu-west-1", "asia-southeast-1"],
    "replication": {
      "mode": "active_active",
      "consistency": "eventual"
    }
  }
}

Disaster Recovery

Implement disaster recovery for critical resources:

{
  "disaster_recovery": {
    "backup_frequency": "daily",
    "backup_retention": "30_days",
    "cross_region_backup": true,
    "automated_failover": {
      "enabled": true,
      "rto_minutes": 15,
      "rpo_minutes": 5
    }
  }
}

Next Steps

Now that you understand resource management:

1. Explore API Integration: Learn how to integrate with your applications
2. Set Up Monitoring: Implement comprehensive monitoring
3. Optimize Costs: Learn advanced cost optimization strategies
4. Security Hardening: Implement enterprise security practices