Configuration Reference
Complete reference for Laminar configuration.
Configuration Methods
For local development, Laminar uses a TOML configuration file:
cargo run -- --config ./laminar.tomlLocal Configuration (laminar.toml)
Complete Example
# Checkpoint configuration
checkpoint-url = "/tmp/laminar/checkpoints"
default-checkpoint-interval = "1s"
# Pipeline settings
[pipeline]
source-batch-size = 512
source-batch-linger = "100ms"
# API server
[api]
bind-address = "0.0.0.0"
http-port = 8000
# Controller
[controller]
rpc-port = 8001
scheduler = "process" # "process" for local, "kubernetes" for k8s
# Compiler
[compiler]
rpc-port = 8002
# Worker
[worker]
task-slots = 16
# Database
[database.rocksdb]
create-if-missing = trueConfiguration Options
| Option | Default | Description |
|---|---|---|
checkpoint-url | /tmp/laminar/checkpoints | Path for checkpoint storage |
default-checkpoint-interval | 1s | How often to checkpoint state |
Pipeline
| Option | Default | Description |
|---|---|---|
source-batch-size | 512 | Number of messages per batch |
source-batch-linger | 100ms | Max time to wait for batch |
API
| Option | Default | Description |
|---|---|---|
bind-address | 0.0.0.0 | Address to bind to |
http-port | 8000 | HTTP API port |
Controller
| Option | Default | Description |
|---|---|---|
rpc-port | 8001 | gRPC port for controller |
scheduler | process | Scheduler type: process or kubernetes |
Compiler
| Option | Default | Description |
|---|---|---|
rpc-port | 8002 | gRPC port for compiler |
Worker
| Option | Default | Description |
|---|---|---|
task-slots | 16 | Number of task slots per worker |
Kubernetes Configuration (Helm)
laminar-core values.yaml
# Console (Web UI)
console:
image:
repository: 792306802931.dkr.ecr.us-east-1.amazonaws.com/laminar/console
tag: "latest"
pullPolicy: IfNotPresent
service:
type: ClusterIP
port: 3000
resources:
limits:
memory: 128Mi
cpu: 100m
requests:
memory: 128Mi
cpu: 100m
# GrepTimeDB URL for metrics/logs (server-side)
greptimeUrl: "http://greptimedb-standalone.laminar-monitoring:4000"
# Grafana URL for server-side health checks
grafanaUrl: "http://grafana.laminar-monitoring:80"
# Public Grafana URL for browser links
grafanaPublicUrl: "https://grafana.example.com"
# Auth URL (public URL of the console)
authUrl: "https://laminar.example.com"
# Enable development mode
devMode: false
# Google OAuth authentication
auth:
secret: "" # Generate with: openssl rand -base64 32
googleClientId: ""
googleClientSecret: ""
nodeSelector: {}
tolerations: []
affinity: {}
podLabels: {}
# Ingress
ingress:
enabled: false
className: nginx
host: laminar.local
annotations: {}
tls:
enabled: false
clusterIssuer: letsencrypt-prod
secretName: laminar-tls
# AWS ALB configuration
alb:
enabled: false
certificateArn: ""
groupName: ""
groupOrder: "999"
healthcheckPath: /console
listenPorts: '[{"HTTPS":443}]'
scheme: internet-facing
securityGroups: ""
sslPolicy: ELBSecurityPolicy-TLS13-1-2-Res-2021-06
targetType: ip
# Engine (Stream Processing)
engine:
# Storage URLs
artifactUrl: "s3://laminar-artifacts/"
checkpointUrl: "s3://laminar-checkpoints/"
image:
repository: 792306802931.dkr.ecr.us-east-1.amazonaws.com/laminar/backend
tag: "latest"
pullPolicy: IfNotPresent
imagePullSecrets: []
# Controller configuration
controller:
resources:
limits:
memory: 256Mi
cpu: 500m
requests:
memory: 256Mi
cpu: 500m
service:
grpcPort: 8001
compilerPort: 8002
adminPort: 8004
httpPort: 8000
# Worker configuration
worker:
resources:
limits:
memory: 2Gi
cpu: 2000m
requests:
memory: 2Gi
cpu: 2000m
slots: 2
nodeSelector: {}
tolerations: []
labels: {}
# S3 configuration
s3:
region: "us-east-1"
# Service account
serviceAccount:
create: true
annotations: {}
name: ""
# RBAC
role:
create: true
# Additional environment variables
env:
- name: CDC_JAR_PATH
value: "/app/cdc-bridge-1.0.0.jar"
- name: JVM_HEAP_SIZE
value: "512m"
- name: DEBEZIUM_LOG_LEVEL
value: "WARN"
# Storage class
storageClass:
enabled: false
name: gp3
isDefault: false
provisioner: ebs.csi.aws.com
parameters:
type: gp3
encrypted: "true"
volumeBindingMode: WaitForFirstConsumer
allowVolumeExpansion: true
reclaimPolicy: Delete
nodeSelector: {}
tolerations: []
affinity: {}
podLabels: {}
podAnnotations: {}
securityContext: {}
volumes: []
volumeMounts: []
existingConfigMap: ""laminar-monitoring values.yaml
# Grafana
grafana:
enabled: true
adminPassword: "laminar"
datasources:
datasources.yaml:
apiVersion: 1
datasources:
- name: Laminar-Metrics
type: prometheus
url: http://greptimedb-standalone:4000/v1/prometheus
isDefault: true
- name: GrepTimeDB-Metrics
type: prometheus
url: http://greptimedb-standalone:4000/v1/prometheus
- name: GrepTimeDB-Logs
type: mysql
url: greptimedb-standalone:4002
database: laminar_logs
user: root
dashboardProviders:
dashboardproviders.yaml:
apiVersion: 1
providers:
- name: default
folder: Laminar
type: file
disableDeletion: false
editable: true
options:
path: /var/lib/grafana/dashboards
# GrepTimeDB
greptimedb:
enabled: true
greptimedb-standalone:
resources:
requests:
cpu: 500m
memory: 1Gi
limits:
cpu: 2000m
memory: 4Gi
# Vector
vector:
enabled: true
# Kubernetes metrics
kubeStateMetrics:
enabled: true
kube-state-metrics:
metricLabelsAllowlist:
- pods=[*]
- deployments=[*]
# Node metrics
nodeExporter:
enabled: true
# Cost monitoring
opencost:
enabled: trueEnvironment-Specific Overrides
KIND (Local Kubernetes)
# values-kind.yaml
console:
devMode: true
authUrl: "http://localhost:3000"
engine:
artifactUrl: "file:///tmp/laminar/artifacts"
checkpointUrl: "file:///tmp/laminar/checkpoints"
ingress:
enabled: true
className: nginx
host: laminar.localEKS (AWS)
# values-eks.yaml
engine:
artifactUrl: "s3://my-bucket/artifacts/"
checkpointUrl: "s3://my-bucket/checkpoints/"
s3:
region: "us-east-1"
serviceAccount:
annotations:
eks.amazonaws.com/role-arn: "arn:aws:iam::123456789:role/laminar"
ingress:
enabled: true
alb:
enabled: true
certificateArn: "arn:aws:acm:us-east-1:123456789:certificate/xxx"
scheme: internet-facingNext Steps
- Upgrades - Upgrade procedures
- High Availability - HA configuration
- Security Hardening - Security best practices