Documentation

Kubernetes

Deploy AppEngine to a Kubernetes cluster with replicas, ingress, persistent volumes, and secrets.

For real production with HA, Kubernetes is the deployment target. AppEngine itself is stateless and scales horizontally; MongoDB and Redis live as stateful sets with persistent volumes. This page covers the manifests, the ingress, and the secrets.

What you'll deploy

┌──────────────────────────────────────────────────────────┐
│  Ingress (nginx-ingress / traefik)                       │
│  TLS termination, path-based routing                     │
└────────────────┬─────────────────────────────────────────┘
                 │
        ┌────────▼────────┐
        │ appengine Svc   │  ClusterIP
        └────────┬────────┘
                 │
        ┌────────▼────────────┐
        │ appengine Deployment│  3 replicas, rolling update
        └────────┬────────────┘
                 │
       ┌─────────┴────────────┐
       │                      │
┌──────▼──────┐        ┌──────▼──────┐
│ mongodb STS │        │  redis STS  │
│ + PVC (50G) │        │ + PVC (10G) │
└─────────────┘        └─────────────┘

For genuine HA on the data tier, MongoDB should be a replica set (3 members) and Redis should run as Redis Sentinel or a managed offering (ElastiCache, Memorystore). The single-pod mongodb and redis shown here are fine for a small production with infrequent restarts.

Prerequisites

  • A Kubernetes cluster (1.28+). EKS, GKE, AKS, k3s, DigitalOcean Kubernetes — any of them.
  • kubectl configured against the cluster
  • An ingress controller installed (nginx-ingress is the default assumption below)
  • Cert-manager installed if you want automatic Let's Encrypt
  • A container registry credential for the AppEngine image (Docker Hub, GHCR, ECR)

Step-by-step

  1. 1

    Create the namespace

    kubectl create namespace appmint
    kubectl config set-context --current --namespace=appmint
    

    Everything below assumes you're in the appmint namespace.

  2. 2

    Create the registry pull secret

    If your AppEngine image is in a private registry:

    kubectl create secret docker-registry registry-cred \
      --docker-server=ghcr.io \
      --docker-username=$DOCKER_USER \
      --docker-password=$DOCKER_TOKEN \
      --docker-email=$DOCKER_EMAIL
    

    Reference this secret as imagePullSecrets in the deployment manifest.

  3. 3

    Create the application secrets

    AppEngine reads dozens of envs. Manage them as a single Secret for ergonomics:

    kubectl create secret generic appengine-env \
      --from-env-file=.env.production
    

    Where .env.production is your fully-populated env file. See Environment variables for what to set.

    For production, prefer a real secrets manager — External Secrets Operator with AWS Secrets Manager, Vault, GCP Secret Manager. The kubectl create secret form is fine for getting started; rotate to managed before serious traffic.

  4. 4

    Provision MongoDB

    Single-pod, with a PVC. Replace storage class with what your cluster offers (gp3 on EKS, pd-ssd on GKE, etc.).

    # mongodb.yaml
    apiVersion: apps/v1
    kind: StatefulSet
    metadata:
      name: mongodb
    spec:
      serviceName: mongodb
      replicas: 1
      selector:
        matchLabels: { app: mongodb }
      template:
        metadata:
          labels: { app: mongodb }
        spec:
          containers:
            - name: mongo
              image: mongo:7
              ports:
                - containerPort: 27017
              env:
                - name: MONGO_INITDB_ROOT_USERNAME
                  valueFrom: { secretKeyRef: { name: appengine-env, key: MONGODB_USERNAME } }
                - name: MONGO_INITDB_ROOT_PASSWORD
                  valueFrom: { secretKeyRef: { name: appengine-env, key: MONGODB_PASSWORD } }
              volumeMounts:
                - name: data
                  mountPath: /data/db
              resources:
                requests: { cpu: 500m, memory: 2Gi }
                limits:   { cpu: 2,    memory: 4Gi }
      volumeClaimTemplates:
        - metadata: { name: data }
          spec:
            accessModes: [ReadWriteOnce]
            storageClassName: gp3
            resources:
              requests: { storage: 50Gi }
    ---
    apiVersion: v1
    kind: Service
    metadata: { name: mongodb }
    spec:
      clusterIP: None   # headless
      selector: { app: mongodb }
      ports: [{ port: 27017 }]
    

    Apply:

    kubectl apply -f mongodb.yaml
    

    For a real replica set, deploy via the official MongoDB Community Operator instead of this manifest. That's a bigger lift but mandatory for HA on Mongo.

  5. 5

    Provision Redis

    # redis.yaml
    apiVersion: apps/v1
    kind: StatefulSet
    metadata: { name: redis }
    spec:
      serviceName: redis
      replicas: 1
      selector: { matchLabels: { app: redis } }
      template:
        metadata: { labels: { app: redis } }
        spec:
          containers:
            - name: redis
              image: redis:7
              args: ['redis-server', '--appendonly', 'yes', '--save', '60', '1000']
              ports: [{ containerPort: 6379 }]
              volumeMounts:
                - name: data
                  mountPath: /data
              resources:
                requests: { cpu: 200m, memory: 512Mi }
                limits:   { cpu: 1,    memory: 2Gi }
      volumeClaimTemplates:
        - metadata: { name: data }
          spec:
            accessModes: [ReadWriteOnce]
            storageClassName: gp3
            resources: { requests: { storage: 10Gi } }
    ---
    apiVersion: v1
    kind: Service
    metadata: { name: redis }
    spec:
      clusterIP: None
      selector: { app: redis }
      ports: [{ port: 6379 }]
    
    kubectl apply -f redis.yaml
    
  6. 6

    Deploy AppEngine

    Three replicas, rolling-update, env from the secret:

    # appengine.yaml
    apiVersion: apps/v1
    kind: Deployment
    metadata: { name: appengine }
    spec:
      replicas: 3
      strategy:
        type: RollingUpdate
        rollingUpdate: { maxSurge: 1, maxUnavailable: 0 }
      selector: { matchLabels: { app: appengine } }
      template:
        metadata: { labels: { app: appengine } }
        spec:
          imagePullSecrets:
            - name: registry-cred
          containers:
            - name: appengine
              image: ghcr.io/appmint/appengine:1.x.x   # pin a version
              ports: [{ containerPort: 3300 }]
              envFrom:
                - secretRef: { name: appengine-env }
              env:
                - name: NODE_ENV
                  value: production
                - name: SERVER_PORT
                  value: '3300'
                - name: MONGODB_HOST
                  value: mongodb
                - name: REDIS_HOST
                  value: redis
              resources:
                requests: { cpu: 500m, memory: 1Gi }
                limits:   { cpu: 2,    memory: 4Gi }
              readinessProbe:
                httpGet: { path: /monitoring/health, port: 3300 }
                initialDelaySeconds: 20
                periodSeconds: 10
              livenessProbe:
                httpGet: { path: /monitoring/health, port: 3300 }
                initialDelaySeconds: 60
                periodSeconds: 30
                failureThreshold: 3
    ---
    apiVersion: v1
    kind: Service
    metadata: { name: appengine }
    spec:
      selector: { app: appengine }
      ports:
        - port: 80
          targetPort: 3300
    
    kubectl apply -f appengine.yaml
    kubectl rollout status deployment/appengine
    

    The first rollout takes a couple of minutes — the container builds caches and runs migrations on boot.

  7. 7

    Set up the ingress

    Standard nginx-ingress with TLS via cert-manager:

    # ingress.yaml
    apiVersion: networking.k8s.io/v1
    kind: Ingress
    metadata:
      name: appengine
      annotations:
        cert-manager.io/cluster-issuer: letsencrypt-prod
        nginx.ingress.kubernetes.io/proxy-body-size: 100m
        nginx.ingress.kubernetes.io/proxy-read-timeout: '300'
        nginx.ingress.kubernetes.io/proxy-send-timeout: '300'
    spec:
      ingressClassName: nginx
      tls:
        - hosts: [appengine.your-domain.com]
          secretName: appengine-tls
      rules:
        - host: appengine.your-domain.com
          http:
            paths:
              - path: /
                pathType: Prefix
                backend:
                  service: { name: appengine, port: { number: 80 } }
    
    kubectl apply -f ingress.yaml
    

    The proxy-body-size annotation matters for file uploads (default nginx limit is 1m). Read/send timeouts handle long-running streaming responses (/ai/agent/stream).

  8. 8

    Configure WebSocket sticky routing (if using chat)

    The chat module uses Socket.IO. With multiple AppEngine replicas you need either:

    • The Redis adapter, which AppEngine wires up automatically when REDIS_HOST is set. With this, any pod can serve any socket. No special ingress config needed.
    • Sticky sessions at the ingress, if you don't want the Redis fan-out. Annotate:
      nginx.ingress.kubernetes.io/affinity: 'cookie'
      nginx.ingress.kubernetes.io/session-cookie-name: 'AE_SOCK'
      nginx.ingress.kubernetes.io/session-cookie-expires: '3600'
      

    Default is the Redis adapter — simpler, scales further. Skip the affinity annotations unless you have a reason.

  9. 9

    Verify

    kubectl get pods -n appmint
    # NAME                          READY   STATUS    RESTARTS   AGE
    # appengine-7c7d9c4f8c-abc12    1/1     Running   0          2m
    # appengine-7c7d9c4f8c-def34    1/1     Running   0          2m
    # appengine-7c7d9c4f8c-ghi56    1/1     Running   0          2m
    # mongodb-0                     1/1     Running   0          5m
    # redis-0                       1/1     Running   0          5m
    
    curl https://appengine.your-domain.com/monitoring/health
    # {"status":"ok","mongodb":"up","redis":"up"}
    

    Sign in with the ROOT_* credentials you set in the secret.

  10. 10

    Set up autoscaling (optional)

    A Horizontal Pod Autoscaler scales AppEngine on CPU:

    # hpa.yaml
    apiVersion: autoscaling/v2
    kind: HorizontalPodAutoscaler
    metadata: { name: appengine }
    spec:
      scaleTargetRef:
        apiVersion: apps/v1
        kind: Deployment
        name: appengine
      minReplicas: 3
      maxReplicas: 10
      metrics:
        - type: Resource
          resource:
            name: cpu
            target: { type: Utilization, averageUtilization: 70 }
    
    kubectl apply -f hpa.yaml
    

    For metrics-driven autoscaling on queue depth or response time, install Keda and target a Prometheus metric.

Backups

For Mongo, the simplest path is a CronJob that runs mongodump to S3:

apiVersion: batch/v1
kind: CronJob
metadata: { name: mongo-backup }
spec:
  schedule: '0 2 * * *'   # 2am daily
  jobTemplate:
    spec:
      template:
        spec:
          restartPolicy: OnFailure
          containers:
            - name: backup
              image: amazon/aws-cli:latest
              command:
                - /bin/sh
                - -c
                - |
                  mongodump --uri="$MONGODB_CONN" --gzip --archive=/tmp/dump.gz
                  aws s3 cp /tmp/dump.gz s3://my-backups/mongo-$(date +%Y%m%d).gz
              env:
                - name: MONGODB_CONN
                  valueFrom: { secretKeyRef: { name: appengine-env, key: MONGODB_CONN } }
                - name: AWS_ACCESS_KEY_ID
                  valueFrom: { secretKeyRef: { name: aws-creds, key: access_key } }
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom: { secretKeyRef: { name: aws-creds, key: secret_key } }

Test restoration in a separate cluster monthly. Backup that hasn't been restored isn't a backup.

What's next