Traefik Reverse Proxy

Traefik is the edge router that handles SSL/TLS termination, routing, and load balancing for all Aegis services.

Overview

• Version: Traefik v2.11
• Container: traefik
• Location: /home/agent/stacks/traefik/
• Ports:
• 80 (HTTP → redirect to HTTPS)
• 443 (HTTPS)
• 8081 (Dashboard, internal only)
• Network: traefik_proxy (external bridge network)

Architecture

Internet
   ↓
[Traefik :443]
   ↓
   ├─→ aegisagent.ai → aegis-dashboard:8080
   ├─→ intel.aegisagent.ai → aegis-dashboard:8080
   ├─→ code.aegisagent.ai → 10.10.10.103:8443 (VS Code)
   ├─→ traefik.rbnk.uk → Traefik Dashboard :8080
   └─→ aegis.rbnk.uk → (301 redirect to aegisagent.ai)

Configuration

Docker Compose

Location: /home/agent/stacks/traefik/docker-compose.yml

services:
  traefik:
    image: traefik:v2.11
    container_name: traefik
    restart: unless-stopped
    command:
      # API Dashboard
      - "--api.dashboard=true"
      - "--api.insecure=true"

      # Docker provider
      - "--providers.docker=true"
      - "--providers.docker.exposedbydefault=false"
      - "--providers.docker.network=traefik_proxy"

      # File provider for non-Docker services
      - "--providers.file.directory=/etc/traefik/dynamic"
      - "--providers.file.watch=true"

      # Entrypoints
      - "--entrypoints.web.address=:80"
      - "--entrypoints.web.http.redirections.entrypoint.to=websecure"
      - "--entrypoints.web.http.redirections.entrypoint.scheme=https"
      - "--entrypoints.websecure.address=:443"

      # Cloudflare DNS Challenge for SSL
      - "--certificatesresolvers.cf.acme.dnschallenge=true"
      - "--certificatesresolvers.cf.acme.dnschallenge.provider=cloudflare"
      - "--certificatesresolvers.cf.acme.email=aegis@richardbankole.com"
      - "--certificatesresolvers.cf.acme.storage=/letsencrypt/acme.json"

      # Logging
      - "--log.level=INFO"

    ports:
      - "80:80"
      - "443:443"
      - "8081:8080"  # Dashboard on 8081 (8080 used by aegis-dashboard)

    environment:
      - CF_DNS_API_TOKEN=${CF_DNS_API_TOKEN}

    volumes:
      - /var/run/docker.sock:/var/run/docker.sock:ro
      - ./letsencrypt:/letsencrypt
      - ./dynamic:/etc/traefik/dynamic:ro

    networks:
      - traefik_proxy

Providers

Docker Provider

Automatically discovers services with traefik.enable=true label.

Configuration: - exposedbydefault=false - Only route explicitly enabled services - network=traefik_proxy - Use this network for container discovery

Example service labels:

labels:
  - "traefik.enable=true"
  - "traefik.http.routers.myapp.rule=Host(`myapp.aegisagent.ai`)"
  - "traefik.http.routers.myapp.entrypoints=websecure"
  - "traefik.http.routers.myapp.tls.certresolver=cf"
  - "traefik.http.services.myapp.loadbalancer.server.port=8080"

File Provider

Loads YAML configuration from /etc/traefik/dynamic/ (mounted from /home/agent/stacks/traefik/dynamic/).

Purpose: Route to non-Docker services (e.g., VS Code on host)

Watch mode: Files are reloaded automatically when changed

SSL/TLS Certificates

Let's Encrypt with Cloudflare DNS Challenge

Why DNS challenge? - Works behind firewalls (no port 80 required on Dockerhost) - Supports wildcard certificates - No rate limits for DNS validation

Configuration:

# Certificate resolver
--certificatesresolvers.cf.acme.dnschallenge=true
--certificatesresolvers.cf.acme.dnschallenge.provider=cloudflare
--certificatesresolvers.cf.acme.email=aegis@richardbankole.com
--certificatesresolvers.cf.acme.storage=/letsencrypt/acme.json

Environment:

CF_DNS_API_TOKEN=<Cloudflare API token>

Token permissions required: - Zone: DNS: Edit - Zone: Zone: Read

Storage: /home/agent/stacks/traefik/letsencrypt/acme.json

Permissions: Must be 600 (Traefik requirement)

chmod 600 /home/agent/stacks/traefik/letsencrypt/acme.json

Routing Rules

Host-Based Routing

Route requests based on Host header:

traefik.http.routers.myapp.rule=Host(`myapp.aegisagent.ai`)

Path-Based Routing

Route based on URL path:

traefik.http.routers.api.rule=Host(`aegisagent.ai`) && PathPrefix(`/api`)

Combining Rules

Use logical operators:

traefik.http.routers.app.rule=Host(`aegisagent.ai`) && (PathPrefix(`/app`) || PathPrefix(`/admin`))

Middlewares

Redirect Middleware

Permanently redirect old domains to new:

Example: aegis.rbnk.uk → aegisagent.ai

traefik.http.routers.aegis-redirect.middlewares=aegis-redirect-mw
traefik.http.middlewares.aegis-redirect-mw.redirectregex.regex=^https://aegis\\.rbnk\\.uk(.*)
traefik.http.middlewares.aegis-redirect-mw.redirectregex.replacement=https://aegisagent.ai$$1
traefik.http.middlewares.aegis-redirect-mw.redirectregex.permanent=true

Note: $$ escapes $ in Docker Compose YAML

Common Middleware Patterns

Rate limiting:

traefik.http.middlewares.ratelimit.ratelimit.average=100
traefik.http.middlewares.ratelimit.ratelimit.burst=50

Basic auth:

traefik.http.middlewares.auth.basicauth.users=user:$$apr1$$hash$$

Headers:

traefik.http.middlewares.secure.headers.sslredirect=true
traefik.http.middlewares.secure.headers.stsSeconds=31536000

Dynamic Configuration Files

VS Code Router

Location: /home/agent/stacks/traefik/dynamic/code-server.yml

http:
  routers:
    code:
      rule: "Host(`code.aegisagent.ai`)"
      entryPoints:
        - websecure
      service: code
      tls:
        certResolver: cf

    code-redirect:
      rule: "Host(`aegis-code.rbnk.uk`)"
      entryPoints:
        - websecure
      middlewares:
        - code-redirect-mw
      service: code
      tls:
        certResolver: cf

  middlewares:
    code-redirect-mw:
      redirectRegex:
        regex: "^https://aegis-code\\.rbnk\\.uk(.*)"
        replacement: "https://code.aegisagent.ai${1}"
        permanent: true

  services:
    code:
      loadBalancer:
        servers:
          - url: "http://10.10.10.103:8443"

Purpose: Route code.aegisagent.ai to VS Code running on Aegis LXC host at port 8443.

Adding New Dynamic Routes

1. Create YAML file in /home/agent/stacks/traefik/dynamic/
2. Define routers, services, and middlewares
3. Traefik auto-reloads (no restart needed)

Example:

# /home/agent/stacks/traefik/dynamic/myapp.yml
http:
  routers:
    myapp:
      rule: "Host(`myapp.aegisagent.ai`)"
      entryPoints:
        - websecure
      service: myapp
      tls:
        certResolver: cf

  services:
    myapp:
      loadBalancer:
        servers:
          - url: "http://10.10.10.103:9000"

Monitoring

Traefik Dashboard

URL: http://traefik.rbnk.uk (internal access only, port 8081)

Features: - View all routers and their rules - Service health status - Middleware configuration - Certificate status - Real-time metrics

Access:

# Port forward if needed
ssh -L 8081:localhost:8081 aegis

Then open: http://localhost:8081

Logs

# View Traefik logs
docker logs traefik -f

# Filter for errors
docker logs traefik 2>&1 | grep -i error

# Filter for certificate issues
docker logs traefik 2>&1 | grep -i cert

Certificate Status

Check stored certificates:

cat /home/agent/stacks/traefik/letsencrypt/acme.json | jq '.cf.Certificates[] | {domain: .domain.main, expires: .certificate}'

Network Architecture

Internal Network

The traefik_proxy network connects: - Traefik container - aegis-dashboard container - aegis-scheduler container - falkordb container - playwright container

Create network (if not exists):

docker network create traefik_proxy

Inspect network:

docker network inspect traefik_proxy

External Routing

Traffic flow from internet:

Internet
   ↓
Proxmox Host (157.180.63.15:443)
   ↓
Dockerhost Traefik (10.10.10.10)
   ↓ (TCP passthrough for Aegis subdomains)
Aegis LXC (10.10.10.103:443)
   ↓
Aegis Traefik
   ↓
Backend Services

Dockerhost Passthrough Config: Location: /srv/dockerdata/traefik/dynamic/aegis-passthrough.yml (on Dockerhost)

tcp:
  routers:
    aegis-passthrough:
      rule: "HostSNI(`aegisagent.ai`, `*.aegisagent.ai`)"
      service: aegis-backend
      tls:
        passthrough: true

  services:
    aegis-backend:
      loadBalancer:
        servers:
          - address: "10.10.10.103:443"

Purpose: Forward all *.aegisagent.ai traffic to Aegis LXC without terminating TLS.

Troubleshooting

Certificate not renewing

1. Check Cloudflare API token:

   docker exec traefik env | grep CF_DNS_API_TOKEN
   

2. Check DNS propagation:

   dig _acme-challenge.aegisagent.ai TXT
   

3. Check acme.json permissions:

   ls -la /home/agent/stacks/traefik/letsencrypt/acme.json
   # Should be: -rw------- (600)
   

4. Force certificate renewal:

   # Delete certificate from acme.json
   # Restart Traefik
   docker compose -f /home/agent/stacks/traefik/docker-compose.yml restart
   

Service not accessible

1. Check router is registered:
2. Visit Traefik dashboard: http://traefik.rbnk.uk

3. Look for your router in HTTP section

4. Check service label syntax:

   docker inspect <container> | jq '.[0].Config.Labels'
   

5. Check container is on traefik_proxy network:

   docker inspect <container> | jq '.[0].NetworkSettings.Networks'
   

6. Test backend directly:

   curl http://<container_ip>:<port>
   

DNS not resolving

1. Check DNS records in Cloudflare:

   dig aegisagent.ai +short
   # Should return: 157.180.63.15
   

2. Check subdomain:

   dig intel.aegisagent.ai +short
   # Should return: 157.180.63.15 (same as root)
   

SSL/TLS errors

1. Check certificate matches domain:

   openssl s_client -connect aegisagent.ai:443 -servername aegisagent.ai | openssl x509 -noout -text
   

2. Check certificate validity:

   echo | openssl s_client -connect aegisagent.ai:443 -servername aegisagent.ai 2>/dev/null | openssl x509 -noout -dates
   

3. Test TLS handshake:

   curl -vI https://aegisagent.ai
   

Development

Adding a New Domain

1. Add DNS A record in Cloudflare:
2. Name: newapp (or @ for root)
3. Type: A
4. Content: 157.180.63.15

5. Proxied: No (important for DNS challenge)

6. Add Dockerhost passthrough (if needed): Edit /srv/dockerdata/traefik/dynamic/aegis-passthrough.yml on Dockerhost:

   tcp:
     routers:
       aegis-passthrough:
         rule: "HostSNI(`newapp.aegisagent.ai`, `*.aegisagent.ai`)"
   

7. Add Traefik labels to service OR create dynamic config file

8. Test:

   curl https://newapp.aegisagent.ai
   

Local Development

For local testing without DNS:

1. Add to /etc/hosts:

   127.0.0.1 test.aegisagent.ai
   

2. Use Traefik with insecureSkipVerify (dev only):

   traefik.http.routers.test.tls=true
   

3. Test with curl:

   curl -k https://test.aegisagent.ai
   

Related Documentation

• dashboard.md - Main service behind Traefik
• postgresql.md - Database service
• playwright.md - Browser automation service