KMTPUMIT — Arsitektur Sistem & Deployment

Kubernetes kubeadm 2 node · Traefik + Calico + local-path · namespace kmtxpumit-development · hasil pembacaan langsung dari API server (kubectl), bukan estimasi

1 — Kondisi saat ini (as-is)

HTTPS NodePort 30625 (https) / 32144 (http) routing per Host() http://api:3000 DATABASE_URL http://seaweedfs:8333 Users Browser / dashboard Host publik 187.77.112.39 2 VM Ubuntu 24.04 · 2 vCPU / 8 GB / node wg0 WireGuard 10.10.10.0/24 (IP node) Service traefik LoadBalancer masih EXTERNAL-IP <pending> KUBERNETES CLUSTER — kubeadm v1.36.4 · containerd 2.2.1 · pod 192.168.0.0/16 · svc 10.96.0.0/12 · usia 36d ingress layer — namespace traefik-system · pod berjalan di k8s-worker Traefik v3.7.12 • entrypoints: web:8000 → redirect :443 • websecure:8443 · metrics:9100 • ACME Let's Encrypt (certResolver: le) • Helm release v2 · PVC /data/acme.json IngressRoute (CRD, 4 rute) • development-api.kmtxpumit.my.id • development.kmtxpumit.my.id • development-ops / -vendor .kmtxpumit.my.id • provider: kubernetescrd · IngressClass default Middleware & transport • cors-kmtxpumit (whitelist 3 origin) • timeout-kmtxpumit (ServerTransport) • TLS terminate di ingress (ACME) • hanya rute api yang pakai CORS k8s-master · control-plane · taint NoSchedule Control plane (static pod) • kube-apiserver :6443 • etcd (single node, /var/lib/etcd) • kube-controller-manager • kube-scheduler • authZ Node,RBAC · NodeRestriction • tanpa audit log · tanpa enkripsi at-rest Node agent • kubelet (webhook authZ, rotateCerts) • containerd 2.2.1 · cgroup systemd • kube-proxy (mode iptables) • calico-node + csi-node-driver • staticPodPath /etc/kubernetes/manifests Titik kegagalan tunggal • 1 control plane · 1 etcd lokal • tanpa snapshot etcd terjadwal • tidak menjalankan workload aplikasi • master mati = klaster tidak bisa diubah (pod yang sudah jalan tetap hidup) k8s-worker · SEMUA workload aplikasi ada di sini marketplace svc :80 → :8080 image …-marketplace:development ops svc :80 → :8080 image …-ops:development vendor svc :80 → :8080 image …-vendor:development api svc :3000 · liveness + readiness PVC artifacts 2Gi → /data worker (queue/async) tanpa probe · PVC artifacts 2Gi DATABASE_URL + SERVICE_TOKEN postgres-0 StatefulSet · 16-alpine headless svc :5432 · 5Gi seaweedfs-0 StatefulSet · 3.68 (S3) headless svc :8333 · 5Gi hermes-agent nousresearch v2026.9.7 SA-nya = cluster-admin ⚠ PVC local-path (RWO, reclaim Delete): pgdata 5Gi · seaweed-data 5Gi · artifacts 2Gi · hermes-agent-data 2Gi ConfigMap kmtxpumit-config (21 key) · Secret kmtxpumit-secrets · imagePullSecret ghcr-pull http://api:3000 (INTERNAL_API_URL) · http://seaweedfs:8333 (STORAGE_ENDPOINT) namespace kmtxpumit-development — semua Deployment/StatefulSet replicas: 1 · tanpa HPA / PDB / NetworkPolicy / quota pod aplikasi tanpa securityContext (jalan sebagai root) · AUTH_LUAR_DEV=1 · tag image :development platform & addons — cluster-wide (Calico dikelola Tigera Operator, bukan manifest statis) Calico CNI v3.32.1 • VXLAN encap · iptables dataplane • IP pool 192.168.0.0/16 (block /26) • calico-typha 1 replika + PDB • 0 NetworkPolicy (jaringan datar) CoreDNS 1.14.2 • 2 replika · podAntiAffinity • cache 30 · prometheus :9153 • readOnlyRootFilesystem ✓ • keduanya di k8s-worker local-path-provisioner • StorageClass local-path (default) • WaitForFirstConsumer · Delete • path node /opt/local-path-provisioner • disk lokal node, bukan shared kube-proxy · CSI · operator • kube-proxy DaemonSet (iptables) • csi-node-driver (Calico CSI) • tigera-operator v1.42.3 • tanpa metrics-server Let's Encrypt ACME HTTP-01 Google OAuth client secret (login eksternal) GHCR ghcr.io/pt-sagung-… Legend Frontend / ingress Backend / service Database / storage Platform / infra node Eksternal / tooling Alur keluar (ACME, OAuth, pull image) Batas klaster / node Batas namespace aplikasi

2 — Target yang disarankan (to-be): GitOps + IaC + observability

push/PR trigger push watch commit manifest (image digest) → Git, bukan kubectl apply sync / reconcile bootstrap node inventory LANE A — aplikasi: source → CI → registry → CD (GitOps, tanpa kubectl manual) Developer • branch + pull request • review, bukan apply • commit = satu-satunya cara mengubah produksi Git repo (GitHub) • main = kode aplikasi • /deploy = manifest K8s • Kustomize overlay dev/prod • branch protection + CODEOWNERS GitHub Actions • lint · unit test · build • Trivy scan (fail on HIGH) • SBOM + cosign sign • push GHCR by digest GHCR • tag immutable + digest • ganti :development • retention policy • pull secret per namespace Argo CD • watch repo /deploy (auto-sync) • deteksi drift + rollback • ApplicationSet per environment • Argo Rollouts (canary/blue-green) LANE B — infrastruktur sebagai kode (dari "setup manual" ke reproducibility) Terraform • provision VM/VPS, volume, snapshot • WireGuard wg0 + firewall rule • DNS record (api / ops / vendor / dev) • remote state (S3/MinIO seaweedfs) • tls/ssh key material, bukan manual Ansible • kubeadm init / join (idempoten) • containerd, cgroups, kernel module • harden node (sysctl, firewall, user) • install Calico / Traefik / storage • jadi "runbook hidup" yang bisa diuji ulang LANE C — observability & scaling metrik, log, alert • metrics-server → kubectl top + HPA • kube-prometheus-stack + Grafana • Loki/Vector untuk log container • alertmanager → Telegram/Slack • dashboard SLO: error rate, latency TARGET CLUSTER — perbaikan wajib sebelum menambah lapisan baru • backup etcd terjadwal (CronJob snapshot → object storage) + uji restore — ini yang paling mendesak • RBAC: SA hermes-agent diturunkan dari cluster-admin ke Role spesifik · PSA restricted + NetworkPolicy default-deny • replicas ≥ 2 + PDB + podAntiAffinity · secret terenkripsi at-rest · audit log apiserver aktif • secret dari Sealed Secrets / External Secrets + SOPS (bukan file .env atau kubectl create secret manual) • opsional: 3 control plane HA + HAProxy/VIP, MetalLB untuk menggantikan LoadBalancer <pending> Komponen baru / diubah Infrastruktur sebagai kode Observability Umpan balik GitOps (manifest, bukan kubectl)

Yang sudah benar

  • • RBAC + Node authorization, admission NodeRestriction
  • • kubelet: anonymous auth off, authZ Webhook, rotateCertificates
  • • etcd TLS penuh + client/peer cert auth
  • • Calico dikelola operator (bukan manifest statis), typha punya PDB
  • • CoreDNS 2 replika + anti-affinity, readOnlyRootFilesystem
  • • ConfigMap untuk config, Secret untuk kredensial (tidak hardcoded)
  • • TLS otomatis Let's Encrypt + CORS middleware
  • • CoreDNS & Traefik sudah drop ALL capability + seccomp

Keamanan — prioritas 1

  • • SA hermes-agent di-bind ke cluster-admin: satu pod = seluruh klaster
  • • 0 NetworkPolicy: semua pod bisa saling akses (network datar)
  • • Namespace tanpa label Pod Security Admission (efektif privileged)
  • • Pod aplikasi tanpa securityContext → jalan sebagai root
  • • Tanpa audit log apiserver: tidak ada jejak siapa mengubah apa
  • • Secret tidak terenkripsi at-rest di etcd
  • • AUTH_LUAR_DEV=1 + tag :development di domain publik

Ketersediaan & risiko

  • • 1 control plane + 1 etcd lokal, tanpa snapshot terjadwal
  • • Semua workload di satu node (k8s-worker)
  • • Semua Deployment/StatefulSet replicas: 1, tanpa PDB
  • • Tanpa HPA, tanpa metrics-server → tidak bisa scaling otomatis
  • • Traefik LoadBalancer <pending> → bergantung NodePort
  • • worker: liveness/readiness dan seaweedfs: tanpa probe
  • • Storage node-local (RWO) → pod tidak bisa pindah node bebas

Alur trafik produksi

  • • Users → host :443/:80 → NodePort 30625/32144
  • • Traefik → IngressRoute (4 Host rules) → Service → Pod
  • • api dipanggil internal via http://api:3000
  • • artefak disimpan ke S3 http://seaweedfs:8333 (bucket kmtxpumit)
  • • Postgres & SeaweedFS headless service, 1 replika masing-masing
  • • Login eksternal Google OAuth (client secret di Secret)

Data & backup (belum ada)

  • • Postgres 16 single instance, tanpa replika/backup terjadwal
  • • SeaweedFS 1 node (S3) — artefak & bucket kmtxpumit
  • • PVC local-path: hilang jika node/disk rusak (reclaim Delete)
  • • Tanpa Velero / pg_dump CronJob / snapshot etcd
  • • Uji restore belum pernah dilakukan → backup tanpa uji = asumsi

Peta belajar (urutan yang disarankan)

  • • Fase 0 — backup & restore etcd + Postgres (prasyarat semua fase lain)
  • • Fase 1 — Git + GitHub Actions: build, test, Trivy, push GHCR (digest)
  • • Fase 2 — Argo CD: manifest di Git, sync otomatis, hapus kebiasaan kubectl apply
  • • Fase 3 — Ansible (bootstrap node) lalu Terraform (VM, wg0, DNS, state)
  • • Fase 4 — observability, NetworkPolicy, PSA, RBAC diperkecil
  • • Fase 5 — HA control plane, managed DB, Velero, MetalLB

KMTPUMIT · kubeadm v1.36.4 · 2 node · 12 Deployment/StatefulSet · 5 PVC · 4 IngressRoute · 0 NetworkPolicy · 0 HPA · 0 ResourceQuota · 1 PDB — data dibaca langsung dari API server (kubectl), 29 Sep 2026