SNprogSN Platform

Multi-Cloud Internal Developer Platform

Production-grade Kubernetes platform spanning on-premises (Proxmox), Azure (AKS), and AWS (EKS) — fully automated with Infrastructure as Code, GitOps, and CI/CD pipelines.

About This Repository

This is a portfolio showcase of a production Internal Developer Platform (IDP) I designed and built. The platform source code lives across 7 private repositories (linked below). This repo documents the architecture, design decisions, and engineering practices behind the system.

Interested in the source code? I'm happy to walk through the implementation in detail or provide access upon request.

Overview

This project is a self-built Internal Developer Platform (IDP) designed to manage the full lifecycle of Kubernetes workloads across three infrastructure providers. It reflects real-world enterprise patterns: environment isolation, automated provisioning, GitOps-driven deployments, secret management, and comprehensive observability.

Key design principle: Developers push code → the platform handles everything else.

What This Platform Does

Architecture

graph TD
    META["<b>sn-platform</b><br/>Meta-repo<br/>Git Submodules · Unified Makefile"]

    META --> INFRA["<b>sn-k8s-infra</b><br/>Terraform · Ansible<br/>K8s v1.33"]
    META --> GITOPS["<b>sn-gitops</b><br/>ArgoCD · Kustomize<br/>Helm"]
    META --> AKS["<b>sn-cloud-aks</b><br/>Terraform<br/>azurerm"]
    META --> EKS["<b>sn-cloud-eks</b><br/>Terraform<br/>aws"]
    META --> PACKER["<b>sn-proxmox-templates</b><br/>Packer<br/>Cloud-Init"]
    META --> NET["<b>sn-network-config</b><br/>MikroTik<br/>SSH Backup"]
    META --> AI["<b>sn-ai</b><br/>MCP Server<br/>GitHub"]

    style META fill:#2d333b,stroke:#58a6ff,stroke-width:2px,color:#e6edf3
    style INFRA fill:#1a1e24,stroke:#7ee787,color:#e6edf3
    style GITOPS fill:#1a1e24,stroke:#f78166,color:#e6edf3
    style AKS fill:#1a1e24,stroke:#79c0ff,color:#e6edf3
    style EKS fill:#1a1e24,stroke:#d2a8ff,color:#e6edf3
    style PACKER fill:#1a1e24,stroke:#ffa657,color:#e6edf3
    style NET fill:#1a1e24,stroke:#a5d6ff,color:#e6edf3
    style AI fill:#1a1e24,stroke:#cea5fb,color:#e6edf3

End-to-End Deployment Flow

flowchart LR
    A["<b>VM Template</b><br/>Packer<br/><i>Ubuntu 24.04<br/>containerd<br/>kubeadm · kubelet</i>"] --> B["<b>Cluster Provisioning</b><br/>Terraform<br/><i>Create VMs<br/>Calico CNI<br/>kube-vip HA</i>"]
    B --> C["<b>GitOps Bootstrap</b><br/>ArgoCD<br/><i>Install ArgoCD<br/>Root app-of-apps<br/>→ sn-gitops</i>"]
    C --> D["<b>Application Delivery</b><br/>Kustomize<br/><i>Auto-sync from Git<br/>Environment-specific<br/>overlays</i>"]

    style A fill:#1a1e24,stroke:#ffa657,color:#e6edf3
    style B fill:#1a1e24,stroke:#7ee787,color:#e6edf3
    style C fill:#1a1e24,stroke:#f78166,color:#e6edf3
    style D fill:#1a1e24,stroke:#79c0ff,color:#e6edf3

Multi-Environment Architecture (On-Premises)

graph TD
    PVE["<b>Proxmox VE Hypervisor</b>"]

    PVE --> DEV["<b>Development</b><br/>VLAN 120 · 10.120.0.0/24<br/>1 master · 2 workers<br/>Auto-sync"]
    PVE --> STG["<b>Staging</b><br/>VLAN 130 · 10.130.0.0/24<br/>3 masters HA · 3 workers<br/>kube-vip · Auto-sync"]
    PVE --> PRD["<b>Production</b><br/>VLAN 140 · 10.140.0.0/24<br/>3 masters HA · 3 workers<br/>kube-vip · Manual sync"]

    DEV --> ARGO["<b>ArgoCD</b> per-cluster<br/>App-of-Apps → Kustomize overlays"]
    STG --> ARGO
    PRD --> ARGO

    ARGO --> APPS["Prometheus · Grafana · Cloudflared<br/>Sealed Secrets · Webshop App"]

    style PVE fill:#2d333b,stroke:#58a6ff,stroke-width:2px,color:#e6edf3
    style DEV fill:#1a1e24,stroke:#7ee787,color:#e6edf3
    style STG fill:#1a1e24,stroke:#ffa657,color:#e6edf3
    style PRD fill:#1a1e24,stroke:#f47067,color:#e6edf3
    style ARGO fill:#1a1e24,stroke:#f78166,color:#e6edf3
    style APPS fill:#1a1e24,stroke:#a5d6ff,color:#e6edf3

GitOps Delivery Model

sequenceDiagram
    actor Dev as Developer
    participant Git as Git Repository
    participant CI as GitHub Actions
    participant Argo as ArgoCD
    participant K8s as Kubernetes Cluster

    Dev->>Git: git push
    Git->>CI: Trigger pipeline
    CI->>CI: yamllint
    CI->>CI: kube-linter
    CI->>CI: kustomize build
    CI->>CI: helm lint
    CI-->>Git: ✅ All checks pass
    Git->>Git: Merge to main
    Argo->>Git: Detect change
    Argo->>K8s: Sync (auto or manual)
    K8s->>K8s: Sealed Secrets decrypt
    K8s-->>Dev: Workloads live

Technology Stack

Platform & Orchestration

Infrastructure as Code

Observability

CI/CD & Automation

Networking

Repository Structure

The platform is organized as a multi-repo architecture connected via Git submodules:

Source repositories are private. Access can be provided upon request for code review.

Platform Engineering Highlights

1. Self-Service Infrastructure

A single make deploy-dev command provisions an entire Kubernetes environment end-to-end:

Packer template → Terraform VMs → Ansible bootstrap → ArgoCD installed → Apps synced

No manual steps. No SSH. No kubectl apply. The platform handles it.

2. Multi-Cloud with Unified GitOps

All three infrastructure types (Proxmox on-prem, Azure AKS, AWS EKS) converge on the same GitOps repository. A single sn-gitops repo serves all clusters through Kustomize overlays:

base/                    ← shared manifests (write once)
overlays/
  on-prem/dev/           ← on-prem development patches
  on-prem/staging/       ← on-prem staging patches  
  on-prem/production/    ← on-prem production patches
  aks/dev/               ← Azure dev patches
  aks/prod/              ← Azure prod patches
  eks/dev/               ← AWS dev patches
  eks/prod/              ← AWS prod patches

3. Environment Promotion with Safety Gates

4. Immutable Infrastructure Pipeline

Ubuntu cloud image → Packer provisions → containerd + kubeadm installed
    → machine-id reset → cloud-init cleaned → Template VM created
        → Terraform clones template → Cloud-Init injects SSH keys → Ready

VMs are never patched in place. New template → new VMs → old ones destroyed.

5. Secret Management (Zero Secrets in Git)

Three-layer model ensuring no credentials ever touch version control:

6. Production-Grade HA

• Control plane: 3-node HA with kube-vip (ARP mode VIP) for staging + production
• etcd: Local NVMe storage for performance
• Boot ordering: Sequential cluster startup (dev → staging → prod) on Proxmox reboot
• NUMA pinning: Optional CPU/memory affinity for predictable performance

7. Network as Code

MikroTik router and switch configurations are versioned in Git with:

• Weekly automated backups via GitHub Actions (self-hosted runner)
• PR-based config review with automatic diff comments
• VLAN segmentation (120/130/140) for environment isolation

8. Developer Experience

• Dev Containers: All tools pre-installed (Terraform, Ansible, kubectl, Helm, kubeseal)
• Unified Makefile: 88 targets covering every operation
• One-command bootstrap: ./scripts/bootstrap.sh sets up the entire workspace
• Kubeconfig switching: ./switch-kube.sh dev|staging|production
• Interactive onboarding: make setup-secrets guides through credential setup

CI/CD Pipeline Overview

Every repository has automated quality gates:

Production safety: Cloud deployments to production environments require manual approval through GitHub Environment protection rules.

Architecture Decisions

Key decisions are documented as Architecture Decision Records (ADRs):

By the Numbers

Project Motivation

This platform was built to solve a real problem: managing multiple Kubernetes environments across hybrid infrastructure without manual intervention. Every component — from VM templates to application delivery — follows the principle that infrastructure should be declarative, version-controlled, and self-healing.

The goal is not just to run containers, but to provide a platform that other developers can use to deploy their applications safely and efficiently, with built-in guardrails for security, observability, and environment promotion.

Built with platform engineering principles: automate everything, version everything, trust nothing manually.