k3s vs k8s — battle
You want Kubernetes. But “the cluster” is a Raspberry Pi on a factory floor, or a CI runner that lives for ninety seconds, or three tiny VMs at a branch office — or it’s a fifty-node, multi-team platform in the cloud that has to bend to every compliance checkbox you own. Two answers shout back. One says k3s: a single binary, batteries included, up in seconds. The other says k8s: the full upstream distribution, every component yours to choose and operate.
Here’s the thing almost everyone gets wrong on the way in: k3s is not a different Kubernetes. It’s a CNCF-certified, fully-conformant Kubernetes distribution — the same API, the same kubectl, the same YAML. A Deployment that runs on one runs on the other. So this was never a fight about what you can do. It’s a fight about how much of the machinery you carry yourself.
That collapses the whole decision to one question:
Who carries the operational weight — the distro, or you? And where does the cluster run?
- Lean footprint, edge/IoT/ARM, small teams, CI, dev, single-to-small production, “I want a cluster, not a second job” — that’s k3s.
- Many nodes, cloud-native at scale, deep customization, multi-tenant, every knob exposed and audited — that’s full k8s (vanilla / upstream, the kind you stand up with
kubeadmor buy managed).
The one line to tattoo on the decision: k3s is Kubernetes with the operations pre-decided; vanilla k8s leaves them to you. Same API on top — a very different amount of you underneath. (Note: throughout, “k8s” means a full upstream install like kubeadm; k3s is itself a k8s distribution, so the honest comparison is packaged-and-slimmed vs modular-and-assembled.)
k3s — Kubernetes, packaged and slimmed
k3s (from Rancher Labs, now SUSE; a CNCF Sandbox project) ships the entire control plane as one self-contained binary under 100 MB. The name is a wink, not an acronym — Kubernetes is ten letters stylised k8s; something half the size is a five-letter word, k3s. There is no long form.
What that binary swallows is the whole point. Instead of running kube-apiserver, the scheduler, and the controller-manager as separate processes against an external etcd, k3s runs them inside one process and, by default, stores cluster state in SQLite through a shim called Kine (Kine Is Not etcd) that speaks the etcd API to a SQL backend. It bundles containerd + runc as the runtime, flannel for networking, CoreDNS, Traefik for ingress, ServiceLB (Klipper) for LoadBalancer services, local-path-provisioner as the default storage class, and metrics-server — all on by default, each one --disable-able when you’d rather bring your own. To shrink, it drops legacy, alpha, and in-tree cloud-provider/storage plugins you almost never want on the edge.
A k3s node runs as a server (control plane + datastore) or an agent (worker) — the same binary, a different flag. An agent registers to the server’s supervisor on :6443 and opens a websocket tunnel back, so the control plane can reach each kubelet without inbound ports on the workers. The result is a cluster you stand up with one curl … | sh. A k3s server asks for roughly the same ~2 GB of RAM as an upstream control-plane node — that’s the official minimum — but runs leaner in practice (a real server measures ~1.2–1.6 GB) with far fewer processes; an agent (worker / edge) node runs in ~512 MB (measured ~270 MB), which is what lets it shine on ARM. It is production-grade — not a toy — and it scales out: for high availability you swap SQLite for embedded etcd (an odd number of servers, ≥ 3) or an external database (MySQL/Postgres).
k8s (vanilla / upstream) — the full platform you assemble
Full upstream Kubernetes is modular by design, and that modularity is the feature. The control plane is a set of separate components — kube-apiserver, etcd (mandatory, and yours to run and back up), kube-scheduler, kube-controller-manager, and an optional cloud-controller-manager — typically run as static pods across multiple control-plane nodes. Workers run the kubelet, kube-proxy, and a CRI runtime.
Crucially, upstream Kubernetes ships almost nothing above the core. You choose your CNI (Calico, Cilium, …), your ingress controller, your load-balancer (a cloud LB or MetalLB), your CSI storage driver, and your monitoring. (CoreDNS is the standard cluster DNS on both — kubeadm deploys it as a default add-on, just as k3s bundles it.) You stand the whole thing up with kubeadm (or kops/kubespray, or you let a cloud run the control plane for you with EKS/GKE/AKS). That’s more moving parts, a heavier footprint — budget ~2 GB RAM and 2 vCPUs for a control-plane node (etcd colocated, in kubeadm’s default stacked topology) — and a real operational surface. In exchange you get the thing k3s deliberately hides: every component is swappable and every knob is exposed, which is exactly what you need when you’re running many nodes, many teams, and an audit trail for each.
What’s in the box vs what you assemble
The single clearest way to feel the difference is to line up every layer a real cluster needs and ask “who provides it?” k3s answers “me, already running”; upstream k8s answers “you, your choice.” Neither answer is wrong — bundled defaults are a gift at the edge and a constraint at scale; assemble-it-yourself is friction on a Pi and freedom in a platform team.
--disable any of them); vanilla k8s makes you pick and run each one — the friction you don't want on an edge box, and the control you do want in a platform.Main differences
| k3s | k8s (vanilla / upstream) | |
|---|---|---|
| What it is | a CNCF-conformant distribution of Kubernetes | upstream Kubernetes itself |
| Packaging | one binary (< 100 MB), all components in-process | separate components, assembled |
| Default datastore | SQLite via Kine (etcd / external DB optional) | etcd (mandatory, you operate it) |
| Footprint | server ~2 GB (leaner in practice, ~1.2–1.6 GB) · agent ~512 MB; great on ARM | ~2 GB / 2 vCPU per control-plane node (etcd colocated) |
| Networking / ingress / LB / storage | bundled (flannel · Traefik · ServiceLB · local-path) | you choose (Calico/Cilium · nginx · MetalLB · CSI) |
| Runtime | containerd baked in | any CRI (containerd, CRI-O) — you install it |
| Install | curl … | sh in seconds | kubeadm / kops / kubespray / managed |
| HA | embedded etcd (≥ 3 servers) or external DB | etcd quorum across control-plane nodes |
| Agent → control plane | websocket tunnel to the supervisor (no inbound ports) | standard kubelet ↔ apiserver |
| Customisation | opinionated defaults; swap by --disable | everything pluggable from the start |
| Best fit | edge · IoT · ARM · CI · dev · small-to-mid prod | large, multi-tenant, cloud-native at scale |
↔ scroll the table sideways to see both columns.
Pros & cons
| 👍 Pros | 👎 Cons | |
|---|---|---|
| k3s | tiny single binary; ~512 MB agent footprint (server leaner than upstream in practice); one-command install; batteries included; runs on ARM/edge; same conformant API; fast cluster bring-up; easy to embed/automate | SQLite default doesn’t scale writes (swap for etcd at HA); opinionated bundled add-ons to replace if unwanted; fewer alpha/legacy/in-tree features; smaller (if growing) ecosystem of tutorials than upstream |
| k8s | infinitely modular — every component swappable; the reference platform with the deepest ecosystem & tooling; battle-tested at huge scale; first-class multi-tenancy & customization; managed offerings (EKS/GKE/AKS) remove control-plane ops | heavy footprint (~2 GB+ control plane, incl. etcd); you assemble and operate CNI, ingress, LB, storage, etcd backups; steeper learning curve; slower to stand up; overkill for edge/small clusters |
”I want to…” → which one
| I want to… | Reach for | Why |
|---|---|---|
| Run Kubernetes on a Raspberry Pi / ARM / edge gateway | k3s | single binary, tiny on ARM, a worker runs in ~512 MB — a core design goal |
| Spin up a throwaway cluster in CI or local dev | k3s (or k3d) | up in seconds, tear down just as fast |
| Run a 50-node, multi-team platform in the cloud | k8s | you need every knob, multi-tenancy, and deep customization |
| Use Cilium/Calico, MetalLB, a specific CSI by design | k8s | upstream assumes you bring your own — full freedom |
| Ship an appliance / embedded product with K8s inside | k3s | one binary is trivial to package and update |
| Offload control-plane operations entirely | managed k8s | EKS/GKE/AKS run apiserver + etcd for you |
| Meet CIS/FIPS hardening on a self-run datacenter cluster | RKE2 | Rancher’s hardened sibling — k3s ergonomics, upstream-aligned security |
| Avoid running and backing up etcd yourself | k3s | SQLite-via-Kine by default; etcd only when you opt into HA |
It’s the same Kubernetes — choose by weight, not capability
Re-read the tables and the trap dissolves: there is no capability axis here. Both are conformant Kubernetes; your manifests don’t care which one they land on. What actually differs is operational weight — footprint, moving parts, and how much of the cluster you personally run. So you don’t pick the “more powerful” one; you pick the one whose weight matches your hardware, your team, and your blast radius.
That makes it a spectrum, not a binary. The same project even gives you the rungs in between: k3s for lean and edge, RKE2 when you want k3s ergonomics with CIS/FIPS hardening in a datacenter, plain kubeadm when you want to assemble full upstream yourself, and managed (EKS/GKE/AKS) when you’d rather the cloud run the control plane.
The two anti-patterns are the two ways of mismatching weight to need. Don’t reach for full k8s on a three-node edge cluster “to be safe” — you’ll spend your life operating etcd, a CNI, and an ingress controller for a workload k3s would have run from a single binary. And don’t run k3s’s default SQLite under a hundred-node, write-heavy, multi-tenant production cluster — that’s the moment to move to embedded etcd (or RKE2/kubeadm) and the operational rigor that scale demands. Match the packaging to the place it runs.
TL;DR: k3s isn’t a lite alternative to Kubernetes — it is Kubernetes, CNCF-conformant, packaged as one binary under 100 MB with the control plane in-process, SQLite-via-Kine instead of mandatory etcd, and containerd · flannel · Traefik · ServiceLB · CoreDNS · local-path bundled and on by default. A worker/edge node runs in ~512 MB (the server, like upstream, wants ~2 GB but runs leaner, ~1.2–1.6 GB), it installs with one
curl … \| sh, and is built for edge · IoT · ARM · CI · dev · small-to-mid prod (HA via embedded etcd or an external DB). Vanilla k8s is the same API delivered modular: separate components against a mandatory etcd, you choose and operate CNI/ingress/LB/storage, ~2 GB+ per control-plane node — the right call for large, multi-tenant, cloud-native clusters that need every knob (or buy it managed as EKS/GKE/AKS). The mnemonic: k3s pre-decides the operations; k8s leaves them to you. It’s a spectrum — k3s → RKE2 → kubeadm → managed — so choose by operational weight, not capability: both run your YAML unchanged.
Sources
- k3s — official site and docs
- k3s architecture (server/agent, supervisor, Kine, tunnel) — docs
- Why “k3s”? Project FAQ (the name has no long form)
- k3s on GitHub — “Lightweight Kubernetes”
- Kine — an etcd shim for SQLite/MySQL/Postgres
- Kubernetes components (control plane, etcd, kubelet) — kubernetes.io
- Bootstrapping a cluster with kubeadm — kubernetes.io
- RKE2 — Rancher’s hardened, upstream-aligned distribution
- k3s is a CNCF Sandbox project