k3s vs k8s — battle

· updated

You want Kubernetes. But “the cluster” is a Raspberry Pi on a factory floor, or a CI runner that lives for ninety seconds, or three tiny VMs at a branch office — or it’s a fifty-node, multi-team platform in the cloud that has to bend to every compliance checkbox you own. Two answers shout back. One says k3s: a single binary, batteries included, up in seconds. The other says k8s: the full upstream distribution, every component yours to choose and operate.

Here’s the thing almost everyone gets wrong on the way in: k3s is not a different Kubernetes. It’s a CNCF-certified, fully-conformant Kubernetes distribution — the same API, the same kubectl, the same YAML. A Deployment that runs on one runs on the other. So this was never a fight about what you can do. It’s a fight about how much of the machinery you carry yourself.

That collapses the whole decision to one question:

Who carries the operational weight — the distro, or you? And where does the cluster run?

The one line to tattoo on the decision: k3s is Kubernetes with the operations pre-decided; vanilla k8s leaves them to you. Same API on top — a very different amount of you underneath. (Note: throughout, “k8s” means a full upstream install like kubeadm; k3s is itself a k8s distribution, so the honest comparison is packaged-and-slimmed vs modular-and-assembled.)

k3s — Kubernetes, packaged and slimmed

k3s (from Rancher Labs, now SUSE; a CNCF Sandbox project) ships the entire control plane as one self-contained binary under 100 MB. The name is a wink, not an acronym — Kubernetes is ten letters stylised k8s; something half the size is a five-letter word, k3s. There is no long form.

What that binary swallows is the whole point. Instead of running kube-apiserver, the scheduler, and the controller-manager as separate processes against an external etcd, k3s runs them inside one process and, by default, stores cluster state in SQLite through a shim called Kine (Kine Is Not etcd) that speaks the etcd API to a SQL backend. It bundles containerd + runc as the runtime, flannel for networking, CoreDNS, Traefik for ingress, ServiceLB (Klipper) for LoadBalancer services, local-path-provisioner as the default storage class, and metrics-server — all on by default, each one --disable-able when you’d rather bring your own. To shrink, it drops legacy, alpha, and in-tree cloud-provider/storage plugins you almost never want on the edge.

A k3s node runs as a server (control plane + datastore) or an agent (worker) — the same binary, a different flag. An agent registers to the server’s supervisor on :6443 and opens a websocket tunnel back, so the control plane can reach each kubelet without inbound ports on the workers. The result is a cluster you stand up with one curl … | sh. A k3s server asks for roughly the same ~2 GB of RAM as an upstream control-plane node — that’s the official minimum — but runs leaner in practice (a real server measures ~1.2–1.6 GB) with far fewer processes; an agent (worker / edge) node runs in ~512 MB (measured ~270 MB), which is what lets it shine on ARM. It is production-grade — not a toy — and it scales out: for high availability you swap SQLite for embedded etcd (an odd number of servers, ≥ 3) or an external database (MySQL/Postgres).

k8s (vanilla / upstream) — the full platform you assemble

Full upstream Kubernetes is modular by design, and that modularity is the feature. The control plane is a set of separate components — kube-apiserver, etcd (mandatory, and yours to run and back up), kube-scheduler, kube-controller-manager, and an optional cloud-controller-manager — typically run as static pods across multiple control-plane nodes. Workers run the kubelet, kube-proxy, and a CRI runtime.

Crucially, upstream Kubernetes ships almost nothing above the core. You choose your CNI (Calico, Cilium, …), your ingress controller, your load-balancer (a cloud LB or MetalLB), your CSI storage driver, and your monitoring. (CoreDNS is the standard cluster DNS on both — kubeadm deploys it as a default add-on, just as k3s bundles it.) You stand the whole thing up with kubeadm (or kops/kubespray, or you let a cloud run the control plane for you with EKS/GKE/AKS). That’s more moving parts, a heavier footprint — budget ~2 GB RAM and 2 vCPUs for a control-plane node (etcd colocated, in kubeadm’s default stacked topology) — and a real operational surface. In exchange you get the thing k3s deliberately hides: every component is swappable and every knob is exposed, which is exactly what you need when you’re running many nodes, many teams, and an audit trail for each.

k3s packages the whole control plane, datastore, runtime and networking into one binary with bundled add-ons; vanilla k8s runs separate components against a mandatory etcd and leaves networking, ingress, load-balancing and storage for you to assemble Same Kubernetes API · both CNCF-conformant — the difference is packaging & operations, not capability k3s — one binary server & agent are the same process k8s — modular, assembled the control plane is yours to run k3s server — single process (< 100 MB) API server · scheduler · controller-manager Kine → SQLite (default) · embedded etcd for HA containerd · runc · flannel · kube-proxy the supervisor wraps it all on :6443 k3s agent — same binary, --agent kubelet · kube-proxy · flannel · containerd ↑ tunnels back to the server supervisor (websocket) Bundled & on by default CoreDNS · Traefik · ServiceLB · local-path · metrics-server any can be --disabled or swapped Install: curl … | sh · server ~2 GB · agent ~512 MB Control-plane node(s) kube-apiserver etcd ✱ required kube-scheduler controller-manager cloud-controller-manager (optional) separate binaries / static pods · you run etcd Worker node(s) kubelet · kube-proxy · container runtime (CRI) join with kubeadm / managed (EKS · GKE · AKS) You choose & operate CNI (Calico/Cilium) · Ingress · LoadBalancer · CSI · DNS nothing bundled — maximum control Install: kubeadm + your add-ons · ~2 GB+ control plane
Two packagings of the same Kubernetes. k3s folds the control plane, datastore (Kine → SQLite), runtime and CNI into one binary — server and agent are the same process — and ships working defaults for ingress, load-balancing and storage. Vanilla k8s runs those components separately against a mandatory etcd and hands you the rest to assemble and operate.

What’s in the box vs what you assemble

The single clearest way to feel the difference is to line up every layer a real cluster needs and ask “who provides it?” k3s answers “me, already running”; upstream k8s answers “you, your choice.” Neither answer is wrong — bundled defaults are a gift at the edge and a constraint at scale; assemble-it-yourself is friction on a Pi and freedom in a platform team.

For each cluster layer, k3s ships a bundled default while vanilla k8s makes you choose and operate it Layer k3s — bundled default k8s — bring your own Datastore SQLite via Kine (etcd optional) etcd — you run & back it up Networking flannel (VXLAN) pick a CNI: Calico / Cilium Ingress Traefik install one: ingress-nginx … LoadBalancer ServiceLB (Klipper) cloud LB / MetalLB Storage local-path-provisioner pick a CSI driver DNS CoreDNS CoreDNS (default add-on) Install one-line: curl … | sh kubeadm / kops / managed Every k3s default is replaceable; every k8s choice is also a thing you now operate.
The same six layers, two philosophies. k3s ships a working default for each (and lets you --disable any of them); vanilla k8s makes you pick and run each one — the friction you don't want on an edge box, and the control you do want in a platform.

Main differences

k3sk8s (vanilla / upstream)
What it isa CNCF-conformant distribution of Kubernetesupstream Kubernetes itself
Packagingone binary (< 100 MB), all components in-processseparate components, assembled
Default datastoreSQLite via Kine (etcd / external DB optional)etcd (mandatory, you operate it)
Footprintserver ~2 GB (leaner in practice, ~1.2–1.6 GB) · agent ~512 MB; great on ARM~2 GB / 2 vCPU per control-plane node (etcd colocated)
Networking / ingress / LB / storagebundled (flannel · Traefik · ServiceLB · local-path)you choose (Calico/Cilium · nginx · MetalLB · CSI)
Runtimecontainerd baked inany CRI (containerd, CRI-O) — you install it
Installcurl … | sh in secondskubeadm / kops / kubespray / managed
HAembedded etcd (≥ 3 servers) or external DBetcd quorum across control-plane nodes
Agent → control planewebsocket tunnel to the supervisor (no inbound ports)standard kubelet ↔ apiserver
Customisationopinionated defaults; swap by --disableeverything pluggable from the start
Best fitedge · IoT · ARM · CI · dev · small-to-mid prodlarge, multi-tenant, cloud-native at scale

↔ scroll the table sideways to see both columns.

Pros & cons

👍 Pros👎 Cons
k3stiny single binary; ~512 MB agent footprint (server leaner than upstream in practice); one-command install; batteries included; runs on ARM/edge; same conformant API; fast cluster bring-up; easy to embed/automateSQLite default doesn’t scale writes (swap for etcd at HA); opinionated bundled add-ons to replace if unwanted; fewer alpha/legacy/in-tree features; smaller (if growing) ecosystem of tutorials than upstream
k8sinfinitely modular — every component swappable; the reference platform with the deepest ecosystem & tooling; battle-tested at huge scale; first-class multi-tenancy & customization; managed offerings (EKS/GKE/AKS) remove control-plane opsheavy footprint (~2 GB+ control plane, incl. etcd); you assemble and operate CNI, ingress, LB, storage, etcd backups; steeper learning curve; slower to stand up; overkill for edge/small clusters

”I want to…” → which one

I want to…Reach forWhy
Run Kubernetes on a Raspberry Pi / ARM / edge gatewayk3ssingle binary, tiny on ARM, a worker runs in ~512 MB — a core design goal
Spin up a throwaway cluster in CI or local devk3s (or k3d)up in seconds, tear down just as fast
Run a 50-node, multi-team platform in the cloudk8syou need every knob, multi-tenancy, and deep customization
Use Cilium/Calico, MetalLB, a specific CSI by designk8supstream assumes you bring your own — full freedom
Ship an appliance / embedded product with K8s insidek3sone binary is trivial to package and update
Offload control-plane operations entirelymanaged k8sEKS/GKE/AKS run apiserver + etcd for you
Meet CIS/FIPS hardening on a self-run datacenter clusterRKE2Rancher’s hardened sibling — k3s ergonomics, upstream-aligned security
Avoid running and backing up etcd yourselfk3sSQLite-via-Kine by default; etcd only when you opt into HA

It’s the same Kubernetes — choose by weight, not capability

Re-read the tables and the trap dissolves: there is no capability axis here. Both are conformant Kubernetes; your manifests don’t care which one they land on. What actually differs is operational weight — footprint, moving parts, and how much of the cluster you personally run. So you don’t pick the “more powerful” one; you pick the one whose weight matches your hardware, your team, and your blast radius.

That makes it a spectrum, not a binary. The same project even gives you the rungs in between: k3s for lean and edge, RKE2 when you want k3s ergonomics with CIS/FIPS hardening in a datacenter, plain kubeadm when you want to assemble full upstream yourself, and managed (EKS/GKE/AKS) when you’d rather the cloud run the control plane.

Where each option fits along a spectrum from lean/edge to heavy/cloud-scale, with k3s, RKE2, kubeadm and managed Kubernetes as rungs Where each one fits k3s — edge · IoT · ARM · CI/dev · small-to-mid prod vanilla k8s — large · multi-tenant · cloud-native at scale lean · few nodes · low ops heavy · many nodes · max control → k3s one binary, batteries in edge / small / dev agent ~512 MB RKE2 k3s-style, hardened CIS / FIPS, etcd datacenter / gov kubeadm full upstream you assemble it you operate it managed EKS · GKE · AKS cloud runs the control plane
One spectrum, four rungs. k3s sits at the lean/edge end; RKE2 adds hardening with k3s ergonomics; kubeadm is full upstream you assemble; managed k8s hands the control-plane operations to a cloud. Pick by the weight you can carry, not by which is "more Kubernetes" — they all are.

The two anti-patterns are the two ways of mismatching weight to need. Don’t reach for full k8s on a three-node edge cluster “to be safe” — you’ll spend your life operating etcd, a CNI, and an ingress controller for a workload k3s would have run from a single binary. And don’t run k3s’s default SQLite under a hundred-node, write-heavy, multi-tenant production cluster — that’s the moment to move to embedded etcd (or RKE2/kubeadm) and the operational rigor that scale demands. Match the packaging to the place it runs.

TL;DR: k3s isn’t a lite alternative to Kubernetes — it is Kubernetes, CNCF-conformant, packaged as one binary under 100 MB with the control plane in-process, SQLite-via-Kine instead of mandatory etcd, and containerd · flannel · Traefik · ServiceLB · CoreDNS · local-path bundled and on by default. A worker/edge node runs in ~512 MB (the server, like upstream, wants ~2 GB but runs leaner, ~1.2–1.6 GB), it installs with one curl … \| sh, and is built for edge · IoT · ARM · CI · dev · small-to-mid prod (HA via embedded etcd or an external DB). Vanilla k8s is the same API delivered modular: separate components against a mandatory etcd, you choose and operate CNI/ingress/LB/storage, ~2 GB+ per control-plane node — the right call for large, multi-tenant, cloud-native clusters that need every knob (or buy it managed as EKS/GKE/AKS). The mnemonic: k3s pre-decides the operations; k8s leaves them to you. It’s a spectrum — k3s → RKE2 → kubeadm → managed — so choose by operational weight, not capability: both run your YAML unchanged.

Sources

← kubernetes