cld-toys › Guided exercises › Docker / containers

Namespaces in Isolation

One sh process, two PIDs. Inside the container it is PID 1 with two children and nothing else in the world; on the host it is PID 19555 among ~190 processes — and kill 19570 from outside removes a process the container cannot even name.


Concept

A container is not a box. There is no container object in the Linux kernel, no lightweight VM, nothing that "contains" anything. There is one ordinary process tree on one ordinary kernel, and a set of namespaces — per-process filters that change which entries of a kernel table that process is allowed to see. docker run is a clone() with a few extra flags.

This exercise makes that concrete. You'll take one sh process running in a container and look at it from two directions at once. Then you'll kill it from outside using the number it can't see, and finally build a PID namespace by hand with unshare — no Docker, no image, no daemon — to show there was never any magic.


Mental model: three vantage points, and eight kinds of blindness

The vantage points

On Linux, "inside the container" and "the host" are the only two places you can stand. On macOS there are three, and knowing which one you're standing on is most of the exercise:

┌──────────────────────────────────────────────────────────┐ │ macOS (Darwin, arm64) │ │ • no /proc at all — it isn't a Linux kernel │ │ • `docker` here is just a client talking to a socket │ │ │ │ ┌──────────────────────────────────────────────────┐ │ │ │ the Linux VM (6.12.x-linuxkit, aarch64) │ │ │ │ • THIS is "the host" every Docker doc means │ │ │ │ • PID 1 = initd, ~190 processes │ │ │ │ • initial namespaces: pid:[4026531836], ... │ │ │ │ │ │ │ │ ┌──────────────────────────────────┐ │ │ │ │ │ container ns-a │ │ │ │ │ │ • PID 1 = sh, 3 processes │ │ │ │ │ │ • pid:[4026533793] │ │ │ │ │ └──────────────────────────────────┘ │ │ │ └──────────────────────────────────────────────────┘ │ └──────────────────────────────────────────────────────────┘
Why the usual instruction can't be typed on a Mac "From the host, run ls /proc/<pid>/ns/" is impossible here — /proc does not exist on macOS. The way in is a container that opts out of the PID namespace: docker run --privileged --pid=host. That's not a workaround for a missing feature; it is the same mechanism this exercise is about, used in reverse. A container is only isolated because of the namespaces it was given, so hand one back and the "container" is a shell on the host.

If you're on native Linux, everything below works identically — just use your own shell wherever the exercise says Session B, and drop the --pid=host container. The /proc paths, inode numbers, NSpid field and unshare behavior are all kernel features, not Docker ones.

The eight namespaces

ls -l /proc/<pid>/ns/ lists them. Each is a kernel object identified by an inode number; two processes are "in the same namespace" iff they show the same inode. That's the whole ontology.

NamespaceWhat it filtersWhat breaks without itFlag to disable
pid The process ID table. Your first process becomes PID 1; you cannot see or signal anything outside. ps aux in the container lists every host process, and kill reaches them. --pid=host, or --pid=container:NAME
mnt The mount table. This is what makes the image's filesystem be /. The container sees the host's filesystem tree. no flag; you'd have to nsenter
net Interfaces, addresses, routes, iptables, socket port space. Two containers can't both bind :80. --network host
uts hostname and domainname, nothing else. The oldest and smallest one (UTS = UNIX Time-sharing System). hostname foo in a container renames the host. --uts=host
ipc SysV shared memory / semaphores / message queues, and POSIX message queues. Two Postgres containers with the same shm key collide. --ipc=host, --ipc=container:NAME
user UID/GID mapping. Lets uid 0 inside be uid 100000 outside. Root in the container is root on the host — which is the default. on by default; see Part 6
cgroup The cgroup path you can see, so a container can't read its position in the host's hierarchy. /proc/self/cgroup leaks the container ID and the host's layout. --cgroupns=host
time CLOCK_MONOTONIC and CLOCK_BOOTTIME offsets. Newest (Linux 5.6), and the only one Docker gives you with no offset applied. Checkpoint/restore can't rewind a process's idea of uptime. n/a
Two properties to have in your head going in

A namespace is a filter, not a wall. It changes what a process can name. It does not move the process anywhere, does not stop the host from acting on it, and does not limit what it can consume — that last one is cgroups' job, a completely separate mechanism.

The isolation is per-namespace-type and individually revocable. There is no "container namespace". --pid=container:ns-a shares exactly one of the eight and leaves the other seven alone, which is precisely how a Kubernetes pod's sidecars work.


Setup

Three terminals. A · macOS is your Mac shell, B · Linux VM is a shell on the "host", C · container is a shell inside the container.

A · macOS — start the container under test. It runs a shell with two identifiable children, so there's a small process tree to look at:

docker run -d --name ns-a alpine sh -c 'sleep 4242 & sleep 4243 & wait'

B · Linux VM--pid=host opts out of the PID namespace; --privileged keeps the kernel from redacting other processes' /proc entries:

docker run --rm -it --privileged --pid=host --cgroupns=host alpine sh

C · container:

docker exec -it ns-a sh
Your inode numbers will differ Every inode below (4026533793 and friends) will be different on your machine — the kernel hands them out sequentially and recycles them. What matters is which numbers match each other, never their values. One exception: 4026531836 and its neighbours in the 402653183x range are the kernel's initial namespaces, created at boot, and are the same on every Linux system.

Part 1 — the same processes, twice

What this part tests: whether "the container's processes" and "the host's processes" are two sets of processes or one. Per the mental model there is only ever one process tree, so the two views must be the same processes under different names.

Step 1 · the container's world
C · container
ps -o pid,ppid,args
PID PPID COMMAND 1 0 sh -c sleep 4242 & sleep 4243 & wait 7 1 sleep 4242 8 1 sleep 4243 9 0 ps -o pid,ppid,args

Four processes, and PID 1 is your sh. Note PPID 0 — PID 1's parent isn't merely hidden, it is unrepresentable, because it lives in a namespace this process has no way to name.

Step 2 · ask the daemon instead
A · macOS
docker top ns-a
Predict: docker top asks the daemon what's running in the container. Same list? Click to check.
C · inside the container
PID PPID COMMAND 1 0 sh -c sleep 4242 ... 7 1 sleep 4242 8 1 sleep 4243
A · docker top ns-a
UID PID PPID CMD root 19555 19532 sh -c sleep 4242 ... root 19569 19555 sleep 4242 root 19570 19555 sleep 4243

The same three commands, with completely different PIDs — and PID 1's phantom parent has a number now, 19532. docker inspect -f '{{.State.Pid}}' ns-a returns 19555 to match. This is already the aha in miniature: 1 and 19555 are the same process, and neither number is more real than the other.

Step 3 · what every Docker tutorial tells you to do next
A · macOS
ls /proc/19555/ns/
ls: /proc: No such file or directory

Not "permission denied" — macOS has no /proc at all. The PID docker top just printed is not a PID in any table macOS maintains. That's what Session B is for.


Part 2 — a namespace is an inode

What this part tests: the claim from the mental model that "same namespace" is literally "same inode number". If the container's sh and the VM's PID 19555 are one process, their namespace inodes must be identical — and both must differ from the VM's own.

Step 4 · the namespaces, from the inside
C · container
ls -l /proc/1/ns/
lrwxrwxrwx 1 root root 0 Jul 22 15:58 cgroup -> cgroup:[4026533794] lrwxrwxrwx 1 root root 0 Jul 22 15:58 ipc -> ipc:[4026533792] lrwxrwxrwx 1 root root 0 Jul 22 15:58 mnt -> mnt:[4026533790] lrwxrwxrwx 1 root root 0 Jul 22 15:58 net -> net:[4026533795] lrwxrwxrwx 1 root root 0 Jul 22 15:58 pid -> pid:[4026533793] lrwxrwxrwx 1 root root 0 Jul 22 15:58 pid_for_children -> pid:[4026533793] lrwxrwxrwx 1 root root 0 Jul 22 15:58 time -> time:[4026534170] lrwxrwxrwx 1 root root 0 Jul 22 15:58 time_for_children -> time:[4026534170] lrwxrwxrwx 1 root root 0 Jul 22 15:58 user -> user:[4026531837] lrwxrwxrwx 1 root root 0 Jul 22 15:58 uts -> uts:[4026533791]
Step 5 · the same process, from the VM
B · Linux VM
ls -l /proc/19555/ns/
Predict: do the inodes match what the container reported, or does each side see its own private numbering? Click to check.
lrwxrwxrwx 1 root root 0 Jul 22 15:58 cgroup -> cgroup:[4026533794] lrwxrwxrwx 1 root root 0 Jul 22 15:58 ipc -> ipc:[4026533792] lrwxrwxrwx 1 root root 0 Jul 22 15:58 mnt -> mnt:[4026533790] lrwxrwxrwx 1 root root 0 Jul 22 15:58 net -> net:[4026533795] lrwxrwxrwx 1 root root 0 Jul 22 15:58 pid -> pid:[4026533793] lrwxrwxrwx 1 root root 0 Jul 22 15:58 pid_for_children -> pid:[4026533793] lrwxrwxrwx 1 root root 0 Jul 22 15:58 time -> time:[4026534170] lrwxrwxrwx 1 root root 0 Jul 22 15:58 time_for_children -> time:[4026534170] lrwxrwxrwx 1 root root 0 Jul 22 15:58 user -> user:[4026531837] lrwxrwxrwx 1 root root 0 Jul 22 15:58 uts -> uts:[4026533791]

Byte-for-byte the same list. The namespace inode is a global fact about the process, not a per-viewer one. It has to be: it's how the kernel answers "are these two processes in the same namespace?"

Step 6 · the VM's own init, for contrast
B · Linux VM
ls -l /proc/1/ns/
cat /proc/1/comm
cgroup -> cgroup:[4026531835] ipc -> ipc:[4026531839] mnt -> mnt:[4026531841] net -> net:[4026531840] pid -> pid:[4026531836] time -> time:[4026531834] user -> user:[4026531837] uts -> uts:[4026531838] initd

40265318344026531841 is the kernel's initial set, allocated at boot; you'll see those exact numbers on any Linux box. Nine of the ten lines differ from the container's. The tenth — user:[4026531837] — is identical, and Part 6 is about why that one matters more than the other nine.

The same process, asked two waysC · inside ns-aB · in the Linux VM
Its PID119555
Its parent0 (unnameable)19532 (containerd-shim)
Processes in view4~193
pid namespacepid:[4026533793] — identical from both sides
user namespaceuser:[4026531837] — and the VM's init has the same one
Its uiduid=0(root)root, Uid: 0 0 0 0
/proc/self/cgroup0::/0::/docker/4700e65f5d12…
Hostname4700e65f5d12docker-desktop
Step 7 · the single line that states the whole exercise
B · Linux VM
grep -E '^(Name|Pid|NSpid|Uid):' /proc/19555/status
Name: sh Pid: 19555 Uid: 0 0 0 0 NSpid: 19555 1

NSpid: 19555 1 — one process, listed once per PID namespace it is visible in, outermost first. Not "the host PID and the mapped PID": the kernel genuinely holds both, because PID namespaces nest, and a process gets an ID in every ancestor namespace. Nesting is why the direction is asymmetric — an outer namespace can always name an inner process, never the reverse.

Step 8 · the parent the container can't see
B · Linux VM
ps -o pid,args | grep '[c]ontainerd-shim' | head -2
488 /usr/bin/containerd-shim-runc-v2 -namespace moby -id 14fcb7e6... -address /run/containerd/containerd.sock 15673 /usr/bin/containerd-shim-runc-v2 -namespace moby -id c1711bd6... -address /run/containerd/containerd.sock

One shim per container, and 19555's parent 19532 is one of them. "Ordinary process supervised by an ordinary parent" is the entire architecture; the shim is what lets dockerd be restarted without killing your containers.


Part 3 — the isolation is a view, not a wall

What this part tests: the first bullet of the mental model. A namespace controls what a process can name. It does not put the process out of reach of anything outside.

Step 9 · signal outward
C · container — try to signal the process you now know is PID 19555:
kill 19555
sh: can't kill pid 19555: No such process

Not "permission denied" — no such process. In this namespace that integer names nothing, and the kernel's answer is a perfectly honest ESRCH.

Step 10 · signal inward
B · Linux VM — kill sleep 4243, whose in-container name is PID 8:
kill 19570
(no output)
Predict: does the container's PID namespace protect it? Run ps -o pid,args in C to check.
C · container
PID COMMAND 1 sh -c sleep 4242 & sleep 4243 & wait 7 sleep 4242 71 ps -o pid,args

PID 8 is gone. The container has no way to name 19570 and no way to refuse it either. Isolation runs strictly outward: hierarchical, and one-way.

Step 11 · and the filesystem is just as porous from outside
B · Linux VM/proc/<pid>/root is the kernel's door into another process's mount view:
ls /proc/19555/root/
cat /proc/19555/root/etc/hostname
bin dev etc home lib media mnt opt proc root run sbin srv sys tmp usr var 4700e65f5d12

That's the container's own root filesystem, read from outside with no docker exec involved — and 4700e65f5d12 is the container ID, which Docker writes into /etc/hostname at start. This is how host-level tooling (backup agents, security scanners, docker cp) reaches into a container without entering it.


Part 4 — joining one namespace and not the others

What this part tests: the second bullet of the mental model — that the eight namespaces are independent, and "join a container" is not an atomic thing you do.

Step 12 · a new container in an old PID namespace
A · macOS
docker run --rm --pid=container:ns-a alpine ps -o pid,args
Predict: this is a brand-new container that shares ns-a's PID namespace. Whose processes does its ps list, and whose files does its ls see? Click to check.
PID COMMAND 1 sh -c sleep 4242 & sleep 4243 & wait 7 sleep 4242 77 ps -o pid,args

A separate container, a separate image, its own writable layer — and it can see ns-a's process tree with ns-a's numbering. Check what it did not inherit:

pid ns: pid:[4026533793] <- ns-a's, shared mnt ns: mnt:[4026534595] <- its own net ns: net:[4026534599] <- its own $ cat /etc/hostname e0d11827af34 <- its own uts ns; ns-a is 4700e65f5d12

One namespace shared, seven fresh. This is exactly a Kubernetes pod: containers in a pod share the net (and optionally pid) namespace so they can reach each other on localhost and see each other's processes, while keeping separate filesystems.

Step 13 · the same thing without Docker
B · Linux VMnsenter is the setns(2) syscall with a CLI. Enter 19555's pid and mount namespaces:
nsenter -t 19555 -p -m ps -o pid,args
PID COMMAND 1 sh -c sleep 4242 & sleep 4243 & wait 7 sleep 4242 93 ps -o pid,args

That is docker exec, in one command, with no daemon in the loop.

Step 14 · drop the mount namespace
B · Linux VM
nsenter -t 19555 -p ps -o pid,args
Predict: you're still joining the container's PID namespace, just keeping the VM's mount table. ps should still be filtered to the container, right? Click to check.
PID COMMAND 1 /initd 2 [kthreadd] 3 [pool_workqueue_] ... (194 lines)

The VM's entire process list, from a process that really is in the container's PID namespace. ps doesn't ask the kernel who exists — it reads /proc, and /proc is a mount, still the VM's. The PID namespace changed which processes the kernel would report; nobody changed which /proc was being read. Remember this for Part 7.


Part 5 — the other namespaces, side by side

What this part tests: that the remaining namespace types work the same way, and that "disabled" and "empty" are different states.

Step 15 · a container with no network
A · macOS
docker run -d --name ns-b --network none --hostname ns-b-box alpine sleep 3600
docker exec ns-b ls -l /proc/self/ns/
docker exec ns-b ip -o addr show
Predict: --network none. Does ns-b get no network namespace, or an empty one? Click to check.
net -> net:[4026534330] <- ns-a is 4026533795, the VM is 4026531840 uts -> uts:[4026534326] user -> user:[4026531837] <- same as everyone 1: lo inet 127.0.0.1/8 scope host lo 1: lo inet6 ::1/128 scope host

Its own private network namespace, containing a loopback interface and nothing else. --network none doesn't remove the namespace, it declines to plug anything into it — every namespace exists in exactly one state, and "empty" is a normal one. Compare ns-a, which has the same kind of namespace with a veth in it:

11: eth0 inet 172.17.0.3/16 brd 172.17.255.255 scope global eth0

And the hostnames — ns-b-box vs ns-a's 4700e65f5d12 — differ because of one namespace whose entire contents is a string.

Step 16 · the exception that sharpens the rule
A · macOS — the UTS namespace is the container's own. Try writing to it:
docker run --rm alpine sh -c 'hostname changed-me; hostname'
docker run --rm --cap-add SYS_ADMIN alpine sh -c 'hostname changed-me; hostname'
hostname: sethostname: Operation not permitted 3ac7a388ecf0 changed-me

Namespaces decide which hostname you affect; capabilities decide whether you may set one at all. Docker drops CAP_SYS_ADMIN by default, so the answer is no — two independent mechanisms, and real container security is always both.

Step 17 · the opposite end: --network host
A · macOS
docker run --rm --network host alpine sh -c \
  'echo "net ns: $(readlink /proc/self/ns/net)"; ip -o addr show | head -4'
net ns: net:[4026531840] 1: lo inet 127.0.0.1/8 scope host lo 4: eth0 inet 192.168.65.3/24 brd 192.168.65.255 scope global eth0

4026531840 is the initial namespace from step 6 — not an empty namespace, the VM's own. Note what "host" means here on a Mac: 192.168.65.3 is the VM's address, not your Mac's. A third reminder that "host" is the middle box in the diagram.

Step 18 · the cgroup namespace is pure cartography
A · macOS then B · Linux VM — same cgroup, two different paths:
docker exec ns-a cat /proc/self/cgroup     # A
cat /proc/19555/cgroup                     # B
0::/ 0::/docker/4700e65f5d12a9cc0d7cba22d7082afafe44e3f57d2e889d2df91906bf12b7fd

The container believes it's at the root of the hierarchy. Nothing about its limits changed; only the path it's allowed to see did. (Your B shell needs --cgroupns=host for this, or it gets its own relative view and prints 0::/../4700e65….)


Part 6 — the namespace you didn't get

What this part tests: the one line in step 6's output that was identical between the container and the VM's init. Seven namespaces were unshared for you; the security-critical one was not.

Step 19 · a container running as uid 1000
A · macOS
docker run -d --name ns-c --user 1000 alpine sleep 5150
docker exec ns-a id
docker exec ns-c id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),... uid=1000 gid=0(root) groups=0(root)
Predict: ns-a is root inside, ns-c is uid 1000 inside. What UIDs does the VM report for them? Click to check.
B · Linux VM
ps -o pid,user,args | grep -E '[s]leep 5150|[s]leep 4242'
grep '^Uid:' /proc/19555/status
19555 root sh -c sleep 4242 & sleep 4243 & wait 19569 root sleep 4242 20438 1000 sleep 5150 Uid: 0 0 0 0

No translation whatsoever. The container's root is the host's root — the same uid 0, the same credential the kernel checks on every file access. user:[4026531837] was the initial user namespace, and Docker left every container in it.

This is why "container escape" is a short sentence With uid 0 unmapped, a process that gets out of its other seven namespaces is root on the host. It's also why --privileged --pid=host in Session B handed you the whole VM — nothing had to be defeated. And why --user 1000 is the cheapest hardening step available: 1000 outside is a nobody. The real fix Linux offers is a user namespace with a UID map, so uid 0 inside is uid 100000 outside — Docker can do it (dockerd --userns-remap=default), Podman does it by default for rootless containers, and both pay for it in volume-permission pain, which is roughly why it isn't the default here.

Part 7 — none of this was Docker

What this part tests: the claim in the concept — that docker run is clone() with flags, and every namespace above is a kernel feature you can reach directly.

Step 20 · establish where you're standing
B · Linux VM
readlink /proc/self/ns/pid
ps -o pid,args | wc -l
pid:[4026531836] 197

The initial PID namespace, 196 processes plus a header.

Step 21 · a container in one command, no Docker
B · Linux VM
unshare --pid --fork --mount-proc sh -c 'readlink /proc/self/ns/pid; ps -o pid,args'
Predict: no image, no daemon, no docker anywhere. What does ps print inside it? Click to check.
pid:[4026534596] PID COMMAND 1 ps -o pid,args

A brand-new PID namespace and a world containing one process. That's the entire trick a container plays, in a busybox one-liner. Docker's remaining value-add is images, networking, and lifecycle management — not isolation, which is 30 characters of unshare.

Step 22 · now drop --mount-proc
B · Linux VM
unshare --pid --fork sh -c 'echo "pid ns: $(readlink /proc/self/ns/pid)"; echo "my pid: $$"; ps -o pid,args | wc -l; ps -o pid,args | head -3'
Predict: you're still PID 1 in a brand-new PID namespace — only the /proc mount is different. What does ps show? Click to check.
pid ns: pid:[4026534595] my pid: 1 198 PID COMMAND 1 /initd 2 [kthreadd]

PID 1 that can see 198 processes. Two different processes are both "PID 1" in that listing — the shell (correctly, in its new namespace) and initd (as read from a /proc mounted in the old one). The namespace is completely real; ps is reading a stale filesystem. --mount-proc is shorthand for "also unshare the mount namespace and mount a fresh /proc", which is why every container gets a mount namespace whether or not it wants a private filesystem. Step 14 was the same bug from the other direction.

Step 23 · the smallest namespace of all
B · Linux VM
hostname
unshare --uts sh -c 'hostname ns-by-hand; hostname'
hostname
d5c398ed68ff ns-by-hand d5c398ed68ff

A namespace whose whole content is one string, unshared and modified and discarded in one line, with the outer hostname untouched. (This works because B is --privileged and therefore holds CAP_SYS_ADMIN — in an ordinary container it fails, exactly as in step 16.)


What you should see

One sh process that is PID 1 with two children when it looks at itself, and PID 19555 with a containerd-shim parent among ~190 processes when the VM looks at it — with NSpid: 19555 1 in /proc/19555/status recording both facts on one line. Identical namespace inodes (pid:[4026533793]) read from either side, all differing from the VM's initial pid:[4026531836] — except user:[4026531837], which matches. kill 19555 from inside failing with No such process, while kill 19570 from outside silently removes the container's PID 8. A second container joining exactly one of ns-a's eight namespaces and keeping its own hostname. And unshare --pid --fork --mount-proc producing a one-process world with no Docker involved — plus, without --mount-proc, a PID 1 that still lists 198 processes.


Why

Because namespaces virtualize names, not resources. A PID namespace gives the kernel a second table in which to register your process, and /proc shows a process only the table entries reachable from its own. So the container's ps is not lying and the VM's ps is not privileged information — they're two indexes into one list of tasks, and NSpid is the row that holds both keys.

That design forces the asymmetry you saw in Part 3. PID namespaces form a tree; a task gets an ID in its own namespace and in every ancestor. Naming therefore works outward-in only, and permission checks are unaffected by namespaces at all — kill 19570 succeeded because the caller was uid 0 and the target was uid 0, exactly as it would have with no containers in the picture. Nothing consulted the boundary.

It also explains why isolation has to be assembled rather than switched on. Each namespace type is a separate kernel object with a separate CLONE_NEW* flag, and docker run sets seven of them. The eighth, user, it leaves alone — so the uid 0 in your container is the uid 0 that owns the VM. Every practical container-security control (--user, seccomp, dropped capabilities, read-only rootfs, userns-remap) exists to shore up that gap, because namespaces were built to make processes not see each other, which is a different goal from making them safe from each other.

The shape of every container bug you'll hit The --mount-proc surprise in Part 7 is the general case: the kernel state changed, and something that caches or re-reads the old view didn't. Same reason a process that had a file open before the mount namespace was unshared keeps reading it, and the same reason /proc/<pid>/root in Part 3 works at all.

Go deeper

Sources: man 7 namespaces — the authoritative list, including the NSpid field and the /proc/[pid]/ns/ semantics · man 7 pid_namespaces · man 7 user_namespaces · Docker: isolate containers with a user namespace


Cleanup

Only the containers this exercise created, by name — never prune:

docker rm -f ns-a ns-b ns-c

Then exit the B shell (--rm removes it).