Articles on DevOps, infrastructure engineering, CI/CD, and production systems by Yasas Banuka Malavige. https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog en-US Fri, 11 Sep 2026 06:11:54 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) ybmalavige@gmail.com (Yasas Banuka Malavige) https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-6-security https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-6-security Tue, 14 Jul 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker security devops containers linux The fix is two lines in your Dockerfile: ```dockerfile FROM node:22-alpine WORKDIR /app COPY package.json . RUN npm install COPY . . # Create a dedicated user and switch to it RUN addgroup --system --gid 1001 appgroup && \ adduser --system --uid 1001 --ingroup appgroup appuser # Give ownership of the app directory to the new user COPY --chown=appuser:appgroup . . # Never run as root in production USER appuser CMD ["node", "server.js"] ``` Or in your Compose file for third-party images: ```yaml services: nginx: image: nginx:alpine user: "1001:1001" ``` Non-root users can't bind to ports below 1024. If your app needs port 80, run it on port 3000 internally and let nginx handle port 80 on the outside — or add `CAP_NET_BIND_SERVICE` capability specifically for that purpose. --- ## Layer 2 - Linux Capabilities: Root Isn't One Thing Most developers think of "root" as a binary state, you either have it or you don't. The reality is more nuanced, and understanding it gives you much more precise control. Linux splits root's powers into approximately 40 separate **capabilities**. Each one is an independent permission that can be granted or revoked. Docker gives containers a limited but still generous set of capabilities by default. The right approach is to drop everything and add back only exactly what your app needs: ```yaml services: backend: security_opt: - no-new-privileges:true # processes can't gain new privileges cap_drop: - ALL # strip every capability cap_add: - NET_BIND_SERVICE # add back only what's needed ``` `no-new-privileges` is the important one most people miss. It prevents any process inside the container from gaining new privileges through setuid binaries, even if root was already dropped, this closes a remaining escalation path. Never add `SYS_ADMIN` unless you have a very specific and understood reason. It grants such broad privileges that it essentially undoes everything else you've done. And never use `--privileged` in production — a privileged container has full access to the host. It completely destroys container isolation. --- ## Think About It This command lists every running container and shows whether it's running as root: ```bash docker ps -q | xargs -I{} docker inspect \ --format '{{.Name}} → User: {{.Config.User}}' {} ``` Any container with a blank `User` field is running as root. Run it on your production server. Count the blanks. For most setups that have never thought about this, the number will be higher than expected. That's not a crisis, it's a starting point. Each blank is a container to harden on your next deploy. --- ## Layer 3 - The Image Problem You Can't See Your application code might be perfectly written. Your Dockerfile might be clean and minimal. But you're still inheriting whatever vulnerabilities exist in the base image you started with. This is the **supply chain problem** in containers. When you write `FROM node:22-alpine`, you're trusting that image's contents. And that image was built weeks or months ago. Libraries have been patched on the open internet since then. The image hasn't been. **Practice 1 - Scan your images regularly.** Docker Scout is built directly into Docker for this: ```bash docker scout cves myapp/backend:1.0.0 ``` Output: ``` ✗ CRITICAL CVE-2024-XXXXX libssl 3.0.1 ✗ HIGH CVE-2023-XXXXX libcurl 7.88.0 ✓ No vulnerabilities in application layer ``` The critical finding is almost never your code. It's almost always a library in the base OS. The fix is simple: rebuild with a newer base image. **Practice 2 - Rebuild base images regularly.** A Docker image built 6 months ago with `FROM node:22-alpine` is not the same as one built today with the same instruction. The tag is the same but Docker Hub has updated what that tag points to. Libraries are patched. CVEs are fixed. Add a weekly scheduled rebuild to your CI/CD pipeline: ```yaml # .github/workflows/weekly-rebuild.yml on: schedule: - cron: '0 2 * * 1' # every Monday at 2am jobs: rebuild: steps: - uses: docker/build-push-action@v5 with: push: true tags: myapp/backend:latest no-cache: true # force fresh pull of base image ``` Your app code didn't change. But your image is now built on a base that has this week's security patches. Automated security with zero developer effort. --- ## Layer 4 - Secrets: The Most Common Mistake If this is in your compose file, that's a problem: ```yaml environment: DB_PASSWORD: mypassword123 JWT_SECRET: supersecretkey API_KEY: sk-1234567890abcdef ``` And if that compose file is in a git repository, even a private one, those credentials have now been committed to version history. Deleting them later doesn't remove them from git history. Anyone with access to the repository, present or future, can find them. The correct pattern has two levels depending on your security requirements. **Level 1 - Environment variables from a server-only `.env` file:** ```yaml # docker-compose.yml (safe to commit) environment: DB_PASSWORD: ${DB_PASSWORD} JWT_SECRET: ${JWT_SECRET} ``` ```bash # .env - lives only on the server, never committed DB_PASSWORD=your-actual-password JWT_SECRET=your-actual-secret ``` Add `.env` to `.gitignore`. The compose file is safe to push. The secrets never leave the server. **Level 2 - Docker secrets (for higher security requirements):** ```yaml services: backend: secrets: - db_password - jwt_secret secrets: db_password: file: ./secrets/db_password.txt jwt_secret: file: ./secrets/jwt_secret.txt ``` Docker secrets mount as files at `/run/secrets/` inside the container. They're stored in memory, never written to the container's filesystem, and never visible in `docker inspect`. Your app reads the secret by reading a file rather than an environment variable. ```java // reading a Docker secret in Java String dbPassword = Files.readString(Path.of("/run/secrets/db_password")).strip(); ``` The advantage over environment variables: environment variables can leak through crash reports, debug output, and third-party monitoring tools that dump process environment. Files at `/run/secrets/` don't have that exposure surface. --- ## Layer 5 - Lock the Filesystem If your application doesn't need to write to its own filesystem and most web apps don't, they write to databases and object storage - lock it: ```yaml services: frontend: image: nginx:alpine read_only: true tmpfs: - /tmp - /var/cache/nginx - /var/run ``` `read_only: true` makes the entire container filesystem immutable. An attacker who exploits your application and tries to install tools, drop scripts, or modify binaries can't. The filesystem rejects every write. `tmpfs` mounts give specific directories write access in RAM only - no disk persistence. nginx needs to write cache and PID files; those go to RAM and disappear when the container stops. For your backend containers that write to the database but not to local disk: ```yaml services: backend: read_only: true tmpfs: - /tmp # Java and Spring Boot need temp space volumes: - uploads:/app/uploads # only mount what genuinely needs writes ``` This pattern - read-only root with minimal tmpfs exceptions is one of the highest-impact security improvements you can make with almost no downside. --- ## Layer 6 - seccomp This one requires almost no work from you because Docker does it automatically but it's worth understanding because it's quietly protecting every container you run. **seccomp** (secure computing mode) is a Linux kernel feature that lets you define which system calls a process is allowed to make. There are over 300 syscalls in Linux. Most apps use fewer than 50. The rest - `reboot`, `mount`, `ptrace`, `init_module` — have no business being called from inside a container. Docker ships with a default seccomp profile that blocks about 44 dangerous syscalls automatically, on every container, without any configuration needed from you. For most applications this default profile is more than sufficient. For highly sensitive workloads you can apply stricter custom profiles: ```yaml services: backend: security_opt: - seccomp:/path/to/strict-profile.json ``` If someone tells you to run a container with `--security-opt seccomp=unconfined`, they're disabling this protection entirely. Don't do it unless you have a very specific and understood reason. --- ## The Security Checklist - Before Every Production Deploy **Image security:** - Base image version is pinned (not `:latest`) - Image has been scanned (`docker scout cves`) - Multi-stage build used to minimize image contents - `.dockerignore` excludes `.env`, `.git`, credentials **Runtime security:** - `USER` instruction set to non-root in Dockerfile - `no-new-privileges: true` in Compose `security_opt` - `cap_drop: ALL` with only necessary caps added back - `read_only: true` where the app doesn't write to filesystem **Secrets:** - No credentials hardcoded in Dockerfile or Compose file - `.env` file in `.gitignore` - Secrets use `${VARIABLE}` with no fallback defaults **Network:** - Databases bound to `127.0.0.1`, not `0.0.0.0` - Only ports that need public access are published - Services use custom named networks, not default bridge **Ongoing:** - Weekly image rebuilds scheduled in CI/CD - Automated scanning in CI/CD pipeline --- ## The Mindset Shift That Makes This Stick It's easy to skip because the consequences aren't immediate. Your container runs fine without any of these practices. Until it doesn't. Tesla's cloud was mining cryptocurrency for someone else for weeks before anyone noticed. Because default configurations are designed for convenience, not security. And convenience is what ships. The question isn't whether to secure your setup. It's whether you do it before or after something goes wrong. Every item on that checklist above takes minutes to implement. Together they reduce your attack surface dramatically - not to zero, but to a place where an attacker has to work significantly harder to do real damage. --- ## What's Next The final article in this series ties everything together: **CI/CD with Docker** - how to build, scan, tag, and deploy images automatically on every code push, with zero manual steps. If you've been doing any part of your Docker workflow manually, that article will show you how to automate the whole thing end-to-end. *What's the most surprising Docker security gap you've discovered in a project you were working on? Drop it in the comments.* ]]> https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-5-docker-compose https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-5-docker-compose Thu, 25 Jun 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker compose devops containers infrastructure If you could type fast enough, you could recreate everything Compose does manually. Compose just does it reliably, in order, every time, from one file. The one thing Compose adds that plain `docker run` doesn't: it automatically creates a **dedicated network for your project** and connects every service to it. That's why your backend can reach your database by typing `postgres:5432`, they're both on the same custom network, and Docker's embedded DNS resolves service names automatically. Now let's read an actual file and understand every line. --- ## Reading a Real Compose File - Line by Line Here's a realistic production-style compose file. The kind you've probably seen but maybe not fully understood: ```yaml services: postgres: image: postgres:16-alpine container_name: myapp-postgres restart: unless-stopped environment: POSTGRES_USER: ${DB_USER} POSTGRES_PASSWORD: ${DB_PASSWORD} POSTGRES_DB: myapp_db volumes: - postgres-data:/var/lib/postgresql/data ports: - "127.0.0.1:5432:5432" healthcheck: test: ["CMD-SHELL", "pg_isready -U ${DB_USER}"] interval: 10s timeout: 5s retries: 5 start_period: 10s backend: image: myapp/backend:1.2.0 container_name: myapp-backend restart: unless-stopped depends_on: postgres: condition: service_healthy environment: DB_URL: jdbc:postgresql://postgres:5432/myapp_db DB_USER: ${DB_USER} DB_PASSWORD: ${DB_PASSWORD} JWT_SECRET: ${JWT_SECRET} ports: - "127.0.0.1:8080:8080" volumes: postgres-data: ``` Let's go through every pattern here. --- ### `restart: unless-stopped` ```yaml restart: unless-stopped ``` Four options exist. Here's what each actually means: - `no` - never restart. Container dies, it stays dead. - `always` - restart no matter what, even if you manually stopped it, even after a server reboot. - `on-failure` - restart only if the container exits with an error code. - `unless-stopped` - restart always **except** if you deliberately stopped it. `unless-stopped` is the right choice for production services. Your postgres crashes at 3am, it restarts automatically. But when you run `docker compose stop` for planned maintenance, it stays stopped. `always` sounds better but it's actually annoying. If you stop a container intentionally to debug something and the server reboots, `always` starts it again automatically. `unless-stopped` respects your intentions. --- ### The environment variable patterns - three very different behaviors ```yaml environment: POSTGRES_USER: ${DB_USER} # pattern 1 DB_PASSWORD: ${DB_PASSWORD} # pattern 1 JWT_SECRET: ${JWT_SECRET:?JWT_SECRET must be set} # pattern 2 NODE_ENV: production # pattern 3 ``` **Pattern 1** - `${VARIABLE}` with no fallback. If `DB_USER` isn't set in your `.env` file or shell environment, Compose substitutes an empty string. Your app starts with a blank database user. Silent failure. **Pattern 2** - same syntax, but for secrets like `JWT_SECRET` this is actually the right choice. You want it to fail loudly if the secret is missing, not silently use an empty value. Pair this with `${JWT_SECRET:?JWT_SECRET must be set}` if you want Compose to outright refuse to start when it's missing. **Pattern 3** - hardcoded value. Fine for things that genuinely don't change between environments. The `.env` file lives in the same folder as your compose file. Compose loads it automatically. It never gets committed to git, that's the whole point. Your compose file is safe to push to GitHub. The actual secrets live only on the server. --- ### The `depends_on` trap that has crashed real production systems This is the most misunderstood thing in Docker Compose. And it has caused real outages. ```yaml # what most people write backend: depends_on: - postgres ``` This looks right. Backend won't start until postgres starts. But "postgres starts" means the **container** starts, not that postgres is **ready to accept connections**. PostgreSQL takes several seconds to initialize after the container starts. Your backend starts, immediately tries to connect, postgres is still warming up so connection refused, crash. On a fast developer laptop this sometimes works by luck, postgres initializes quickly enough. On a slower CI server or a fresh production deployment, it fails every time. One developer documented this exact scenario in a legacy project where the database initialization script had to import 500MB of test data, that process alone took 40 seconds. With basic `depends_on`, the application container crashed and restarted at least 5 times before it could connect. The fix is two parts working together: ```yaml # part 1: teach postgres to report when it's actually ready postgres: healthcheck: test: ["CMD-SHELL", "pg_isready -U myuser -d mydb"] interval: 10s timeout: 5s retries: 5 start_period: 10s # give postgres grace time before checks begin # part 2: make backend wait for that signal backend: depends_on: postgres: condition: service_healthy # not just "started" — actually healthy ``` `pg_isready` is a built-in postgres utility that returns success only when the database is genuinely accepting connections. `start_period` gives postgres a grace period before healthcheck failures start counting, crucial because postgres always looks unhealthy for the first few seconds while it's initializing. This one pattern eliminates an entire class of production startup failures. --- ### `127.0.0.1:5432:5432` - the security detail hiding in the ports ```yaml ports: - "127.0.0.1:5432:5432" # secure - "5432:5432" # not secure ``` The difference is one IP address prefix, but the security implication is significant. Without `127.0.0.1`, Docker publishes the port on all network interfaces, including your server's public IP. Anyone on the internet who knows your server's IP can attempt to connect to your postgres database. Your firewall probably blocks it, but Docker's iptables rules can bypass firewall rules entirely. With `127.0.0.1`, Docker only creates a NAT rule for loopback traffic. The port is only reachable from processes running on the same machine. Your nginx reverse proxy on the host can reach it. The internet cannot. Databases, admin panels, internal APIs - anything that shouldn't be publicly accessible should always bind to `127.0.0.1`. Your frontend nginx is the only thing that earns a public port. --- ## Think About It Look at this setup: ```yaml services: backend: environment: DB_URL: jdbc:postgresql://localhost:5432/mydb ``` The backend is trying to connect to the database. On the surface this looks fine; postgres is running, port 5432 is published, `localhost` should work. **Will this connect successfully?** No. And this exact mistake has cost teams hours of debugging. `localhost` inside a container means *the container itself* - its own loopback interface. Not the host machine. Not another container. The backend is looking for a postgres process running inside itself, finding nothing, and failing. The correct connection string inside Docker: ```yaml DB_URL: jdbc:postgresql://postgres:5432/mydb ``` `postgres`, the service name gets resolved by Docker's embedded DNS to whatever IP the postgres container currently has. This works even when containers restart and get new IPs, because the DNS always stays current. If your connection string says `localhost` and it's in a container, that's the bug. --- ## The Three Patterns in Production Setup **Pattern 1 - Healthchecks on every stateful service, conditions on every dependent.** If a service stores data (database, cache, queue), it needs a healthcheck that actually tests readiness. Every service that connects to it needs `condition: service_healthy`. **Pattern 2 - Secrets without fallbacks, configs with fallbacks.** ```yaml NODE_ENV: ${NODE_ENV:-production} # fine, has a safe default JWT_SECRET: ${JWT_SECRET} # no default — fail loudly if missing DB_PASSWORD: ${DB_PASSWORD} # no default — fail loudly if missing ``` The moment you add a fallback to a secret, you create a silent security hole. A developer forgets to set `JWT_SECRET` on the server, it falls back to an empty string, authentication silently breaks in a completely confusing way. **Pattern 3 - Explicit named networks and 127.0.0.1 port bindings.** ```yaml networks: app-network: driver: bridge services: backend: networks: - app-network ports: - "127.0.0.1:8080:8080" ``` Named networks give you intentional topology. `127.0.0.1` bindings give you intentional exposure. "It works" and "it's secure" are different things. --- ## The Compose File as Documentation Your `docker-compose.yml` is not just a configuration file. It's **living documentation of your entire application architecture**. Every service is listed. Every dependency is declared. Every port, every volume, every environment variable; it's all there. A new developer joining your team can read your compose file and understand your entire system in 10 minutes. This is why naming matters. `container_name: myapp-postgres` instead of the auto-generated `myapp_postgres_1`. Explicit network names instead of the default. Volume names that describe what they store. The best compose files read like a clear description of the system. The worst ones read like a pile of configuration that only the original author understands. --- ## What's Next In the next article we're going into **Docker security**. Running containers as root by default, what Linux capabilities actually are, and the practical patterns that separate a container setup that "works" from one that won't get you in serious trouble if something goes wrong. *Have you ever spent hours debugging a compose file because of a `localhost` trap or missing healthchecks? What was the actual cause once you found it? Drop it in the comments.* ]]> https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-4-networking https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-4-networking Tue, 16 Jun 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker networking devops containers security All containers plugged into the same bridge can talk to each other over this virtual network. Each one gets its own private IP address, like `172.17.0.2`, `172.17.0.3`, and so on. So in theory, your backend container could connect to Postgres using its IP address — `172.17.0.3:5432`. And that would actually work. But here's the problem. Container IPs change. Every time you restart a container, it might get a different IP. Hardcoding `172.17.0.3` into your connection string is a massive issue. What you actually want is to say "connect to whatever container is named `postgres`" and have Docker figure out the IP for you. This is where the most misunderstood part of Docker networking comes in. --- ## The DNS Trap That Catches Almost Everyone **Docker has two completely different "default" behaviors depending on how you create your network, and one of them has no name resolution at all.** When Docker installs, it automatically creates a network simply called `bridge` — the **default bridge network**. If you run a container without specifying a network, it lands here. On this default network, containers can talk to each other only by IP address. There is **no DNS**. None. If your backend tries to look up `postgres` by name on the default bridge network, it fails `bad address 'postgres'` even if the Postgres container is sitting right there on the same network. ```bash # both containers on the DEFAULT bridge network docker run -d --name postgres postgres:16 docker run -d --name backend myapp docker exec backend ping postgres # ping: bad address 'postgres' (Fails) ``` But create your own network, a **user-defined bridge** and everything changes: ```bash docker network create app-network docker run -d --name postgres --network app-network postgres:16 docker run -d --name backend --network app-network myapp docker exec backend ping postgres # PING postgres (172.18.0.2): 56 data bytes (Works) ``` Same setup. Same containers. The only difference is which network they're on. One has DNS. One doesn't. Why does this gap exist? The default bridge network predates Docker's embedded DNS system. When Docker first shipped, the only way to connect containers by name was something called `--link` - a flag that created hardcoded, one-directional links between specific containers, and even leaked environment variables between them as a side effect. When Docker introduced user-defined networks, they came with a proper embedded DNS server running quietly inside every container at the address `127.0.0.11`. But to avoid breaking existing setups, the old default bridge network was left exactly as it was. No DNS, ever. The practical takeaway: **never use the default bridge network for anything with more than one container.** Always create your own network or better - use Docker Compose, which creates one for you automatically. This single habit eliminates an entire category of "why can't my containers see each other" bugs before they happen. --- ## Think About It Here's a quick experiment you can run in under a minute. Create any container on a custom network and peek inside at its DNS configuration: ```bash docker network create test-net docker run --rm --network test-net alpine cat /etc/resolv.conf ``` You'll see something like: ``` nameserver 127.0.0.11 options ndots:0 ``` That `127.0.0.11` is Docker's embedded DNS server. It's not your router. It's not Google DNS. It's a tiny DNS resolver that Docker injects into every container on a custom network and its only job is answering one question: *"what's the current IP address of the container named X?"* If you give **multiple containers the same network alias**, Docker's embedded DNS will return **all of their IP addresses** when you look up that name and clients that try multiple results (which most do) will essentially round-robin between them. ```bash docker run -d --network test-net --network-alias workers myapp docker run -d --network test-net --network-alias workers myapp docker run -d --network test-net --network-alias workers myapp docker run --rm --network test-net alpine nslookup workers # returns THREE different IP addresses ``` That's primitive load balancing, built directly into DNS, with zero extra tools. No nginx, no load balancer container, nothing. Just a naming convention. --- ## The Firewall That Doesn't Apply to Docker Say you're running a server with `ufw` (Uncomplicated Firewall) - a common Linux firewall tool. You've locked things down. You've explicitly denied incoming traffic on port 5432, because you don't want Postgres reachable from the internet. Then you run: ```yaml services: postgres: image: postgres:16 ports: - "5432:5432" ``` Your firewall rule says **deny**. Docker says **allow**. Docker wins. Here's why. When you publish a port with `-p` or `ports:`, Docker doesn't ask your firewall for permission. It writes its own rules directly into a part of the Linux networking stack called the **NAT table** and crucially, the NAT table is processed *before* your firewall's filter rules even get a chance to run. By the time your firewall rule would normally apply, Docker has already rewritten the destination and forwarded the packet straight to the container. Your `ufw deny 5432` rule never even gets a chance to fire. This isn't a bug. It's intentional - Docker manages its own networking because containers need dynamic, programmatic firewall rules that change every time a container starts or stops. But it means **the mental model "my firewall protects this server" is incomplete the moment Docker enters the picture.** The fix isn't a firewall rule. It's controlling *what Docker publishes in the first place*: ```yaml # Exposed to the entire internet — anyone can connect ports: - "5432:5432" # Only reachable from this machine itself — the right default for databases ports: - "127.0.0.1:5432:5432" ``` That `127.0.0.1:` prefix tells Docker to only bind the published port to the loopback interface. No NAT rule gets created for external traffic at all. This is the single most important Docker networking habit for anyone running databases, admin panels, or internal tools - bind to localhost, and let only your reverse proxy (nginx, Caddy, etc.) be reachable from outside. --- ## What `-p 8080:80` Actually Means One more thing, These two numbers don't need to match and usually shouldn't. You could run five different nginx containers, each listening on port 80 *inside* their own container, but published on the host as 8081, 8082, 8083, 8084, 8085. No conflicts, because each container has its own private network namespace - its own "apartment" where port 80 means something different in each one. The host port is the only thing that needs to be unique. The container port is just "what port does the app inside listen on" - completely independent of how the outside world reaches it. --- ## Bringing It All Together Let's connect this back to where we started - your backend trying to reach Postgres. The fix is simple once you see the whole picture: ```yaml services: backend: environment: DB_URL: jdbc:postgresql://postgres:5432/mydb networks: - app-network postgres: image: postgres:16 ports: - "127.0.0.1:5432:5432" # only reachable from this host networks: - app-network networks: app-network: ``` `postgres:5432` not `localhost`, not an IP address - works because both containers are on the same custom network, and Docker's embedded DNS at `127.0.0.11` resolves `postgres` to whatever IP that container currently has, every single time, automatically. The port binding to `127.0.0.1` means even though `ports:` is set, nothing outside this machine can ever reach Postgres directly, no matter what your firewall thinks it's doing. And Docker Compose does the "create a custom network" step for you automatically - which is exactly why this just works the moment you use Compose, and exactly why it silently fails the moment you try to recreate the same setup with plain `docker run` on the default bridge. --- ## The Core Mental Model Three things to carry forward: - **`localhost` inside a container means "myself," never "the host" and never "another container."** Every container is its own private apartment. - **Container name resolution only works on custom networks - never on the default bridge.** This single fact explains most "containers can't find each other" issues. Always create your own network, or use Compose. - **Docker's port publishing happens at the NAT level, before your firewall rules apply.** Your `ufw` or `iptables` rules don't protect published container ports. Binding to `127.0.0.1` does. Networking in Docker is a virtual switch, a tiny DNS server, and some NAT rules, working together in a way that's actually pretty elegant once you've seen behind the curtain. --- ## What's Next Next up, we're opening the Docker Compose file you've probably copy-pasted a dozen times and reading it the way Docker actually reads it. `depends_on`, healthchecks, volumes, secrets - the patterns that quietly separate a toy setup from something you'd trust in production. --- *Have you ever spent way too long debugging a "connection refused" between two containers that should obviously be able to talk to each other? What was the actual cause once you found it? Drop it in the comments.* ]]> https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-3-images-layers-builds https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-3-images-layers-builds Wed, 20 May 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker dockerfile devops containers builds Watch what happens with this typical Dockerfile: ```dockerfile FROM node:22-alpine COPY . . # copies ALL your code RUN npm install # installs all dependencies ``` You change one line in your `index.js`. Just one line. What happens? - `COPY . .` detects that files changed, so the layer is invalidated - `RUN npm install` comes after it, so it also runs from scratch - npm downloads and installs every dependency again - Build time: 3 to 4 minutes. For one line change. Now look at this version: ```dockerfile FROM node:22-alpine COPY package.json . # copies only the dependency list RUN npm install # installs dependencies COPY . . # copies your actual code last ``` Same one-line change to `index.js`. What happens now? - `COPY package.json .` has not changed, so it is a cache hit - `RUN npm install` has nothing changed above it, so it is also a cache hit - `COPY . .` sees the code change and rebuilds, but this is nearly instant - Build time: 3 to 4 seconds. Same app. Same dependencies. The only difference is the order of two lines. **Put what changes least at the top. Put what changes most at the bottom.** Your code changes dozens of times a day. Your dependencies change maybe once a week. So dependencies should always sit above your code copy in any Dockerfile you write, regardless of language: ```text Node.js -> COPY package.json first, then RUN npm install Python -> COPY requirements.txt first, then RUN pip install Java -> COPY pom.xml first, then RUN mvn dependency:go-offline Go -> COPY go.mod go.sum first, then RUN go mod download ``` Same principle. Every language. Every project. ## Try It Right Now If you have Docker installed, do this. It takes about three minutes and you will feel the difference directly. Create a `Dockerfile.slow`: ```dockerfile FROM node:22-alpine WORKDIR /app COPY . . RUN npm init -y && npm install express ``` And a `Dockerfile.fast`: ```dockerfile FROM node:22-alpine WORKDIR /app COPY package.json* ./ RUN npm init -y && npm install express COPY . . ``` Build both. Note the time. Then make a tiny change to any file, even just add a comment. Build both again. The first rebuild takes almost as long as the initial build. The second rebuild is nearly instant. Same result, completely different experience. That is the cache working for you instead of against you. ## The Hidden Truth About Image Layers Here is the lesser-known angle that changes how you think about Docker images entirely. Docker images are internally structured almost identically to Git. In Git, every commit is identified by a SHA256 hash of its contents. Change one character anywhere and you get a completely different hash. Branches and tags are just human-readable pointers to specific commit hashes. The actual data is content-addressable, stored and retrieved by what it contains, not by a name. Docker image layers work exactly the same way. Every layer is identified by the SHA256 hash of its contents. Pull the `nginx` image and you will see something like: ```text sha256:a1b2c3d4e5f6... <- not a name, a fingerprint of the actual content ``` This means two completely different images that happen to share the same base OS layer share that layer physically on disk. Docker stores it once. Pulls it once. Uses it everywhere. It also means Docker can detect tampering. If someone intercepts your image pull and modifies a layer, the hash will not match and Docker knows immediately. This is the same guarantee that makes Git commits trustworthy. And just like Git tags point to specific commits, Docker tags point to specific image manifests. When you push `myapp:latest`, you are creating a pointer. When you push `myapp:latest` again with a new build, the pointer moves. The old content is still there but the tag now points somewhere new. This is why version tags matter so much in production. `myapp:1.0.0` is an immutable pointer forever. `myapp:latest` is a moving target that means whatever was last pushed. ## The Factory Floor Problem Now let's talk about image size, because this is where most production images quietly go wrong. Think about a car factory. The factory floor is massive: heavy machinery, welding robots, painting rigs, assembly lines, quality control stations. Every tool you need to build a car is there. But when the car rolls off the line and goes to a customer, do they get the factory too? Obviously not. They get the car. Just the car. **Most Docker images ship the factory.** To *build* a Node.js app you need the Node.js runtime, npm, all your source code, the TypeScript compiler, ESLint, test frameworks, and all your dev dependencies. To *run* that same app in production you need the compiled output files, maybe a web server, and production dependencies only. That is it. A naive Dockerfile ships everything. The factory floor, the tools, the raw materials, the mess. The result is an image that is 900MB, sometimes 1GB or more. Slow to pull, slow to deploy, and a much larger attack surface. **Multi-stage builds** solve this. You use multiple `FROM` statements in one Dockerfile, where each one starts a completely fresh environment. The final stage copies only what it needs from the previous stages. ```dockerfile # Stage 1: The Factory FROM node:22-alpine AS builder WORKDIR /app COPY package.json . RUN npm install COPY . . RUN npm run build # Stage 2: The Car FROM nginx:alpine COPY --from=builder /dist /usr/share/nginx/html # nginx + your compiled files. Nothing else. ``` The final image contains zero trace of Node.js, npm, your source code, or your dev dependencies. Just nginx and the compiled output. The size difference is not subtle. Here are real numbers from production: | Language | Before Multi-Stage | After Multi-Stage | Reduction | |---|---|---|---| | Node.js | 1,100 MB | ~148 MB | ~87% | | Python | 920 MB | ~85 MB | ~91% | | Java | 700 MB | ~180 MB | ~74% | | Go | 850 MB | **~12 MB** | **~99%** | Go's 99% reduction deserves a special mention. Go compiles to a single static binary with no external dependencies. The entire runtime is baked in. So the final image can be just that one binary sitting on a minimal base with no OS tools, no package manager, nothing. 850MB of build environment produces a 12MB production image. ## Two Common Mistakes Worth Fixing Today **Mistake 1: Not using `.dockerignore`** When you run `docker build`, Docker sends your entire project directory to the daemon as the build context. Everything in the folder. Including `node_modules` (often hundreds of MB), your `.git` history, test fixtures, and log files. Even if you never explicitly `COPY` these files, they inflate your build context and slow down every single build. The fix is a `.dockerignore` file in your project root: ```text node_modules .git *.log .env dist coverage ``` Same concept as `.gitignore` but for Docker builds. It takes two minutes to add and can save tens of seconds per build. **Mistake 2: Splitting `apt-get update` and `apt-get install`** This one is subtle: ```dockerfile # Wrong: two separate layers RUN apt-get update RUN apt-get install -y curl # Right: one layer, always fresh RUN apt-get update && apt-get install -y curl \ && rm -rf /var/lib/apt/lists/* ``` When they are on separate lines, `apt-get update` can get cached from a week ago while `apt-get install` runs fresh, resulting in package version mismatches and mysterious build failures. Combining them ensures they always run together. The `rm -rf /var/lib/apt/lists/*` at the end cleans up the package index files so they do not bloat your layer. ## Three Things to Do Right Now Docker images are not zip files. They are stacks of layers, each identified by a fingerprint of its content, each cached independently. The layer cache is your most powerful build optimization, and the order of your Dockerfile instructions is the single biggest lever you have over it. Take any Dockerfile you are working on right now and check these three things: 1. **Move dependency installation above your code copy.** `package.json` first, then install, then `COPY . .` 2. **Add a `.dockerignore`** if you do not have one. It takes two minutes. 3. **Check if your production image needs multi-stage builds.** If it is over 500MB, it almost certainly does. ## What's Next In the next article we are going into Docker networking. Why does `localhost` break inside a container? How do your containers actually find and talk to each other by name? What is really happening when you do `-p 8080:80`? If you have ever seen `connection refused` between two containers that should be talking to each other, that article will make it completely clear. --- *What is the worst slow build you have ever sat through? How long before you found the Dockerfile ordering issue? Drop it in the comments.* ]]> https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-2-whats-happening https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-2-whats-happening Sun, 17 May 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker linux containers architecture devops So your 10 Nginx containers don't store 10 × 200MB on disk. They share one 200MB set of read-only layers, and each has a tiny writable layer on top that only holds whatever that specific container has written. This is also why containers start so fast. There is nothing to copy. The filesystem is already there, shared and ready. The kernel just needs to create a new writable layer and attach it on top. ## Think About It You have a container running Nginx. Inside the container, you run: ```bash echo "hello" > /etc/nginx/test.txt ``` You just wrote a file to what appears to be the Nginx config directory. Now you stop and delete that container. Then you spin up a brand new container from the exact same Nginx image. **Is that `test.txt` file there?** No. It is gone completely. Because that file only existed in the container's writable layer, a thin overlay that gets thrown away when the container is removed. The shared read-only image layers underneath were never touched. This is why containers are called **ephemeral**; they are designed to be disposable. Any data you want to survive must be stored outside the container's writable layer, in a Docker volume or a bind mount. The container itself is meant to be replaceable at any moment. ## The Call Chain: From Your Keyboard to a Running Process Now let's connect all three features to what actually happens when you type `docker run nginx`. Most people think Docker does everything. It doesn't. Docker is more like a coordinator. It delegates the actual work down a chain of highly specialized tools: Each layer has one job and hands off to the next. Here is what each one actually does: **Docker CLI:** Just a client. When you type `docker run`, it sends an HTTP request to the Docker daemon over a Unix socket. The CLI does no container work itself. **dockerd:** The manager. It handles the big picture: is the image available locally? If not, pull it from Docker Hub. Then it hands the actual container creation over to containerd. **containerd:** The lifecycle manager. It prepares everything the container needs; the filesystem bundle, the network setup, the configuration, and then asks runc to actually create it. **runc:** This tiny binary makes a special system call called `clone()` with specific flags that tell the kernel to create all six namespaces at once. It sets up cgroups for resource limits. It mounts the OverlayFS layers. Then it executes your process inside that isolated environment. And then runc does something that surprises almost everyone who learns about it. **runc exits.** It doesn't stick around to monitor the container. Its job is to *create* the environment and *start* the process. Once that's done, it's gone. If you look for runc processes on a machine with 20 running containers, you will find none. So who keeps the containers alive? **The shim.** That `containerd-shim` sitting between containerd and runc. There is one per container, silently running the whole time. The shim holds the container's stdio open, reports the exit code when the container finishes, and most importantly, keeps the container alive even if containerd or dockerd crashes and restarts. This is why if the Docker daemon dies, your running containers survive. They aren't held by Docker. They are held by their shims. ## What This Means for You Practically - **"Why did my container crash with exit code 137?"** The kernel OOM-killed it. Your container hit its cgroup memory limit. Add a memory limit and watch `docker stats` you'll see it coming. - **"Why do containers start so fast?"** There is no OS to boot. The kernel is already running. OverlayFS just attaches a writable layer to shared read-only layers. Milliseconds, not minutes. - **"If the Docker daemon crashes, do my containers die?"** No. The shim keeps them alive. You just can't manage them until dockerd restarts. - **"Why can't I find my file after the container restarts?"** Container writable layers are ephemeral. Use volumes for anything you want to persist. ## The Three Pillars, One Sentence Each After all of this, here is the clean summary you can carry in your head: - **Namespaces** give each container it's own private view of the world, its own processes, network, and filesystem. - **cgroups** enforce hard limits on how much of the machine each container can consume. - **OverlayFS** makes images efficient by sharing read-only layers and only copying files when they're actually written. Docker didn't invent any of these. It built a beautiful, developer-friendly UX on top of them. And now that you know what's *underneath*. ## What's Next In the next article, we are going deep into **Docker images and Dockerfiles**,how images are actually built layer by layer, why the order of your instructions matters for build speed, and the multi-stage build pattern that can take your image from 900MB down to 25MB. If you've ever copied a Dockerfile from Stack Overflow without fully understanding why it was structured that way, that article is for you. --- *Which of these three; namespaces, cgroups, or OverlayFS surprised you the most? And did you know runc exits immediately after starting a container? Drop your reaction in the comments.* ]]> https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-1-why-docker-exists https://portfolio-v2-burcpl0ve-yasas-malavige-s-projects.vercel.app/blog/docker-zero-to-production-part-1-why-docker-exists Fri, 15 May 2026 00:00:00 GMT ybmalavige@gmail.com (Yasas Banuka Malavige) docker containers devops linux history ## The Problem Every Developer Knows You've felt this. Even if you didn't have a name for it. You spend days building a feature on your laptop. The tests pass. The app compiles. Everything works flawlessly. You push it to the staging server, and it instantly crashes. Wrong version of Node. Missing C++ library. Different Linux distribution. A ghost environment variable that exists on your machine but nowhere else. The error stack trace makes absolutely no sense. You spend two days debugging infrastructure instead of writing code. **"Works on my machine."** This phrase became the running joke of a generation of software engineers. It was funny because it was universally, agonizingly true. Now, multiply that single failure across an entire organization: - **5 developers** with slightly different local setups - **A staging environment** that drifted away from production months ago - **A production server** that gets its packages updated independently - **Three different microservices**, each demanding conflicting runtime versions Every deployment turned into an archaeology project. You weren't just shipping code anymore - you were shipping a fragile, invisible web of assumptions. ## The First Attempt: Virtual Machines The industry actually had a solution for this long before Docker arrived. **Virtual Machines (VMs).** In theory, the idea was brilliant: instead of hoping the server's environment matches your code's expectations, just ship the entire environment. Pack up an operating system, your runtime, your libraries, your code - all of it - into one massive file. Run it anywhere. And it worked. Mostly. But VMs came with a hidden tax that developers hated. A VM isn't just a box your app lives in. It is a **full computer simulation**. It has its own operating system kernel, its own virtualized memory management, and its own fake hardware. All of that has to run on top of the real machine's hardware through a hypervisor. > Imagine you want to move to a new city for work. A VM is like picking up your entire house - foundations, plumbing, wiring, roof - and physically moving it to the new city. It works, but the truck is enormous, moving day takes hours, and you're paying to transport a lot of stuff you could have just found in the new city. The numbers tell the real story. On a standard server with 128GB of RAM: - You could comfortably run maybe **10-15 virtual machines**. - Each VM eats **600MB+** just to boot its guest OS - before your app even starts. - Startup time is measured in **minutes**, not seconds. And the problem wasn't just resource bloat. It was **rigidity**. Once you configured a VM, it was heavy, slow to spin up, and painful to replicate perfectly. Developers still had to write lengthy, brittle bash scripts and pray that what worked on the staging VM matched the production VM. The "works on my machine" problem hadn't been solved. It had just moved one layer deeper. ## The Insight That Changed Everything Here's the thing nobody initially noticed about VMs: most of that weight was completely unnecessary. Every single VM was carrying its own operating system kernel. But here is the question that unlocked the container revolution: **Why does every app need its own kernel?** The kernel - the core of the OS that manages hardware, memory, and CPU processes - is already running on the host machine. It's shared infrastructure, like the plumbing in an apartment building. Why does every tenant need to install their own water main? What if you could keep the isolation (each app stays in its own private box) but ditch the duplication (no separate OS per box)? Docker's answer: a container doesn't carry a kernel. It **borrows the host's**. The container gets total isolation - its own filesystem, its own network stack, its own process space, but the heavy lifting is handled by the existing host kernel. > Now, instead of dragging your whole house to the new city, you just **pack a suitcase**. You bring your clothes and your laptop. The city already has buildings, plumbing, and electricity. You only bring what is uniquely yours. The suitcase is your container. The city's infrastructure is the host kernel. ## The Efficiency Leap Let's make this concrete. Imagine running three apps on one server: a Node.js API, a Python data processor, and an Nginx web server. **With VMs:** You are running three separate virtual computers. Three full OS installations. Your server is burning a huge chunk of its RAM and CPU just keeping those three mini-computers alive, before your apps process a single user request. **With Containers:** You have three isolated boxes sharing one kernel. The server's power goes almost entirely to your actual apps. And because there's no bulky OS to boot up, each container starts in **milliseconds**. On that same 128GB server where you could only run 15 VMs, you can now comfortably run **400–500 containers**. That isn't just an efficiency gain. It's a fundamentally different way to compute. A container starts instantly because there is no OS to boot. It's just a new, isolated process using the kernel that's already running. Think of it as the difference between booting up a whole new laptop (VM) versus simply opening a new tab in your browser (Container). ## What Docker Actually Did Here is the most fascinating part of the origin story: **Docker didn't invent containers**. The underlying Linux features that make containers possible - namespaces for isolation, cgroups for resource limits, union filesystems for layered images - had existed in the Linux kernel for years. Google had been using similar technology internally since the mid-2000s. But nobody outside of hardcore systems experts could use it. It required deep Linux kernel knowledge, complex configuration files, and a whole lot of duct tape. What Solomon Hykes and the Docker team did was make it accessible. They built a clean, developer-friendly interface on top of these powerful but complex Linux features. A simple CLI. A standard format for packaging apps (the image). A registry for sharing them (Docker Hub). They didn't build the engine. They built the steering wheel, the pedals, and the dashboard. As Solomon himself put it: *"Docker happened to see the opportunity and shift the right tools at the right time... It was going to happen anyway, with or without Docker."* ## The Dockerfile That Didn't Exist Here is a lesser-known fact that perfectly captures Docker's explosive growth: the **Dockerfile** wasn't even in the original launch. The file we now use to define exactly how our app's environment gets built was added later as a *"simple experiment."* As Solomon admitted: *"It was taped together and not in the original version. It became popular, so we had to fix it."* The community grabbed it and ran. Teams started writing Dockerfiles before Docker's own engineers had even stabilized the feature. The tool got pulled forward by the sheer desperation of the people using it. Any rough edges were instantly forgiven because the core value was so obvious. ## Where This Leaves Us Let's step back and look at the arc. The problem was old and painful: **software is fragile because environments differ**. Your app isn't just your code; it's your code plus every assumption it makes about the world around it. VMs tried to solve this by shipping the whole world. It worked, but it was expensive, slow, and bloated. Docker solved it by realizing you only need to ship **your app's specific world**. Just the filesystem, the libraries, and the runtime. The kernel is already there. The result? Portable, lightweight, fast, and mathematically consistent environments that you can run anywhere and share with anyone. And it all started with a 5-minute talk that got cut off. ## What's Next This was the "why." In the next article, we are going to go one level deeper, into what is actually happening inside your machine when you type `docker run`. Spoiler: it involves three Linux kernel features that existed long before Docker, doing some genuinely clever things. Once you see how they work, containers will never feel like magic again. They'll just feel obvious. --- *Did you have a "works on my machine" moment that cost you real time? Drop it in the comments — I'd love to hear the most painful ones.* ]]>