Docker & Reverse Proxies
Docker & Reverse Proxies
So I'm in the process of (re-) setting up my homelab and unsure about how to handle databases. Many images require a database, which the docker-compose usually provides inside the stack....
Mooching off this other post
Primary question: What do people do for their reverse proxies (and associated ACME clients)? Do you have a single unified one? Or do you use separate proxies for each stack? Or some mess in between?
My use case question: For example, I have a (mess that is a) Nextcloud instance with a separate stack with nginx and ACME, a SearXng that wants to run caddy (but has shoved into the nginx).
But now I have a Lemmy docker that has a custom(?) nginx instance, should I just port it to my existing nginx or run them side by side?
I use a single unified traefik to front all of my services, no matter how they ship. Despite the slight overhead, it's closer to a truly idempotent architecture. I've unfortunately had to test that twice now in my selfhosting career.
Traefik is very solid and I've had very few issues with it I didn't self inflict. Documentation is very thorough.
I also run Traefik and it never stutters. But getting it set up at ALL was a chore. I tried four times and failed. Each time I spent several days full time on it. It's not that I skipped the docs, I actually am a RTFM kinda guy. But too much was implied in the docs and I never really felt like I knew why I was doing stuff. At least for me it was harder to set up than Nextcloud, Jellyfin, Gitea, Resilio and Vaultwarden COMBINED.
Some settings only work in a static config file, others in a "dynamic" config file and then there is container-specific labels too. It all needs to fit in with each other and error mesages were of course hidden away in docker logs. You can attach labels to containers with and without escaping them, and choosing wrong sends you down several rabbit holes at once. The config structure is probably intuitive to Go devs, but that really ain't me. Oh, there's also 3 different but equal formats for conf files too.
I read countless guides and it finally worked on attempt 5. All just because I liked the autoconf for all containers. I could have been done with reverse proxies within a day had I just chosen a different one.
Now I am even debating wether I should keep it at all, because I'd rather not mount the docker sock into my reverse proxy, the one software that ultimately connects to the web directly.
I’d rather not mount the docker sock into my reverse proxy,
You don't have to if you use the dynamic file config. I've mentioned this before and debated to the ends of earth for even suggesting such a thing. But it all aspects is dynamic file configuration better.
Of you use IaC in your set up, it gets even easier because then you can just set up templates that automatically create file configs and add them to your reverse proxy seamlessly.
Right now with one Terraform apply, I create my docker container, traefik config and my homepage service.
Yeah, this experience with Traefik lines up pretty well with mine. It can be a steep learning, and the fact that half the search results out there are for Traefik v1 (with a completely different configuration syntax to v2) doesn't help. But once it's up and running, the dynamic configuration based on container labels is pretty darned nice.
Now I am even debating wether I should keep it at all, because I’d rather not mount the docker sock into my reverse proxy, the one software that ultimately connects to the web directly.
You could switch to Podman, in which case you'd give it a non-root, read-only socket that isn't the keys to the kingdom. Or maybe rootless Docker would be an easier switch and still give you some of those benefits.
Do you have a guide for Traefik that you recommend?
I tried about 6 guides:
This is the definitive guide that always works even if you sub in cloudflare for let's encrypt.
https://www.smarthomebeginner.com/traefik-docker-compose-guide-2022/
Since nobody has responded to the ACME / Let's Encrypt part of the question yet, I'll chime in: I also use Traefik as a reverse proxy (and an ACME client), one unified instance per machine. (There are some exceptions, like for Mailu that requires its own nginx reverse proxy.) But for Let's Encrypt, I recently switched from the TLS challenge to the DNS challenge. That required switching my DNS server from CoreDNS to PowerDNS, but thus far it seems totally worth it. Now I can easily get TLS certs for servers on my private network at home without opening them up to the internet for HTTP/TLS challenges.
Can you expand on more detail on how your TLS certs work? Looking to do similar.
I am just running the normal nginx image with
/etc/letsencrypt:/etc/ssl/privateas volume.certbotdoes the rest. If you need help with the exact config just search for relevant keywords, there are tons of good tutorialsHere's an overview of the Let's Encrypt DNS challenge type in case you haven't seen it: https://letsencrypt.org/docs/challenge-types/#dns-01-challenge
Basically, when Traefik goes to request or renew a certificate, Let's Encrypt tries to look up a special DNS record on your domain so you can prove that the request for the certificate is legit. To make that work, Traefik first hits your DNS provider via API and temporarily inserts that special record so it's there when Let's Encrypt performs the lookup for it. In my particular case, I'm using self-hosted PowerDNS and it's built-in API (configured to only respond via a Wireguard tunnel). But you don't have to self-host DNS for this to work.. Traefik has a long list of supported providers: https://doc.traefik.io/traefik/https/acme/#dnschallenge
At work I use Contour, which uses Envoy under the hood. At home I just use nginx for everything.
I run the HAProxy and ACME packages available from PFSense on my firewall.
Certificate rotation is automatic, connected to my domain in cloudflare and I have 1 *:443 listener on a virtual IP with about a dozen backends pointing directly to each app.