Selfhosted @lemmy.world 7Sea_Sailor @lemmy.dbzer0.com 9 mo. ago

After 1.5 years of learning selfhosting, this is where I'm at

Mid 2022, a friend of mine helped me set up a selfhosted Vaultwarden instance. Since then, my "infrastructure" has not stopped growing, and I've been learning each and every day about how services work, how they communicate and how I can move data from one place to another. It's truly incredible, and my favorite hobby by a long shot.

Here's a map of what I've built so far. Right now, I'm mostly done, but surely time will bring more ideas. I've also left out a bunch of "technically revelant" connections like DNS resolution through the AdGuard instance, firewalls and CrowdSec on the main VPS.

Looking at the setups that others have posted, I don't think this is super incredible - but if you have input or questions about the setup, I'll do my best to explain it all. None of my peers really understand what it takes to construct something like this, so I am in need of people who understand my excitement and proudness :)

Edit: the image was compressed a bit too much, so here's the full res image for the curious: https://files.catbox.moe/iyq5vx.png And a dark version for the night owls: https://files.catbox.moe/hy713z.png

124

124 comments

me after 15 years of intermittent learning self hosting:

i have the one random office PC that runs minecraft

....yeah that's it
This seems like work but from/for home.
I've saved this. I set up unraid and docker, have the home media server going, but I'm absolutely overwhelmed trying to understand reverse proxy, Caddy, NGINX and the security framework. I guess that's my next goal.
I'd recommend using Borgbackup over SSH, instead of just using rclone for backups. As far as I know, rclone is like rsync in that you only have one copy of the data. If it gets corrupted at the source, and that gets synced across, your backup will be corrupted too. Borgbackup and Borgmatic are a great way to do backups, and since it's deduplicated you can usually store months of daily backups without issue. I do daily backups and retain 7 daily backups, 4 weekly backups, and 'infinite' monthly backups (until my backup server runs out of space, then I'll start pruning old monthly backups).

Borgbackup also has an append-only mode, which prevents deleting backups. This protects the backup in case the client system is hacked. Right now, someone that has unauthorized access to your main VPS could in theory delete both the system and the backup (by connecting via rclone and deleting it). Borg's append-only mode can be enabled per SSH key, so for example you could have one SSH key on the main VPS that is in append-only mode, and a separate key on your home PC that has full access to delete and prune backups. It's a really nice system overall.
Very nice setup imho. Quite a bit more complicated than mine - mine is basically just the left box without being behind a VPS or anything. I don't expose anything through Caddy except Jellyfin. I'm also running fail2ban in front of my services, so that if it gets hit with too many 404s because someone is poking around, they get IP banned for 30d
Architecture looks dope

Hope you've safeguarded your setup by writing a provisoning script in case anything goes south.

I had to reinstall my server from scratch twice and can't fathom having to reconfigure everything manually anymore
Possible for a dark mode version XD? excalidraw can do that.

Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I've seen in this thread:

Fewer Letters	More Letters
DNS	Domain Name Service/System
HTTP	Hypertext Transfer Protocol, the Web
HTTPS	HTTP over SSL
IP	Internet Protocol
Plex	Brand of media server package
SSH	Secure Shell for remote terminal access
SSL	Secure Sockets Layer, for transparent encryption
TCP	Transmission Control Protocol, most often over IP
VPN	Virtual Private Network
VPS	Virtual Private Server (opposed to shared hosting)
k8s	Kubernetes container management package
nginx	Popular HTTP server

11 acronyms in this thread; the most compressed thread commented on today has 15 acronyms.

[Thread #473 for this sub, first seen 2nd Feb 2024, 05:25] [FAQ] [Full list] [Contact] [Source code]

How do you like crowdsec? I've used it on a tiny VPS (2 vcpu / 1 GB RAM) and it hogs my poor machine. I also found it to have a bit of learning curve, compared to fail2ban (which is much simpler, but dosen't play well with Caddy by default).

Would be happy to see your Caddy / Crowdsec configuration.
Remeber, the more boxes you have, the more advanced you are as an admin! Once you do his job for money, the challenge is the exact opposite. The less parts you have, the better. The more vanilla they are, the better.
I saved this! Yeah, it seems like a lot of work, but I got inspired again (I had a slight self-hosting burnout and nuked my raspberry setup ~year ago) so I appreciate it. :) Can I ask what hardware you run this on? edit: I just wanted to ramble some more: I just fired up my rPI4 again just last week, setup it with just as barebone VPS with wireguard, samba, jellyfin and pi-hole+unbound (as to not burn myself again :D )
What software did you use to make this image? Its very well done
I have taken a picture and shall study it
I've seen Caddy mentioned a few times recently, what do you like about it over other tools?
I am sorry, I am but a worm just starting Docker and I have two questions.

Say I set up pihole in a container. Then say I use Pihole's web UI to change a setting, like setting the web UI to the midnight theme.

Do changes persist when the container updates?

I am under the impression that a container updating is the old one being deleted and a fresh install taking its place. So all the changes in settings vanish.

I understand that I am supposed to write files to define parameters of the install. How am I supposed to know what to write to define the changes I want?

Sorry to hijack, the question doesn't seem big enough for its own post.
What is the proxy in front of crowdsec for?
What is seedbox? Is it part of the homelab or a service like the VPSs?
Sorry if someone already asked this, but do you have any tutorials or guides that you used and found helpful for starting out? I have some small experience with nginx and such, but I would definitely need to follow along with something that tells me what to do and what each part does in a infrastructure like you have haha
This is oddly similar to some informal workups I've done for our work network.

Nice work 👍.
What did you use to chart this? And nicely done.
I just have a UniFi firewall, a Synology Diskstation, and a linux server running everything. Provides torrenting, video streaming with plex, file sharing, game server hosting, music hosting, and more, and I don't ever have to mess with it :). This is impressive but I don't know if I would want to support it personally
Since nobody else asked about this, why ruTorrent over the other typical download clients?
btw why did you choose tailscale over zerotier
New Lemmy Post: After 1.5 years of learning selfhosting, this is where I'm at (https://lemmy.world/post/11462059)
Tagging: #SelfHosted

(Replying in the OP of this thread (NOT THIS BOT!) will appear as a comment in the lemmy discussion.)

I am a FOSS bot. Check my README: https://github.com/db0/lemmy-tagginator/blob/main/README.md
This is needlessly over complicated and bloated just FYI but if you enjoy complexity for the sake of complexity then go for it.
I can't say I'm in support of bring discussion of illegal content to Lemmy but you do you

You've viewed 124 comments.