I’m trying to better understand hosting a Lemmy Instance. Lurking discussions it seems like some people are hosting from the Cloud or VPS. My understanding is that it’s better to futureproof by running your own home server so that you have the data and the top most control of hardware, software etc. My understanding is that by hosting an instance via Cloud or VPS you are offloading the data / information to a 3rd party.
Are people actually running their own actual self-hosted servers from home? Do you have any recommended guides on running a Lemmy Instance?
Self hosting basically means you are running the server application yourself. It doesn’t matter if it’s at home, on a cloud service or anywhere else.
I wouldn’t recommend hosting a social network like lemmy, because you would be legally responsible for all the content served from your servers. That means a lot of moderation work. Also, these types of applications are very demanding in terms of data storage, you end up with an ever growing dataset of posts, pictures etc.
But self hosting is very interesting and empowering. There are a lot of applications you can self host, from media servers (Plex, Jellyfin), personal cloud (like Google Drive) with NextCloud, blocking ads with pihole, sync servers for various apps like Obsidian, password manager BitWarden etc. You can even make your own website by coding it, or using a CMS platform like WordPress.
Check the Awesome Self-hosted list on GitHub, has a ton of great stuff.
And in terms of hardware, any old computer or laptop can be used, just install your favorite server OS (Linux, FreeBSD/OpenBSD, even Windows Server). You can play with virtualization too if you have enough horsepower and memory with ESXI or Proxmox, so you can run multiple severs at once on the same computer.
No that’s a homelab. Selfhosted applies to the software that you install and administrate yourself so you have full control over it. If it was about running hardware at home we’d see more posts about hardware.
I would say that a homelab is more about learning, developing, breaking things.
Running esoteric protocols, strange radio/GPS setups, setting up and tearing down CI/CD pipelines, autoscalers, over-complicated networks and storage arrays.
Whereas (self)hosting is about maintaining functionality and uptime.
You could self-host with hardware at home, or on cloud infra. Ultimately it’s running services yourself instead of paying someone else to do it.I guess self-hosting is a small step away from earning money (or does earn money). Reliable uptime, regular maintenance etc.
Homelabbing is just a money sink for fun, learning and experience. Perhaps your homelab turns into self-hosting. Or perhaps part of your self-hosting infra is dedicated to a lab environment.
Homelab is as much about software as it is about hardware. Trying new filesystems, new OSs, new deployment pipelines, whatever
I ran one for a few months until I woke up one morning and it wasn’t working. As I was the only person using it, I didn’t bother to troubleshoot and just signed up for an account at lemmy.world.
If you want to run your own I recommend you check out the ansible install route. It’s really simple and straightforward once you wrap your head around ansible.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
Fewer Letters More Letters CGNAT Carrier-Grade NAT DNS Domain Name Service/System Git Popular version control system, primarily for code HTTP Hypertext Transfer Protocol, the Web IP Internet Protocol LXC Linux Containers NAS Network-Attached Storage NAT Network Address Translation Plex Brand of media server package SATA Serial AT Attachment interface for mass storage SSD Solid State Drive mass storage VPN Virtual Private Network VPS Virtual Private Server (opposed to shared hosting) nginx Popular HTTP server
[Thread #139 for this sub, first seen 16th Sep 2023, 05:05] [FAQ] [Full list] [Contact] [Source code]
Hey, I love this thread, and I am intrigued by the term "futureproof"ing. can someone direct me to a thread where local networks are self-hosted and the human element of organizing the network is discussed? Thank you. If I don’t come back, it’s because I’m new to Lemmyworld and got lost.
Most people who “self host” things are still doing it on a server somewhere outside their home. Could be a VPS, a cloud instance, colocated bare metal, …
For me, since you can get kicked off a platform eg hetzner vps, it’s not self hosted, they’re hosting you.
actually have a server at home
I haven’t got any piece of hardware that was sold with the firstname “Server”.
But there’s this self-built PC in my room that’s running 24/7 without having to reboot in several years…
Do you have any recommended resources for getting started? I do have a secondary PC…
First, you need a use-case. It’s worthless to have a server just for the sake of it.
For example, you may want to replace google photos by a local save of your photos.
Or you may want to share your movies accross the home network. Or be able to access important documents from any device at home, without hosting them on any kind of cloud storage
Or run a bunch of automation at home.
TL;DR choose a service you use and would like to replace by something more private.
recommended resources for getting started?
I don’t know where to start today, honestly.
I started with books a long time ago:
https://www.amazon.com/Algorithms-Data-Structures-Niklaus-Wirth/dp/0130220051
https://www.amazon.de/-/en/Andrew-S-Tanenbaum/dp/0132126958
https://www.amazon.de/Programming-Language-Prentice-Hall-Software/dp/0131103628
The simple way is to Google ‘yunohost’ and install that on your spare machine, then just play around with what that offers.
If you want, you could also dive deeper by installing Linux (e.g.Ubuntu), then installing Docker, then spin up Portainer as your first container.
Get a copy of vmware (esxi) or proxmox and load it on that secondary pc.
Proxmox absolutely changed the game for me learning Linux. Spinning up LXC containers in seconds to test out applications or simply to learn different Linux OSs without worrying about the install process each time has probably saved me days of my life at this point. Plus being able to use and learn ZFS natively is really cool.
Ive been using esxi (free copy) for years. Same situation. Being able to spin up virtual machines or take a snapshot before a major change has been priceless. I started off with smaller nuc computers and have upgraded to full fledged desktops.
Well, there are specific hardware configurations that are designed to be servers. They probably don’t have graphics cards but do have multiple CPUs, and are often configured to run many active processes at the same time.
But for the most part, “server” is more related to the OS configuration. No GUI, strip out all the software you don’t need, like browsers, and leave just the software you need to do the job that the server is going to do.
As to updates, this also becomes much simpler since you don’t have a lot of the crap that has vulnerabilities. I helped manage comuter department with about 30 servers, many of which were running Windows (gag!). One of the jobs was to go through the huge list of Microsoft patches every few months. The vast majority of which, “require a user to browse to a certain website” in order to activate. Since we simply didn’t have anyone using browsers on them, we could ignore those patches until we did a big “catch up” patch once a year or so.
Our Unix servers, HP-UX or AIX, simply didn’t have the same kind of patches coming out. Some of them ran for years without a reboot.
How do you install security updates etc without restarting?
Linux servers prompt you do restart after certain updates do you just not restart?
Enterprise distributuions can hot-swap kernels, making it unnecessary to reboot in order to make system updates.
Microsoft needs to get its shit together because reboots were a huge point of contention when I was setting up automated patching at my company.
Good luck with that, I have all reboot options off but yesterday it just rebooted like that. Thanks MS.
install security updates etc without restarting?
Actually I am lazy with updates on the “bare metal” debian/proxmox. It does nothing else than host several vm’s. Even the hard disks belong to a vm that provides all the file shares.
You can just restart… with modern SSDs it takes less than a minute. No one is ging to have a problem with 1 minute downtime per month or so.
The right way ™ is to have the application deployed with high availability. That is every component should have more than one server serving it. Then you can take them offline for a reboot sequentially so that there’s always a live one serving users.
This is taken to an extreme in cloud best practices where we don’t even update any servers. We update the versions of the packages we want in some source code file. From that we build a new OS image contains the updated things along with the application that the server will run and it’s ready to boot. Then in some sequence we kill server VMs running the old image and create news ones running the new. Finally the old VMs are deleted.
Years? Lol you should update that software.
Well technically a “server” is a machine dedicated to “serving” something, like a service or website or whatever. A regular desktop can be a server, it’s just not built as well as a “real” server.
There is though reasons to stray from certain consumer products for server equipment.
FWIW, this free app solves for that issue well; I have several clammed Macs running it right now:
just break the screen off. call it a headless sever.
100%, and this is why businesses don’t use laptops as servers… typically 😂.
Just set it to “do nothing” when lid is closed. That’s all.
Yeah I’d stay away from Mac too… but seriously most modern laptops can disable any sleep/hibernation on lid close
My go to lately is Lenovo tiny, can pick them up super cheap with 6-12 month warranties, throw in some extra ram, a new drive, haven’t had any fail on me yet
Lenovo tiny
Doesn’t that mean, tiny fans howling all day long?
Only if you’ve got it cranking all day. I’ve got a couple of Tiny (they’re Micro, which is the same thing) systems that are silent when idle and nearly silent when running less than a load avg of 5. It’s only if I try to spin up a heavy, CPU-bound process that their singular fan spins fast enough to be noticable.
So don’t use one as a Mining rig, but if you want something that runs x64 workloads at 9-20 watts continuously, they’re pretty good.
Even running at full speed mine are pretty quiet but I also have 80mm silent low rpm fans blowing air across them too which seems to help
I also recently went through with fresh thermal paste
You should think before releasing dangerous information on the internet!
You can get a 2core 8GB / 240GB for 75€!!
Uh oh, I think I’ll have to buy one now…
This is my little setup at the moment. Each is 8500t CPU, 32gb ram, 2tb nvme and 1tb SATA SSD all running in a proxmox cluster
Edit: also check out Dell micro or the hp… Uh I want to say it’s g6 micro? You might need to search for what is actually called
Heh, this is awesome 😅
Thanks :D the frame and all parts are self designed and 3d printed… was a fun project
The whole thing runs from just 2 power cables with room for another without adding any extra power cables
Not at all overkill? :-D
Future proofing or is it really used ? I don’t know proxmox, is it some docker launcher thingy?
Very cool anyways!
Proxmox is like esxi, it lets you setup virtual machines. So you can fire up a virtual Linux machine and allocate it like 2gb ram and limit it to 2 cores of the CPU or give it the whole lot depending on what you need to do
Having them in a cluster let’s them move virtual machines between the physical hardware and have complete copies so if one goes down the next can just start up
It is a little overkill, I’m probably only using about 20% of its resources but it’s all for a good cause. I’m currently unable to work due to kidney failure but I’m working towards a transplant. If I do get a transplant and can return to work, being able to say “well this is my home setup and the various things I know how to do” looks a lot better than “I sat on my ass for the last 4 years so I’m very rusty”
This whole setup cost me about $1000aud and uses 65-70w on average
For me it does. I’m sure some other people use a VPS or something and self host using a cloud provider of some kind.
I pay Dreamhost for a beef pc VPS, that’s what “selfhosted” means to me. I host all kinds of shit on it.
What kinda beef are we talking here?
500 GB SSD / 16 GB RAM / 8 vCPUs you won’t go hungry
Yep, big ol’ case under my desk with some 20TB of storage space.
Most of what I host is piracy related 👀
how much of the 20tb is used?
There’s about 3.5 TB to go out of an actual 18 in the server.
I have another 2TB to install but it’s not in yet.
I’m also transcoding a lot of my media library to x265 to save space.
I don’t download for the sake of downloading, usually, and i delete stuff if I don’t see value in keeping it.
What is a good transcoder? I haven’t ran my media through one. Is the space saving significant? Did you lose video quality?
In order:
FFMPEG, yes, no.
I refer you to this comment for more info.
I have seen a saving of 60-80% per file on lower resolutions like 720 or 1080, which makes the server time well worth it.
A folder of 26 files totaling 61GB went down to 10.5, for example.
Free space is wasted space
~19.5 tb of hardcore midget porn
~500 gigs of whale sounds to help me sleep
Whoops that was backwards, my bad
~19.5 tb of hardcore whale porn
~500 gigs of midget sounds to help me sleep
Whale sounds! Oh man, I’m gonna add that to ELF space radio and rain recordings!
There are definitely benefits on running a server at home but you could say the same of a VPS. As long as you control it, it is self hosted in my book.
Isn’t control limited on 3rd party hosts for security purposes?
@ProtecyaTec@lemmy.world not sure what that means but I guess you can do certain thing in your home server not allowed on a VPS. OTOH, an email server at home for example is much more difficult to achieve because your ISP most likely won’t allow you to open port 25
I was just reading through
Lemmy-AnsibleGithub docs and wasn’t sure if a 3rd party server or a home server was going to be better to run. The documentation doesn’t feel super user-friendly or maybe I’m just not as tech-savvy as I thought.@ProtecyaTec@lemmy.world idk about ansible but if you are starting I would recommend starting small, in your personal computer. Docker is a good way to start
This may be way of scope for this thread but I’m not sure a better place yet to ask and learn more about how all this works. Say I run my Docker Container from a home PC, how do I make my Lemmy Instance accessible to the public? I’m familiar with web hosting but only from hosting on a simple 3rd party, where you buy a domain.
Agree with the VPS in this case. For sure you can create public-facing services in a home server or home lab, but to do so you need:
- Domain name hosting.
- an Internet Service Provider who will allow you expose port 80/443 web services and on a Static IP (most do not, or paid extra on business plans). OR use a Cloud proxy like CloudFlare which your home IP can be updated through a DynDNS service and served on private ports.
- Setup NAT/Port Forwarding on your modem to route incoming requests to internal services. First to a firewall or threat gateway like PFSense, a web proxy like Traefik/NGinX, and security harden and maintain your modem, router, network and served applications.
If you’re new to these things, Id start with something more mature for personal or family home use first. Like NextCloud, HomeAssistant or Jellyfin media server. Lots of YouTubers have covered how these can be set up as a reference.
Lemmy is still alpha, full of bugs and security vulnerabilities and needs regular hotfixes and babysitting. Permitting Joe Public into your home services is ripe for disaster unless you have the time and expertise.
Operating internet-facing services in the home, in my opinion, requires a layer-3 managed switch so that internet traffic is 100% separated from home traffic, w/attendant DMZ to bridge home<-> internet-facing services safely.
L3 managed is the simplest method to contain a penetration to just the internet-facing devices (which is still pretty bad). Cloud hosting is more manageable, but you must watch the spend.
The biggest issue is a DDoS attack on the home network, which could impact internet-facing services and home clients (streaming TV, gaming, email, etc.).
@ProtecyaTec@lemmy.world yeah if you want to host something like Lenny you may be able to do it on a home server but in terms of ports blocked, security, etc I guess it is easier to run it in a VPS
It is, if you can host at home and use a cheap vps as proxy is better in my opinion. Like it or not when you host something you will put sensitive info in your vps and ofc you should trust them. Exactly like the vpn providers, they will fight for your rights paying 5 dolars/months? For sure no.
Or your Internet provider, they will also not fight for your rights paying $30 a month.
My services are not accessible from my wan, i forward all my services to a cheap vps,using a proxy and trafic is encrypted. So i am not worried about my isp also my real ip is hiden. I use something like cloudflare zero trust… but is open source.
I’m less worried aobut sensitive info and more worried about owning the data. I’m looking to avoid a sitaution where I offload all the data to a 3rd party and lose access or pricing get expensive (like a Reddit situation). I’m more worried by offloading the data to a 3rd party I can’t verify that they’re not reselling it. etc.
Not really, you’re ideally paying for a server that you have complete control of. The differences are mostly just fundamental limitations.
Example: if you’re hosting off site, you will always be connecting remotely, so your access depends on a network connection. If you’re hosting at home then your stuff is still accessible when your internet goes down
at least i do have 2 servers. one main and one backup
Me, yes. But it’s still selfhosting if you do it on a VPS. And probably easier, too. I mainly do it at home, because I can have multiple large harddisks this way. And storage is kind of expensive in the cloud.
Depending on where you are, a hard drive that runs 24/7 can cost you quite a bit of money (6$/month or even more for just the hard drive). If you consider the upfront cost of a hard drive, the benefit of hosting at home gets even smaller. Nvme is where you really save money hosting at home. Personally I do both, because cloud is cheap and you can have crazy bandwiths.
I know. You pretty much need to know what you’re doing. And do the maths. Know the reliability (MTBF/MTTF) and price. Don’t forget to multiply it by two (as I forgot) because you want backups. And factor in cost of operation. And corresponding hardware you need to run to operate those hdds. My hdd spins down after 5 minutes. I live in Europe and really get to pay for electricity as a consumer. A data center pays way less. My main data, mail, calendar and contacts, OS, databases and everything that needs to be there 24/7 fits on a 1TB solid state disk that doesn’t need much energy while idle. So the hdd is mostly spun down.
Nonetheless, I have a 10TB hdd in the basement. I think it was a bit less than 300€ back when I bought it a few years ago. But I can be mistaken. I pay about 0.34€/kWh for (green) electricity. But the server only uses less than 20W on average. That makes it about 4€ per month in electricity for me. And I think my homeserver cost me about 1000€ and I’ve had it since 2017. So that would be another ~15€ per month if I say I buy hardware for ~1100€ every 6 years. Let’s say I pay about 20€/month for the whole thing. I’m not going to factor in the internet connection, because I need that anyways. (And I probably forgot to factor in I upgraded the SSD 2 times and bought additional RAM when it got super cheap. And I don’t know the reliability of my hdds…)
I also have a cheap VPS and they’d want 76,27€/month … 915.24€ per year if I was to buy 10TB of storage from them. (But I think there are cheaper providers out there.) That would have me protected against hard disk failures. It’ll probably get cheaper with time, I can scale that effortlessly and the harddisks are spun up 24/7. The harddisks are faster ones and their internet connection is also way faster. And I can’t make mistakes like with my own hardware. Maybe having a hdd fail early or buy hardware that needs excessive power. And that’d ruin my calculation… In my case… I’m going with my ~20€/month. And I hope I did the maths correctly. Best bang for the buck is probably: Dont have the data 24/7 available and just buy an external 10TB hard drive if your concern is just pirating movies. Plug it in via USB into whatever device you’re using. And make sure to have backups ;) And if you don’t need vast amounts of space, and want to host a proper service for other people: just pay for a VPS somewhere.
You forgot something in your calculations, you don’t need a complete VPS for the *arrs. App hosting/seedboxes are enough for that and you can have them for very, very cheap.
Ahem, I don’t need a seedbox at all. I use my VPS to host Jitsi-Meet, PeerTube and a few experiments. The *arrs are for the people over at !piracy@lemmy.dbzer0.com
That you probably need a VPS for, yes.
Running on a synology, but it’s not cheap. I like having direct access to my stuff if I can. Next step is cloud backup of my local , i think borg or something is very popular.
I’m going to say that self hosting becomes a fun hobby once you get your core services running. Core in this case means the services that are bringing you into selfhosting.
