2y ago

ADMIN, isn't it time to move from lemmy.world?

Every time I try to access this community, ther's some kind of problem with the server. If you have a look at the status page, it's almost all orange/red. The problem aren't DDoS attack since the server is behind Cloudflare protection. Admin/mods, why don't you move this community to a different ~~server~~ instance? I'm not accusing anybody, I know that maintain a server can be a challenging sometimes, I just want to enjoy this community!

Please!

@Loki

44 comments

ADMIN, isn't it time to move from lemmy.world?
They said, from their lemmy.world account.
- just a user of an instance saying things about that instance - I would find the opposite weird.
- What's wrong with that? I've started here some months ago and I mainly follow this community.
  
  The problem is it's the largest and is being attacked. Use a different instance and you'll have no problem accessing this community. If it weren't for people posting about the outages, I'd never even know.
The problem is DDoS attacks. See: https://lemmy.world/post/2923697
- But isn't Cloudflare supposed to block those attacks?
  
  They explained that the attacks are in the form of requests that tax the database server, not the website itself.
  
  Yep Cloudflare protects against classic DDoS (like many clients doing a lot of small requests). Here attacks are performed presumibly by users that know very well how the Lemmy's backend works and where bottlenecks are, so that with a small number of well made requests they are able to mess up the backend and Cloudflare doesn't notice it
  
  Cloudflare DDos protection isn't a silver bullet; the attacks are distributed and come from shifting source IPs, and are sophisticated in that they exploit resource intensive queries specifically designed to overload a Lemmy instance. If lemmy.world were to pivot to some other instance, who's to say the culprits wouldn't just resume their efforts pointed at the new location? There are theories these may be carried out by the recently-defederated fringe hate communities
  
  Cloudflare has DDoS protection but it can't stop everything 100% of the time. According to the admins, the attackers are very familiar with how lemmy works and are using this knowledge to overwhelm resources. This isn't just a simple script kiddy or bonnet for hire but likely points to someone that has worked within the lemmy community.
  https://www.cloudflare.com/learning/ddos/what-is-a-ddos-attack/
  https://www.cloudflare.com/learning/ddos/glossary/web-application-firewall-waf/
  
  From what I understand, Cloudflare can block some DDoS attacks, but not all of them.
  The attacks on Lemmy have to do with poorly optimized SQL requests; these are requests that shouldn't take long to execute, but do due to some oversight. By spamming these requests, the attackers can bring Lemmy on it's knees.
  Actually, wouldn't this attack better be categorized as a DoS attack ? What's so distributed about it ?
  
  It can help, but is not perfect. There's kind of been an arms race between services like Cloudflare and script kiddies/hackers DDOSing: Their methods became a little more sophisticated to the point that they keep the traffic shifting from address to address so it's harder to track/block.
  They'll do other things to mess with the host too like spamming "white noise" pictures to fill up server storage space, so while DDOS attacks play a role, there are other issues at play on top of that.
  For what it's worth, I've been using Kbin.social and sh.itjust.works as well and they have also had some issues here and there. No host will be perfect and invulnerable from every attack, and when an instance becomes more popular, it becomes increasingly likely to be targeted by attackers.
Ironic. They could self-host for others, but they couldn't self-host themselves.
- There's a difference between self hosting and running a public service. There's a lot more overhead running a pubic service.
- What's wrong with that? I've started here some months ago and I mainly follow this community and I don't blame anyone.
I find this comment section a prime example of dickish hivemind seething over nothing.
There's a dude, obviously quite fresh in the ways of Mastodon. He probably doesn't realize all the nuts & bolts supporting the system and how it all works. He is asking a question that is logical, but it needs clarification, like "it doesn't work like this, my man".
Instead he gets "Hsssssssssssssss, selfhost it, hsssssssssssssssssssss, interloper, hsssssssssssssssssss, you want to destroy this place, hssssssssssssssss..."
Get a life, eejits.
- Your post is more offensive then the others.
  Not to mention that the OP is open for interpretation, and it came across (at least to me) as another entitled person moaning about a free service.
  Next to that, teaching people about the selfhost option is actually what lemmy is all about. Not beeing owned by a single corporate entity, so endusers have freedom to choose.
  If lemmy want to survive, it depends on people who know how to selfhost it.
- Thanks! As you, I thought that this was a community to share thought and knowledge. In fact a couple of guys here pointed out that I could solve my problem accessing this community in another instance, not in the better way, but at least that was helpful!
  
  Yeah. It's good that there are users who actually take time to explain some stuff, rather than just hissssssss like rabid vipers merely because somebody - oh no, what a preposterous idea! - asked a harmless question.
- They are making an assertion as if it is a statement of fact...and they are, in fact, wrong. That's ignorant and not helpful for discussion or helpful for understanfing and solving the actual issue. If they had actually asked an innocent question it would be different, but they didn't. That is why the responses are the way they are.
  
  No. The responses are the way they are, because people who gave them are already thinking they joined some elitistic "muh sikret klub!" group.
  Simple "eh, it won't fix the problem, and here's why and how YOU can help" would be preferable, but no, special elite force of lemmy underground is too privileged to bother.
  Thank heavens not everyone is like that. Saves the number of times I have to hit "block the idiot" button.
Well your account is on lemmy.world so how d'ya know the issue isn't with your own access to the front end?
Many don't interact with the lemmy.world directly, so we might only see delays in post propogation (if there is such an issue on the backend - I don't see any but could be wrong).
I agree picking the biggest instances isn't great from a scaling perspective, but s'gonna be hard to move any community once established.
You can start posting on !selfhost@lemmy.ml instead.
The problem is not the server. One of the reasons is because lemmy world is being DDoS attacked by certain party with grudges. Because lemmy is not used to this kind of attack, which is made difficult by way it is designed and open source nature where the attacker can easily uncover vulnerabilities, it took some times and learning to mitigate them. You can open new account in other instances or move if that's too much inconvenience for you. No one is stopping you.
What makes you think other servers aren't having similar problems?
(spoiler alert: they all are, lemmy is an immature platform)
By claiming that the problem isn't DDOS, you're just advertising your ignorance. Cloudflair is outstanding for protecting static web content against DDOS, and Lemmy.world is well protected against that. The problem is certain dynamic pages and api calls that can only be rendered from costly realtime dynamic database operations...those are the url that the DDOS attackers are focusing on and those are the kinds of content that cannot be easily protected by cloudflair.
Your premise, though, is still accidently correct. The way to mitigate instances being targeted by DDOS is to spread the user base and community hosting across a vast number of instances so that no one instance is such a rewarding target for DDOS attack.
- You're right about my ignorance about Lemmy, I'm a user on this federated ...thing and I know nothing ahout the Lemmy server. Being in a selfhosted community your answer is what I was expecting (maybe with less attack). The API are used only by the federated instance or also by the smartphone apps? For what I see, it seems to be the former, and, if it really is so, the API calls could be allowed only by those server and blocked from everyone else, Cloidflare WAF can do this. I know that the servers are a lot, but it could be possible to insert in the WAF all the IPs of the federated instances. ...or not?
I think the admin of c/selfhosted is the admin of Lemmy.world
Have you donated money so they can?
Another server (and i assume you mean one with better specs) means more money. Its not just challenging, its also costly.
Alternative you could setup your own lemmy instance, you can still join the community’s on lemmy.world or lemmy.ml or whereever :) and its fun to do.
- I mean to a different instance, not the same instance to a new server. The title was clear, I'be edited the comment to avoid misinterpretation.
  
  Its a shame Lemmy doesnt allow us to migrate our profiles (yet) like Mastodon for instance. But i have good hopes this will come in the future, so you can move to a different (better, depending on your definition) server easily :) We’ll just have to wait a while i guess :)
Is it possible to migrate a community from one instance to another?
- No, not programmatically. The only way to 'move' an instance is for a community of users to create a new community elsewhere and arbitrarily start using it as their preferred community. There is no way to force it and there is no mechanic for moving.

44 comments