The internet is worse than it used to be. How did we get here, and can we go back?

Lemmy is the way to go. Decentralized Communities connected via API.

This only works to a degree. Eventually, the communities that allow people to register most easily and see the most active content become the overwhelming majority of the content on the system. And if these communities don't do a good job of self-policing, they just become mini-2008-style Reddits, filling up with the same bot accounts and serial assholes and sex pests that degraded the original.

Bigger sites start swamping smaller sites with traffic and overwhelming the capacity of smaller communities, so you get waves of defederation and new Walled Gardens of content.

The issue isn't the technology, its the participants in that technology. Too many malicious actors piling onto a platform and either corrupting the administration or degrading the quality of content will inevitably lead to enshitification.

Federation only mitigates this by allowing smaller instances to break away and abandon larger ones. It does nothing to screen the sincere and human actors from the malicious and automatic accounts.