Yep but I'll add my two cents, half is lying and half is guessfull ignorance because nobody really knows how big and old systems really work.

Libertarians assemble!

You're supposed to move to a different search engine for the market to work. I already have, have you?

No problem we crowdsource the crawling torrent style.

We outsourced that to google for reasonnable performance reason. But they shit the bed so now there's no choice but to do it ourselves.

Surprisingly, it's very doable, requires basic technical knowledge and relatively minimal computing resources (runs in the background on your computer).

https://yacy.net/ Github

I have tampermonkey script that sends yacy to crawl any websites that I visit, and it's keeping up relatively good index for personal use of the visited websites. Combine yacy with ~300gb of Kiwix databases, add searxng as a frontend and you have pretty strong self hosted search engine.

Of course you need to supplement your searches from other search engines, as yacy does not crawl the whole web, just what you tell it to.

I encourage anyone who's even slightly interested on this stuff to try Yacy, it's ancient piece of software, but it still works very well and is not an abandoned project yet!

--

I personally use Yacy mostly on private mode, but it does have the distributed network there as well.

Federated bookmarks?

Google actually was good, so there's probably some good information in this documentation. If nothing else we can perhaps figure out what "went wrong."

Edit: I've been reading the blog post that appears to be the main person the leak was shared with and there's a lot of in-depth analysis being done there, but I'm not seeing a link to the actual documents. This is a huge article, though, I might be overlooking it.

What it looks like beyond Google and Bing

It would be much harder to know what exists beyond "GBY" (Google, Bing, Yandex) and how it all works without the work of Rohan “Seirdy” Kumar. For three years, Kumar has been updating a heavily annotated list of search engines with their own indexes. It is 7,000 words, but only a portion of it deals with engines offering general indexing, in the English language. You can read Kumar's evaluation methodology for a better understanding of how he compared and assessed sites.

What stands out? Mojeek ("it's not bad… I'd live") and Stract ("a useful supplement to more major engines") are two of Kumar's favorites. Right Dao has "very fast, good results," in part because its crawler starts off from Wikipedia. Yep reaches farther out, showing results that link to and back from sites related to your query and also promises to share ad revenue with creators. All of them show promise, but you get the sense that they're a second car, or a third bicycle, rather than a primary transport.

There are far smaller-scoped engines in other sections of Kumar's post. If you're wondering where that one other search engine you've heard about is, it's probably in the "Semi-independent indexes" section, because it uses a GBY index when its own results are not strong enough. Here, you'll find cryptocurrency-friendly, controversy-courting-founder-having Brave, a few engines that either "resell" GBY results or stuff affiliate links into them, and "the most interesting entry," according to Kumar, Kagi.

Kagi requires an account and uses its own index, Teclis, in combination with Google, Bing, Yandex, Mojeek, and others, including, notably, Brave. Kagi's founder has strong opinions on the AI-based future of search and responding to harmful searches in ways that are not "scalable." How much of that does or does not bother you will vary, but it's worth noting that Kagi also suffers when the GBY triumvirate is restricted.

Ars Technica this week: Bing outage shows just how little competition Google search really has

The referenced search engine comparison by Rohan “Seirdy” Kumar

the only one I know that isn't a proxy search is yacy

Fuck SEOs - that is why you are getting downvoted. Organic content creation has been ruined by you AND google. Own your problems, beg forgiveness, stop playing the stupid game where there are no winners

wait what is "social planning" and how is it different from conventional marketing on social media. That seems pretty far removed from search engines

Wait why is that commit still up if this is a data leak?

Yeah, I know a lot of the smaller, independent search engines are lacking, but the people using the "udm=14" trick to remove Google's AI results now, as if that won't be removed as soon as Google needs to show investors the AI is more profitable.

Why is that better? It may not be ideal but governments have at least some accountability.

I agree decentralized is better, but isn't that an argument in favor of a government having more control than a corporation?

No it's not better.

Now I’m super curious. All of their comments have been removed.