Sites scramble to block ChatGPT web crawler after instructions emerge

As a thought experiment, imagine an online business declaring that it didn't want its website indexed by Google in the year 2002—a self-defeating move when that was the most popular on-ramp for finding information online.

This is exactly what will happen as people rely more and more on these things for "searching", since search by the two biggest search engines (Google and Bing) are both so gummed up with SEO spam and ads (my god, the number of ads) that people just start asking AI for the increasingly accurate answers without the bullshit. Eventually the AI solutions will become enshittified as well, but not before huge blows will be dealt to both search engines and the sites that refuse to expose themselves. Google can probably weather it, but can a site like VentureBeat? I have severe doubts.

1 comments

As a thought experiment, imagine an online business declaring that it didn't want its website indexed by Google in the year 2002—a self-defeating move when that was the most popular on-ramp for finding information online.
This is exactly what will happen as people rely more and more on these things for "searching", since search by the two biggest search engines (Google and Bing) are both so gummed up with SEO spam and ads (my god, the number of ads) that people just start asking AI for the increasingly accurate answers without the bullshit. Eventually the AI solutions will become enshittified as well, but not before huge blows will be dealt to both search engines and the sites that refuse to expose themselves. Google can probably weather it, but can a site like VentureBeat? I have severe doubts.