Cloudflare's new marketplace will let websites charge AI bots for scraping
Cloudflare's new marketplace will let websites charge AI bots for scraping
Cloudflare announced plans on Monday to launch a marketplace in the next year where website owners can sell AI model providers access to scrape their
Does that mean scraping without paying access to scrape it is circumventing a protection and so can be treated like cracked software/piracy of movies? /s
The gall of Cloudflare to think they can be the arbitrator of scraping… this is what happen when all y’all give a singular publicly-traded, US-based major control of the general internet infrastructure by purposefully letting them man-in-the-middle your production sites. Now they get to sell access to what was once an open internet. Instead every time Clouflare or Fastly go down, half of the internet goes down with them.
“We can scrape open internet” is such a CEO take. I’m no fan of Cloudflare but what they’re doing here is good for open web and bad for AI bros
This assumes that just since abuse could happen means we should block access for everyone. Folks might make illegal photocopies of books so we should ban libraries. I & others have done general scraping for our own uses that isn’t done in some abusive manner. But to assume a company beholden to US shareholder is going to “to the right thing” would be to go against the history of US corporations.
And you know who is going to be able to afford to do the scraping? Big US-based “AI Bros” that can do it with venture capital preventing the average user or researcher from grepping the net.
They don't think they're the arbitrer of scraping dumbass, this is a service they're selling to websites who don't want AI scrapers taking adnatage of them.
They are precisely deciding who can scrape what by sitting in the middle of like 30% of internet traffic & denying access. There is no way to tell if this ‘scraping’ is for research, hobby, commercial, or “AI” purposes; conveniently if it can make Cloudflare money, they’ll let you charge a toll. If Cloudflare cared about AI issues, they wouldn’t be having unpaid users solve/train their hCAPTCHA models just for visiting a site from Tor, a VPN, or even just a non-‘Western’ IP address. The fact that folks/businesses bought into this centralization is frightening—with little open access to information or allowing folks to stay anonymous (whatever their motivation).
Also don’t dare call someone “dumbass” if you can’t be bothered to turn on spell check or understand how commas work.
God damn it, stop seeing scrapers as a bad thing. If we rate limit, there's nothing wrong with scraping.
I guess if you can't beat them it's the next best thing.
Another wall to try and make it so only the rich corporations can make AI, what a shock.
This is what all the useful idiots bleating against ai have been working towards.
There are models that have only used data with permissive licenses.