It's such a milquetoast warning, mostly because the science just isn't there yet. And by the time it is, any damage will be done.

So it's not entirely clear what the hell we should do, but it's probably not a knee jerk reaction.

Didn't the new TOS also contain an AUP for any user generated content?

you can just disable it

This is not actually true. Google re-enables it and does not have an account setting to disable AI results. There is a URL flag that can do this, but it's not documented and requires a browser plugin to do it automatically.