Talk of artificial intelligence destroying humanity plays into the tech companies’ agenda, and hinders effective regulation of the societal harms AI is causing right now.
Both are concerning, but as a former academic to me neither of them are as insidious as the harm that LLMs are already doing to training data. A lot of corpora depend on collecting public online data to construct data sets for research, and the assumption is that it's largely human-generated. This balance is about to shift, and it's going to cause significant damage to future research. Even if everyone agreed to make a change right now, the well is already poisoned. We're talking the equivalent of the burning of Alexandria for linguistics research.