I didn't want yay "upgrading" my pacman packages with AUR packages.
But I was just misunderstanding yay. As another comment said before you, one can just run yay without any arguments and it accomplishes the same thing.
Yay doesn't replace normal packages with AUR packages. Btw It's not just an AUR helper, it's a wrapper for Pacman with AUR support built-in. Check out paru btw, it's a more modern version of yay that basically works the same way: https://github.com/Morganamilo/paru