Podman or rootless docker?

I’m moving to a new machine soon and want to re-evaluate some security practices while I’m doing it. My current server is debian with all apps containerized in docker with root. I’d like to harden some stuff, especially vaultwarden but I’m concerned about transitioning to podman while using complex docker setups like nextcloud-aio. Do you have experience hardening your containers by switching? Is it worth it? How long is a piece of string?