Some boards that use the nice!nano have room to tuck a small battery (I think 110mAh) underneath the controller. If they use the 3 middle pins on the nano though, that's not an option. Rechargeable coin cells are an option, but for boards with the usual LiPO pouch cells, you'll probably want some kind of case. Some folks just use double-sided tape to hold the battery to the PCB but I'd rather have some protection.
Batteries require management with control circuitry normally on the MCU. This means you need to pick MCUs that support batteries.
You will also want a on off switch for the battery somewhere.
I always use LIR2450 button cell batteries where I can. The steel body makes them more resistant to damage where as pouch cells can't take much of a scuff.
They're very portable! Just make sure the battery can't be smushed if the board itself gets some pressure, and that it can't get poked by sharp / conductive things.
ZMK for the nice nano and XIAO nrf52840 should even report battery life! Your primary half (generally left) drains faster than the accessory half.
RGB / screens that aren't designed for low power are utterly destructive to battery life. I get ~1-2 weeks on 120mah splits. With RGB I'm lucky to get an hour.
You don't change these batteries as they're rechargeable. Most boards will function while recharging. I offer QI wireless charging on my boards if you request it.
Searching for LIR2450 lead me to coin battery holder with on/off switch. I haven't seen a build with coin battery holders like this yet. I'm more curious now.