This was my attempt to create a a SNES to Bluetooth adapter. It works but the average latency was 18.35ms, which I think is too much to be considered a good controller.
How much are you doing with "rolling your own" vs using off the shelf solutions? For example if you are deep into rolling your own which Bluetooth profile are you using?
So in this case it's my own code to read the state of the controller and using the ble-gamepad library, which uses NimBLE under the hood. There could be some optimizations but getting to be under 10ms probably is going to use a lot of time I could just be playing games.
So you suspect the latency (enough of it anyway) is introduced in your translation code? I absolutely understand not wanting to go to the nth degree for optimization when this is a hobby project. Could you throw hardware at it? Not all ESP32 run at the same speed. Perhaps buy an oversized one that will execute your underperforming code faster?
There probably could be some optimization in the translation, such as passing the controller object as a reference when checking the state of the buttons, rather than doing two loops to read the state and act on it. But I doubt that is the source of the largest latency since this is compiled to run at 240mhz (I did some additional testing at 80 as well, about the same results). I'm using a d1 mini off alliexpress so not the highest quality but I wouldn't expect a huge performance hit.
Without more testing it's hard to say. I don't want say it's the hardware without proof, but would require a much deeper dive into esp32 and c++ than I want to do right now.