I think this is one of the biggest problems with VR gaming. In theory, swinging around like Spider-Man in VR sounds awesome. But in practice, it's just nauseating. You really do need to be able to see the screen with a definite border to combat motion sickness. At least I do, not sure about everyone else.
It sounds lame, but the only VR game that I really enjoy is Real VR Fishing. Mostly because it's about as calm as it gets. No movement what so ever, but it still benefits from being immersive.
There's a game called Climbey (https://store.steampowered.com/app/520010/Climbey/), that was kind of like this. I had a ton of fun playing it, but it's pretty much like you said: this game worked for me, not so much for some of my friends who'd get nauseous as soon as they had to move (let alone look down).
That final comment about needing play testers that suffer from motion sickness is spot on. I played Star Wars Squadrons in VR and with the ship's frame around me, I could play that game for hours with the only problem being my own sweaty face. When my friend tried it out though, he could barely play for a few minutes before the motion sickness would set in and he'd have to break.
I hope someone figures out something that lets more people play cool VR games, because it's been a bummer that it seems like a 50/50 shot whether someone will be able to play the game without feeling sick.