If you want to go one step further, a lot of game development uses a generational index, where the index is both a value and a generation, allowing you to know whether the index you currently have stored references an object that has already been destroyed and replaced by another object. Basically every ECS framework I've ever seen uses this pattern.
I like this approach for Rust in general when it comes to graphs. But it suffers from many of the same kinds of issues we get with pointers, e.g. dangling pointers, use after free, and so on. Tradeoffs!