Creepiest house bug ever. I let them be because they eat other bugs.
If you ever see a lot of them you’ve got another problem, that means they have a lot to eat. Could be an infestation of other bugs like roaches or a dead animal in the walls.
According to a random comment I saw once, these things will happily eat each other if there's no other bugs to eat. So basically yes.... unless your house has a continuous influx of new bugs.
This is also why they aren't actually great at pest control. They are pretty territorial so their populations never get big enough to make a serious dent. It's perfectly fine to kill them.