The prefix Fitz- means "bastard son of," according to my high school English teacher. So... the joke just got funnier, depending on your sense of humour.
It depended on the time and place. Fitzroy is particularly associated with illegitimate children of the king, but until the 17th century revival of the Fitz prefix, it was mostly just meaning "son of" without any particular inclination towards or against legitimacy. But in the Stuart era it was frequently used for illegitimate children of royalty and nobility.
I just gave it a listen today on my bike ride. It's pretty good - not what I expected but not surprising, either. I'd characterize it as Kate Bush meets Dead Can Dance meets early '70s King Crimson, and since I like all three of those acts I like Gibbons' album as well. I was surprised to see that it's her first solo album.