Let NixOS or Home-Manager manage X, not both

I've been spending a couple weeks unable to modify my system, because using my window manager was ungodly slow (like 1fps.) Luckily NixOS lets you pick a previous generation to load so I could make changes, build a new generation, and try again.

It took me too long to find, but I realized I had both the x session managed by both nixos and home manager. Removing this fixed the problem. I assume this had 2 xsessions open and they were competing for resources or something. Be cautious! :)