With free esxi over, not shocking bit sad, I am now about to move away from a virtualisation platform i’ve used for a quarter of a century.
Never having really tried the alternatives, is there anything that looks and feels like esxi out there?
I don’t have anything exceptional I host, I don’t need production quality for myself but in all seriousness what we run at home end up at work at some point so there’s that aspect too.
I try to stick with libvirt/virsh when I don't need any graphical interface (integrates beautifully with ansible [1]), or when I don't need clustering/HA (libvirt does support "clustering" at least in some capability, you can live migrate VMs between hosts, manage remote hypervisors from virsh/virt-manager, etc). On development/lab desktops I bolt virt-manager on top so I have the exact same setup as my production setup, with a nice added GUI. I heard that cockpit could be used as a web interface but have never tried it.
Proxmox on more complex setups (I try to manage it using ansible/the API as much as possible, but the web UI is a nice touch for one-shot operations).
Re incus: I don't know for sure yet. I have an old LXD setup at work that I'd like to migrate to something else, but I figured that since both libvirt and proxmox support management of LXC containers, I might as well consolidate and use one of these instead.
Re incus: I don’t know for sure yet. I have an old LXD setup at work that I’d like to migrate to something else, but I figured that since both libvirt and proxmox support management of LXC containers, I might as well consolidate and use one of these instead.
Maybe you should consider consolidating into Incus. You’re already running on LXC containers why keep using and dragging all the Proxmox bloat and potential issues when you can use LXD/Incus made by the same people who made LXC that is WAY faster, stable, more integrated and free?
It's like a first-try at a hypervisor. Terrible UI, with machine config scattered around. Some stuff can only be done on the command line after you search the web for how to do it (like basic stuff, say run headless by default). Enigmatic error messages.
I've used Hyper-V and in fact moved away from ESXi long ago. VMWare had amazing features but we could not justify the ever-increasing costs. Hyper-V can do just about anything VMWare can do if you know Powershell.
Another vote for Hyper-V. Moved to it from ESXi at home because I had to manage a LOT of Hyper-V hosted machines at work, so I figured I’d may as well get as much exposure to it as I could. Works fine for what I need.
Qemu/virt manager. I've been using it and it's so fast. I still need to get the clipboard sharing working but as of right now it's the best hypervisor I've ever used.
I'm pretty happy with XCP-ng with their XenOrchestra management interface. XenOrchestra has a free and enterprise version, but you can also compile it from source to get all the enterprise features. I'd recommend this script: https://github.com/ronivay/XenOrchestraInstallerUpdater
I'd say it's a slightly more advanced ESXi with vCenter and less confusing UI than Proxmox.
I actually moved everything to docker containers at home... Not an apples to apples, but I don't need so many full OSs it turns out.
At work we have a mix of things running right now to see. I don't think we'll land on ovirt or openstack. It seems like we'll bite the cost bullet and move all the important services to amazon.
In my "testing" at work and private, PVE is miles ahead of xcp-ng n terms of performance. Sure, xcp-ng does it's thing very stable, but everything else...proxmox is faster
As part of the transition of perpetual licensing to new subscription offerings, the VMware vSphere Hypervisor (Free Edition) has been marked as EOGA (End of General Availability). At this time, there is not an equivalent replacement product available.
If you are dipping toes into containers with kvm and proxmox already, then perhaps you could jump into the deep end and look at kubernetes (k8s).
Even though you say you don't need production quality. It actually does a lot for you and you just need to learn a single API framework which has really great documentation.
Personally, if I am choosing a new service to host. One of my first metrics in that decision is how well is it documented.
You could also go the simple route and use docker to make containers. However making your own containers is optional as most services have pre built ones that you can use.
You could even use auto scaling to run your cluster with just 1 node if you don't need it to be highly available with a lot of 9s in uptime.
The trickiest thing with K8s is the networking, certs and DNS but there are services you can host to take care of that for you. I use istio for networking, cert-manager for certs and external-dns for DNS.
I would recommend trying out k8s first on a cloud provider like digital ocean or linode. Managing your own k8s control plane on bare metal has its own complications.
OP here.
Thanks for all your input!
It’s really an embarrasment of riches which will take me some time to navigate. But thats part of the fun, right?
Again, thanks everyone.
It's just a suggestion. Many would probably find that the workload they host is available on containers. I run a Kubernetes cluster on bare metal at home. There's also nothing stopping you from creating VMs that you can ssh to with KubeVirt
Coming from a decade of vmware esxi and then a few years certified Nutanix, I was almost instantly at home clustering proxmox then added ceph across my hosts and went 'wtf did I sell Nutanix for'. I was already running FreeNAS later truenas by then so I was already converted to hosting on Linux but seriously I was impressed.
Business case: With what you save on licensing for Nutanix or vsan, you can place all nvme ssd and run ceph.
What does oVirt offer that proxmox doesn't? I'm asking because I want to move an ESXi server to another hypervisor, I'm 90% sure it'll be Proxmox, but I'd like to know my options.