Last year, I turned my old computer into a home Plex media server: I sold my gpu and bought a silent cpu cooler, an nvme boot drive and a case. I have also found a pretty good deal on a 16tb Exos hdd. I am quite happy with the overall performance because it's my first home server and I am still digging in this rabbit hole ^^
For the next upgrades, I was thinking about increasing the transcoding performance, while keeping the power draw as low as possible. I am planning to run the server 24/7 and power efficiency is really important to me. For now, the pc is running at 30w idle, and I would like to reduce it to maybe 20w.
I had in my mind to probably upgrade the cpu to a i5-8600 or i7-8700 for x265 transcoding support. And probably finally upgrading the PSU to a quite and more efficient one at low power (maybe corsair rm550x but always out of stock).
PS: only for personal use, no massive sharing with family and friends planned.
You're going to simplify your life and your setup if you can avoid transcoding.
In a home environment, I see no need for transcoding anything (direct play of even 90 GB UHD HDR films is no problem to for example iPad, or iPhones or most modern TVs.
Make sure that subtitles are in .SRT format, and you'll never need transcoding for that either.
The ONLY time I see a potential need for transcoding would be if all you media is stored in maximum-quality 80GB rips, and you need to watch a movie on your phone while on the subway. If you use Plex, the easiest solution is to always download or rip you movies in two formats. One 80 GB maximum quality UHD 10 bit file, and one 720p 1 GB x265 file, that will be automatically selected if bandwidth is low. Violá, you'll never need transcoding! :-)