marauding_gibberish142 @ marauding_gibberish142 @lemmy.dbzer0.com

Posts

17

Comments

448

Joined

1 mo. ago

Thanks for the tip on x670, I'll take a look

Thanks for the comment. I don't want to use a networked distributed cluster for AI if I can help it. I'm looking at other options and maybe I'll find something

Your point is valid. Originally I was looking for deals on cheap CPU + Motherboard combos that will offer me a lot of PCIe and won't be very expensive, but I couldn't find anything good for EPYC. I am now looking for used supermicro motherboards and maybe I can get something I like. I don't want to do networking for this project either but it was the only idea I could think of a few hours back

I'd prefer that you reply with examples/an explanation of what I'm doing wrong instead of cursing

Thanks

I see. I must be doing something wrong because the only ones I found were over $1000 on eBay. Do you have any tips/favoured listings?

Used 3090s go for $800. I was planning to wait for the ARC B580s to go down in price to buy a few. The reason for the networked setup is because I didn't find there to be enough PCIe lanes in any of the used computers I was looking at. If there's either an affordable card with good performance and 48GB of VRAM, or there's an affordable motherboard + CPU combo with a lot of PCIe lanes under $200, then I'll gladly drop the idea of the distributed AI. I just need lots of VRAM and this is the only way I could think of.

Thanks

Thank you, and that highlights the problem - I don't see any affordable options (around $200 or so for a motherboard + CPU combo) for a lot of PCIe lanes other than purchasing Frankenstein boards from Aliexpress. Which isn't going to be a thing for much longer with tariffs, so I'm looking elsewhere

Thank you, but which consumer motherboard + CPU combo is giving me 32 lanes of PCIe Gen 4 neatly divided into 2 x16 slots for me to put 2 GPUs in? I only asked this question because I was going to buy used computers and stuff a GPU in each.

Your point about networking is valid, and I'll be hesitant to invest in 25Gbe right now

Thanks but I'm not going to run supercomputers. I just want to run 4 GPUs separately because of inadequate PCIe lanes in a single computer to run 24B-30B models

I'm not going to do anything enterprise. I'm not sure how people seem to think of it this way when I didn't even mention it.

I plan to use 4 GPUs with 16-24GB VRAM each to run smaller 24B models.

I don't understand your point, but I was going to use 4 GPUs (something like used 3090s when they get cheaper or the Arc B580s) to run the smaller models like Mistral small.

Who elects these dumb goblins?

It is because modern consumer GPUs do not have enough VRAM to load the 24B models. I want to run Mistral small locally.

Thank you, I'll take a look

Thank you for the links. I will go through them

Aren't Epyc boards really expensive? I was going to buy 3-4 used computers and stuff a GPU in each.

Are there motherboards on the used market that can run the E5-2600 V4 series CPUs and have multiple PCIe Xi slots? The only ones I found were super expensive/esoteric.

Never heard of it. What is it about?

So, any plans to include LibreWolf and TOR browser

Please at least put Librewolf and Mullvad in there. If you're feeling generous, even TOR browser.

Has Waterfox resolved their security issues?