Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Everyone keeps saying this but it is not really useful. Without a dedicated GPU & VRAM, you are waiting overnight for a response... The MoE models are great but they need dedicated GPU & VRAM to work fast.


Well, yeah, you're supposed to put in a GPU. It's a MoE model, the common tensors should be on the GPU, which also does prompt processing.

The RAM is for the 400gb of experts.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: