Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

llama.cpp runs LLaMa 2 7B on common hardware like a MacBook Pro. Haven't tried it yet on my RTX 3070 (Mobile) but there's no reason why it shouldn't work.


A 7B LLM has a huge quantity of knowledge about the world. You don't need that just to reword sentences. You can use a translation model with English input and English output, or other Text2Text model such as one for textual style transfer. A purpose-built model for rewording into a fixed style different from the input could be easily be 10M parameters or fewer (that's already big enough for translating between two languages, afterall) but you can readily find models in the 100M range for text style transfer.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: