Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Can one of thse models be run on a single machine? What specs do you need?


Absolutely! They have a billion-parameter model that will run on my first computer if we quantize it to 1.5 bits. But realistically yes, if you can fit in total ram you can run it slowly, if you can fit it in gpu ram you can probably run it fast enough to chat.


The 8B models run fine on a M1 pro 16GB.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: