system_info: n_threads = 4 / 24 Am I seeing correctly in the video that this ran... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

		tiffanyh on Sept 7, 2023 \| parent \| context \| favorite \| on: Running a 180B parameter LLM on a single Apple M2 ... `system_info: n_threads = 4 / 24` Am I seeing correctly in the video that this ran on only 4 threads?

wmf on Sept 7, 2023 [–]

It's using the GPU so I guess not that many CPU threads are needed to feed the GPU.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact