This thing is dramatically slower than a 4090 both in prefill and decode. And I ...

		EnPissant 32 days ago \| parent \| context \| favorite \| on: Nvidia DGX Spark: great hardware, early days for t... This thing is dramatically slower than a 4090 both in prefill and decode. And I do mean DRAMATICALLY. I have no immediate numbers for prefill, but the memory bandwidth is ~4x greater on a 4090 which will lead to ~4x faster decode.