MoE Expert Streaming Runs 180B Model on 32GB Mac
A Hugging Face post shows MoE expert streaming running a 180B model on a 32GB Mac at roughly 14 tokens a second, within 6% of Strata.
A Hugging Face post shows MoE expert streaming running a 180B model on a 32GB Mac at roughly 14 tokens a second, within 6% of Strata.