Meta Releases 816B Open-Weight Muse Spark Open
Meta Releases 816B Open-Weight Muse Spark Open
摘要: Meta Releases Muse Spark Open, an 816B-Parameter Open-Weight Model
Meta’s Muse Spark Open Model
Two Mac Studios on a desk can now hol
正文
Meta Releases Muse Spark Open, an 816B-Parameter Open-Weight Model
Meta’s Muse Spark Open Model
Two Mac Studios on a desk can now hold what Meta calls the most capable open-weight model ever built by a US lab. Meta
released the weights for Muse Spark Open
on Thursday, an 816-billion-parameter model built for coding and agentic work that runs locally on a pair of Apple M5 Ultra machines.
Meta Muse Spark – artistic visualization. Image credit: Meta
Key Takeaways
Muse Spark Open has 816 billion parameters, activates 42 billion per token, accepts text, image, video and audio input, and handles a 1-million-token context.
It runs on two M5 Ultra Mac Studios with 512GB of unified memory each, or four 256GB units, while its 4-bit version needs about 700GB of GPU memory on servers.
The weights ship under the Muse Model License with limited commercial terms, and Meta’s benchmark results have not been checked independently.
What Is Under the Hood
Muse Spark Open is a sparse mixture-of-experts model. Only about 42 billion of its 816 billion parameters work on any given token, which keeps compute costs closer to a mid-size model.
The model card lists 62 layers and a hidden dimension of 8,192. Its 256 experts hold roughly 97% of all parameters. It reads text, images, video and audio, and it answers in text.
Meta derived it from Muse Spark, its closed flagship model. The open version received extra safety post-training so it can resist misuse when it runs without Meta’s system-level guardrails.
Meta Muse Spark benchmarks. Image credit: Meta
Two Macs, One Terabyte
Each M5 Ultra Mac Studio can give its GPU up to 512GB of unified memory. Two machines together offer a terabyte, enough to hold the 4-bit NVFP4 checkpoint of about 530GB. The chip’s 1.2 TB/s of memory bandwidth lets it read the small slice of weights each token needs.
Apple’s MLX framework splits every layer across both computers. Each Mac holds half the weights and does half the math, and