RUNLOCALAIv38
->Will it run?Best GPUCompareTroubleshootStartLearnPulseModelsHardwareToolsBench
Run check
RUNLOCALAI

Independently operated catalog for local-AI hardware and software. Hand-written verdicts. Source-cited claims. Reproducible commands when we have them.

OP·Fredoline Eruo
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
TOOLS
  • Will it run?
  • Compare hardware
  • Cost vs cloud
  • Choose my GPU
  • Prompting kits
  • Quick answers
REF
  • All buyer guides
  • Learn local AI
  • Methodology
  • Glossary
  • Errors KB
  • Trust
EDITOR
  • About
  • Author
  • How we make money
  • Editorial policy
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. Read more →

© 2026 runlocalai.coIndependently operated
RUNLOCALAI · v38
  1. >
  2. Home
  3. /Tools
  4. /Exo
server
Open source
free (OSS, GPL-3.0)

Exo

Personal AI cluster software. Auto-discovers Apple Silicon devices on a LAN and shards a model across them via pipeline + tensor parallelism on top of MLX. The 2026 unlock: Thunderbolt 5 + macOS 26.2 RDMA dropped inter-device latency by ~99%, making consumer-Mac clusters credible — DeepSeek V3 671B runs at 5.37 tok/s on 8x M4 Pro Mac Minis. The default answer for 'I have several Macs and want to run a frontier model.'

By Fredoline Eruo·Last verified Jun 12, 2026·28,000 GitHub stars

Overview

Personal AI cluster software. Auto-discovers Apple Silicon devices on a LAN and shards a model across them via pipeline + tensor parallelism on top of MLX. The 2026 unlock: Thunderbolt 5 + macOS 26.2 RDMA dropped inter-device latency by ~99%, making consumer-Mac clusters credible — DeepSeek V3 671B runs at 5.37 tok/s on 8x M4 Pro Mac Minis. The default answer for 'I have several Macs and want to run a frontier model.'

Stack & relationships

How Exo relates to other entries in the catalog — recommended pairings, alternatives, dependencies, and edges to avoid. Each edge carries a one-line operator note from our editorial team.

Exo ↔ ecosystem

Recommended stack

  • Pairs with
    MLX-LM

    Exo is how you scale MLX-LM beyond a single Mac. The 2026 unlock — Thunderbolt 5 + macOS 26.2 RDMA — makes the cluster credible for serious models.

Alternatives

  • Alternative to
    Petals

    Petals shards over WAN volunteers; Exo shards over a controlled LAN cluster. Same architectural shape (pipeline parallel across machines), opposite trust models — public swarm vs personal devices.

  • Alternative to
    vLLM

    Different hardware target. vLLM = NVIDIA/Linux datacenter; Exo = Apple Silicon LAN cluster. Pick by which hardware you already own.

  • Competes with
    Petals

    Both are multi-machine inference; Exo runs over a controlled LAN with strong privacy, Petals runs over WAN volunteers with no privacy. Pick by trust model and what hardware you have.

  • Alternative to
    Ray Serve

    Exo for Apple-Silicon LAN clusters; Ray Serve for datacenter multi-node. Different hardware targets; non-overlapping operating points.

  • Competes with
    Petals

    WAN swarm vs LAN cluster. Petals trades latency for hardware availability; Exo trades hardware specificity for low latency. Different trust models.

  • Alternative to
    Hyperspace (P2P inference network)

    Different consumer-multi-machine paths. Exo is Apple Silicon LAN clustering; Hyperspace targets WAN P2P. Pick by hardware and trust model.

Depends on

  • Depends on
    MLX-LM

    Exo runs MLX under the hood for the per-device inference layer. Pipeline-parallel scheduling is Exo; the actual matmul kernels are MLX.

Featured in these stacks

The L3 execution stacks that pick this tool as a recommended component, with the one-line note explaining the role it plays in each.

  • Stack · L3·Workstation tier·Role: Distributed serving (multi-Mac cluster)
    Build a Mac-native AI stack (May 2026)

    Exo is what makes multi-Mac credible in 2026: auto-discovers nearby Apple Silicon devices on the LAN, shards models across them via pipeline parallel on top of MLX. Thunderbolt 5 + macOS 26.2 RDMA cuts inter-device latency by ~99%, turning consumer-Mac clusters into a real serving option.

  • Stack · L3·Production tier·Role: Cluster orchestrator
    Build a multi-machine Apple Silicon cluster (May 2026)

    Exo is what makes consumer-Mac clustering viable in 2026. Auto-discovery of nearby nodes; pipeline-parallel sharding via MLX. Thunderbolt 5 RDMA + macOS 26.2 cut inter-device latency by ~99% — the breakthrough that turned this from research demo to credible serving option.

Pros

  • Auto-discovers nearby devices, no cluster manager required
  • RDMA over Thunderbolt 5 makes inter-Mac latency nearly local
  • Runs 670B-class models on consumer hardware that can't fit them otherwise

Cons

  • Apple-Silicon-first; Linux/CUDA path is secondary
  • Thunderbolt-5 RDMA requires specific Macs (M4 Pro+, macOS 26.2+)
  • Not a production-serving solution — designed for personal clusters

Compatibility

Operating systems
macOS
Linux
GPU backends
Apple Metal
NVIDIA CUDA
LicenseOpen source · free (OSS, GPL-3.0)

Runtime health

Operator-grade signals on how actively Exo is being maintained, how fresh its measurements are, and what failure classes operators have flagged. Every label below is anchored to a real date or count — we never infer maintainer activity we can't show.

Release cadence

Derived from the most recent editorial signal on this row.

Active
Updated Jun 12, 2026

8 days since last refresh · source: lastUpdated

Benchmark freshness

How recent the editorial measurements on this runtime are.

0editorial benchmarks

No editorial benchmarks for this runtime yet.

Community reproduction

Submissions that match an editorial measurement on similar hardware.

0reproduced reports

No community reproductions on file yet.

Get Exo

Official site
https://exolabs.net
GitHub
https://github.com/exo-explore/exo

Frequently asked

Is Exo free?

Yes — Exo is free to use and open-source.

What operating systems does Exo support?

Exo supports macOS, Linux.

Which GPUs work with Exo?

Exo supports Apple Metal, NVIDIA CUDA. CPU-only operation is also possible but typically slower.
See something off?Report outdated·Suggest a correctionWe read every submission. Editorial review takes 1-7 days.

Reviewed by RunLocalAI Editorial. See our editorial policy for how we evaluate tools.

Related — keep moving

Compare hardware
  • RTX 4090 vs RTX 5090 →
  • Dual 3090 vs RTX 5090 (tensor-parallel) →
  • RTX 5090 vs H100 →
Buyer guides
  • Best GPU for local AI →
  • Best AI PC build under $2,000 →
When it doesn't work
  • vLLM CUDA version mismatch →
  • Tensor parallelism crash →
  • CUDA driver too old →
  • CUDA out of memory →
Recommended hardware
  • RTX 4090 (24 GB) →
  • RTX 5090 (32 GB) →
  • H100 PCIe (datacenter) →
Alternatives
SGLangText Generation Inference (TGI)vLLMQdrantWeaviateGraphiti (Zep)LanceDBRedis (vector search)
Before you buy

Verify Exo runs on your specific hardware before committing money.

Will it run on my hardware? →Custom hardware comparison →GPU recommender (4 questions) →