EVA Llama 3.3 70B
EVA community's storytelling-focused fine-tune of Llama 3.3 70B. Popular in the creative-writing / roleplay community.
Positioning
EVA Llama 3.3 70B is a dense 70B-parameter fine-tune of Meta's Llama 3.3, released by the EVA-Unit-01 community under the Llama Community License. It is purpose-built for creative writing and roleplay, with a 131,072-token context window that supports long-form narrative generation. As a dense architecture, every forward pass uses all 70B parameters, placing it firmly in the datacenter deployment class.
Strengths
- Massive context window: 131,072 tokens of native context allow for extended story arcs, detailed world-building, and long character histories without truncation.
- Community-tuned for narrative: Fine-tuned specifically for creative writing and roleplay, making it a specialized tool for storytelling use cases rather than a general-purpose model.
- Permissive commercial license: The Llama Community License permits commercial use, making this suitable for studios or platforms deploying narrative-generation services.
- Dense architecture simplicity: Unlike Mixture-of-Experts models, dense models have predictable memory and compute requirements — no routing overhead or expert imbalance to manage.
Limitations
- High hardware barrier: At FP16, the model requires ~140 GB of VRAM just for weights, plus significant overhead for KV cache (30–50% more at full context). This effectively demands multi-GPU datacenter hardware.
- No community benchmarks available: We do not yet have independent measurements of this fine-tune's quality or speed. Published vendor metrics should be treated as best-case.
- Narrow specialization: The model's focus on creative writing may limit its performance on factual, analytical, or coding tasks compared to general-purpose models of similar size.
- Large quant size even at low precision: Even at Q2_K (~22.8 GB), the model requires substantial VRAM, and quantization may degrade the nuanced language generation that creative tasks depend on.
What it takes to run this locally
Running EVA Llama 3.3 70B locally requires datacenter-class hardware. Quantization reduces the weight memory footprint:
- FP16: ~140 GB
- Q8_0: ~74 GB
- Q6_K: ~57.8 GB
- Q5_K_M: ~49.9 GB
- Q4_K_M: ~39.4 GB
- Q3_K_M: ~34.1 GB
- Q2_K: ~22.8 GB
Add 30–50% for KV cache and framework overhead at typical context lengths. A single 80 GB GPU (e.g., A100) can run Q4_K_M or smaller quants, but multi-GPU setups (2–4× A100/H100) are recommended for full-precision or large-batch inference.
Should you run this locally?
Yes if you are a creative studio or power user with access to multi-GPU datacenter hardware and need a model specialized for long-form narrative generation with a permissive commercial license.
No if you lack multi-GPU infrastructure, need a general-purpose model for diverse tasks, or require community-verified performance benchmarks before committing to deployment.
Catalog cross-links
- Llama 3.3 70B
- A100
- H100
Overview
EVA community's storytelling-focused fine-tune of Llama 3.3 70B. Popular in the creative-writing / roleplay community.
How to run it
EVA Llama 3.3 70B is a fine-tune of Llama 3.3 70B. EVA (Excellence in Virtual Agents) is a roleplay/conversational fine-tune designed for natural, engaging, persona-driven dialogue. Run at Q4_K_M via Ollama (ollama pull eva-llama3.3:70b) or llama.cpp with -ngl 999 -fa -c 4096. Q4_K_M file size ~40 GB on disk. Minimum VRAM: 48 GB — RTX A6000 (48GB) at Q4_K_M for 4K context. RTX 4090 24GB: Q3_K_M with KV offload. Throughput: ~15-25 tok/s on A6000 at Q4_K_M. Standard Llama 3.3 architecture. EVA's focus: character consistency, emotional tone, conversational flow — prioritized over factual accuracy or code generation. Use for: roleplay, character simulation, creative dialogue, interactive fiction. Not for: factual Q&A, coding, math, agent tasks. The roleplay tuning may make the model verbose and stylized — expect longer, more emotionally expressive outputs than base Llama 3.3. License: verify on hf (may be non-commercial for EVA models). Context: Llama 3.3 128K (practical 4-8K on 48 GB). Conversational contexts are typically shorter (2-4K) — less KV cache pressure.
Hardware guidance
Minimum: RTX 3090 24GB at Q3_K_M (4K). Recommended: RTX A6000 48GB at Q4_K_M (8K). VRAM math: identical to Llama 3.3 70B — 70B at Q4 ≈ 40 GB. KV cache at 8K: ~10 GB. Total ~50 GB. A6000 48GB: borderline. Dual RTX 4090 48 GB: Q4 at 8K. RTX 4090 24GB single: Q3 with KV offload. Mac Studio M4 Max 64GB: Q4 at 5-10 tok/s. Cloud: A100 80GB at $5-10/hr. Roleplay workloads are latency-sensitive — prioritize tok/s over context length. AWQ-INT4 on A100 gives fastest generation.
What breaks first
- Factual accuracy degradation. EVA's roleplay tuning prioritizes conversational engagement over factual precision. The model will confidently produce incorrect facts to maintain character consistency. 2. Out-of-character breaks. EVA may break character under adversarial prompts or complex logical demands. Character consistency degrades with long contexts (>4K tokens). 3. Repetition and looping. Roleplay-tuned models are prone to conversational loops — repeating phrases, circling back to earlier topics. Set repetition_penalty=1.1-1.15 and use stop sequences. 4. Q3 character degradation. Roleplay quality depends on nuanced language. At Q3, subtle emotional tones and character voice degrade more than factual content would. Use Q4_K_M minimum for character-based use.
Runtime recommendation
Common beginner mistakes
Mistake: Using EVA for factual Q&A or coding. Fix: EVA is roleplay-tuned. Factual accuracy is degraded compared to base Llama 3.3. Use Llama 3.3 70B or Qwen 3 72B for knowledge work. Mistake: Juxtaposing EVA outputs as factual. Fix: EVA prioritizes character consistency, not truth. Always fact-check its statements separately. Mistake: Setting temperature=0 and expecting creative dialogue. Fix: EVA needs temperature 0.7-0.9 for natural conversation. Temp=0 produces robotic, repetitive dialogue. Mistake: Expecting EVA to maintain character over 8K+ context. Fix: Character coherence degrades with long context. Keep conversations focused and under 4K tokens for best character consistency.
Family & lineage
How this model relates to others in its lineage. Family members share architecture and training-data roots; parent / children edges record direct distillation or fine-tune relationships.
Strengths
- Strong creative-writing benchmarks
- Long-context narrative coherence
Weaknesses
- Smaller community than base Llama
Quantization variants
Each quantization trades model quality for file size and VRAM. Q4_K_M is the most popular starting point.
| Quantization | File size | VRAM required |
|---|---|---|
| AWQ-INT4 | 40.0 GB | 48 GB |
Get the model
HuggingFace
Original weights
Source repository — direct quantization required.
Hardware that runs this
Cards with enough VRAM for at least one quantization of EVA Llama 3.3 70B.
Models worth comparing
Same parameter band, plus what's one tier above and below — so you can decide what actually fits your hardware.
Frequently asked
What's the minimum VRAM to run EVA Llama 3.3 70B?
Can I use EVA Llama 3.3 70B commercially?
What's the context length of EVA Llama 3.3 70B?
Source: huggingface.co/EVA-UNIT-01/EVA-Llama-3.3-70B
Reviewed by RunLocalAI Editorial. See our editorial policy for how we research and verify model claims.
Related — keep moving
Verify EVA Llama 3.3 70B runs on your specific hardware before committing money.