20. Personal Hardware Plan Project

This final project synthesizes the course concepts into a concrete hardware plan for your specific situation.

Project Structure

Create a document containing:

1. Current situation assessment
2. Use case definition
3. Budget constraints
4. Hardware recommendation
5. Alternative configurations
6. Timeline and staged approach

Section 1: Current Situation Assessment

Document your starting point:

## Current Hardware

- GPU: [Model and VRAM]
- System RAM: [Amount and speed]
- Storage: [Type, capacity, speed]
- CPU: [Model]
- Budget ceiling: $[Amount]

## Current Pain Points

1. [Specific frustration with current setup]
2. [Specific model not usable]
3. [Performance unacceptable for X use case]

Section 2: Use Case Definition

Define specific workloads:

## Primary Use Cases

| Use Case | Model | Precision | Context | Concurrent Users |
|----------|-------|-----------|---------|-----------------|
| Coding assistant | Mistral 7B | INT4 | 16K | 1 |
| Document analysis | 13B | INT8 | 8K | 1 |
| Batch summarization | 70B-8B ensemble | INT4 | 4K | Background |

## Frequency

- Daily interactive use: ~4 hours
- Weekly batch processing: ~8 hours
- Monthly fine-tuning experiments: ~24 hours

Section 3: VRAM Calculation

Calculate minimum requirements:

## VRAM Calculation

Base model requirements (per Chapter 2):

| Model | FP16 (GB) | INT8 (GB) | INT4 (GB) |
|-------|-----------|-----------|-----------|
| Mistral 7B | 14 | 8 | 4 |
| Llama 3 13B | 26 | 14 | 7 |
| Llama 3 70B | 140 | 74 | 37 |

Context requirements: 1-4GB depending on length
Overhead factor: 1.25x

Recommended VRAM: 16GB (for 13B INT4 with headroom)

Section 4: Hardware Recommendation

## Recommended Configuration

### Option A: Mid-Range Build (~$1500)

| Component | Selection | Price |
|-----------|-----------|-------|
| GPU | RTX 4070 Ti 12GB | $700 |
| CPU | Ryzen 7 7700X | $250 |
| RAM | 32GB DDR5 | $100 |
| Storage | 1TB NVMe | $80 |
| PSU | 850W | $100 |
| Case | Mid-tower | $80 |
| **Total** | | **$1310** |

### Option B: High-Performance Build (~$2800)

[Full RTX 4090 build details]

### Option C: Hybrid Approach ([$600] eGPU + existing system)

[Details for adding eGPU to current setup]

Section 5: Alternative Configurations

Include at least three alternatives:

1. Cloud-first approach: Use existing hardware, pay $X/month for cloud A100
2. Used hardware route: RTX 3090 24GB used at $Y, with savings used for RAM
3. Apple Silicon path: MacBook Pro M3 Max for unified memory experience

Calculate break-even points:

## Cloud vs. Purchase Analysis

Cloud A100 40GB: $1.50/hr × 730 hrs (24/7) = $1,095/month
RTX 4090 24GB purchase: $1,700 + $50/month electricity = $2,300 one-time.

Break-even: At 1,533 hours/month use, purchased GPU becomes cheaper.
Current projection: 200 hours/month actual use.
Cloud remains cheaper for this usage pattern.

Section 6: Implementation Timeline

## Implementation Plan

### Phase 1 (Immediate): Cloud + existing hardware
- Maintain current setup
- Use Vast.ai spot instances for 70B model access
- Cost: ~$100/month

### Phase 2 (Month 3): Mid-range upgrade
- Purchase GPU and core components ($1,300)
- Decommission cloud (save $800/month)
- ROI: 1.6 months

### Phase 3 (Month 12): Evaluation
- Assess if current GPU meets growing needs
- Plan for RTX 5090 / next generation upgrade
- Revisit cloud usage patterns

Submission Checklist

Before finalizing your plan:

• VRAM calculation completed with specific numbers
• At least three configurations compared
• Real prices researched (not estimated)
• Break-even analysis for cloud vs. purchase
• Timeline considers actual budget flow
• Failure modes acknowledged and mitigated

My Hardware Plan for Local AI

Current Status

• GPU: nvidia-smi --query-gpu=name,memory.total
• RAM: [system info]
• Primary models: [List of models you run regularly]

VRAM Requirements

Model	FP16 (GB)	INT8 (GB)	INT4 (GB)
[Your table here]

Recommended Build

[Your specific configuration]

Purchase Decision

Cloud [$X]/month vs. Purchase [$Y] one-time Break-even: [Calculated months]

## Course Summary

This course covered the essential hardware knowledge for running AI models locally. You learned that VRAM is the primary constraint on model selection, how to calculate precise VRAM requirements, and how to select GPUs across budget tiers. The course included practical guidance on supporting components (motherboard, PSU, cooling), alternative platforms (AMD ROCm, Apple Silicon), and complementary strategies (eGPU, cloud fallback). Multiple build configurations from entry-level to high-performance provided concrete examples. Armed with this knowledge, you can make informed hardware decisions that match your specific workload, budget, and timeline.

Remember: Hardware planning is not about purchasing the most expensive system—it is about purchasing the right system for your actual use case at the lowest necessary cost.