RUNLOCALAIv38
->Will it run?Best GPUCompareTroubleshootStartLearnPulseModelsHardwareToolsBench
Run check
RUNLOCALAI

Independently operated catalog for local-AI hardware and software. Hand-written verdicts. Source-cited claims. Reproducible commands when we have them.

OP·Eruo Fredoline
DIR
  • Models
  • Hardware
  • Tools
  • Benchmarks
TOOLS
  • Will it run?
  • Compare hardware
  • Cost vs cloud
  • Choose my GPU
  • Prompting kits
  • Quick answers
REF
  • All buyer guides
  • Learn local AI
  • Methodology
  • Glossary
  • Errors KB
  • Trust
EDITOR
  • About
  • Author
  • How we make money
  • Editorial policy
  • Contact
LEGAL
  • Privacy
  • Terms
  • Sitemap
MAIL · MONTHLY DIGEST
Get monthly local AI changes
Monthly recap. No spam.
DISCLOSURE

Some links on this site are affiliate links (Amazon Associates and other first-class retailers). When you buy through them, we earn a small commission at no extra cost to you. Affiliate links do not influence our verdicts — there are cards we rate highly that we don't have affiliate relationships with, and cards that sell well that we refuse to recommend. Read more →

© 2026 runlocalai.coIndependently operated
RUNLOCALAI · v38
  1. >
  2. Home
  3. /Learn
  4. /Courses
  5. /Capstone: Research AI System
  6. /Ch. 13
Capstone: Research AI System

13. Experimental Results

Chapter 13 of 18 · 20 min
KEY INSIGHT

Experimental results are evidence, not decoration. Every number in a results section should support a specific claim, and every claim should require supporting evidence. Present results to build an argument: these experiments, when interpreted correctly, support these conclusions. Readers should be able to trace from claim to evidence. ### Organizing Experiments Group experiments by the claim they support: ``` ## 4. Experimental Results ### 4.1 End-to-End Performance (supports main claim) ### 4.2 Ablation Study (supports claim that components matter) ### 4.3 Error Analysis (supports claim about failure modes) ### 4.4 Scaling Behavior (supports claim about generalization) ``` This structure makes it easy for readers to find evidence for specific claims. ### Quantitative Presentation Use tables for comparisons with baselines: ```python # Results table for paper results_table = """ | System | Accuracy | Latency (ms) | Memory (GB) | |--------|----------|--------------|-------------| | Baseline | 91.8% | 234 | 8.2 | | Ours | 94.2% | 198 | 7.1 | | +Compression | 94.0% | 145 | 4.3 | | +Pruning | 93.1% | 89 | 2.8 | """ ``` Use figures for trends and distributions: ```python import matplotlib.pyplot as plt # Scaling behavior visualization def plot_scaling_results(): fig, axes = plt.subplots(1, 2, figsize=(10, 4)) # Left: accuracy vs dataset size axes[0].plot(dataset_sizes, accuracies, 'o-', label='Ours') axes[0].plot(dataset_sizes, baseline_accuracies, 's--', label='Baseline') axes[0].set_xlabel('Training Data Size') axes[0].set_ylabel('Accuracy') axes[0].legend() # Right: latency vs batch size (log-log) axes[1].loglog(batch_sizes, latencies, 'o-', label='Ours') axes[1].loglog(batch_sizes, baseline_latencies, 's--', label='Baseline') axes[1].set_xlabel('Batch Size') axes[1].set_ylabel('Latency (ms)') plt.tight_layout() plt.savefig('figures/scaling_results.pdf') ``` ### Statistical Rigor AI systems have inherent variance. Report uncertainty: - **Standard deviation**: For repeated runs with different random seeds - **Confidence intervals**: For estimated population parameters - **Statistical tests**: For comparing systems (t-test, bootstrap) ```python # Calculate and report confidence intervals def report_with_ci(values, confidence=0.95): import numpy as np from scipy import stats mean = np.mean(values) sem = stats.sem(values) # Standard error of mean ci = stats.t.interval(confidence, len(values)-1, loc=mean, scale=sem) return f"{mean:.2f} ± {sem * 1.96:.2f} (95% CI)" ``` ### Handling Negative Results Don't hide experiments where your approach didn't win. Negative results are valuable information: > "Surprisingly, adding retrieval augmentation decreased performance on tasks requiring precise factual recall (Table 4, rows 3-4). Analysis reveals that retrieved passages occasionally contained contradictory information that confused the model. We address this in Section 5.2."

4. Experimental Results

4.1 End-to-End Performance (supports main claim)

4.2 Ablation Study (supports claim that components matter)

4.3 Error Analysis (supports claim about failure modes)

4.4 Scaling Behavior (supports claim about generalization)


This structure makes it easy for readers to find evidence for specific claims.

### Quantitative Presentation

Use tables for comparisons with baselines:

```python
# Results table for paper
results_table = """
| System | Accuracy | Latency (ms) | Memory (GB) |
|--------|----------|--------------|-------------|
| Baseline | 91.8% | 234 | 8.2 |
| Ours | 94.2% | 198 | 7.1 |
| +Compression | 94.0% | 145 | 4.3 |
| +Pruning | 93.1% | 89 | 2.8 |
"""

Use figures for trends and distributions:

import matplotlib.pyplot as plt

# Scaling behavior visualization
def plot_scaling_results():
    fig, axes = plt.subplots(1, 2, figsize=(10, 4))
    
    # Left: accuracy vs dataset size
    axes[0].plot(dataset_sizes, accuracies, 'o-', label='Ours')
    axes[0].plot(dataset_sizes, baseline_accuracies, 's--', label='Baseline')
    axes[0].set_xlabel('Training Data Size')
    axes[0].set_ylabel('Accuracy')
    axes[0].legend()
    
    # Right: latency vs batch size (log-log)
    axes[1].loglog(batch_sizes, latencies, 'o-', label='Ours')
    axes[1].loglog(batch_sizes, baseline_latencies, 's--', label='Baseline')
    axes[1].set_xlabel('Batch Size')
    axes[1].set_ylabel('Latency (ms)')
    
    plt.tight_layout()
    plt.savefig('figures/scaling_results.pdf')

Statistical Rigor

AI systems have inherent variance. Report uncertainty:

  • Standard deviation: For repeated runs with different random seeds
  • Confidence intervals: For estimated population parameters
  • Statistical tests: For comparing systems (t-test, bootstrap)
# Calculate and report confidence intervals
def report_with_ci(values, confidence=0.95):
    import numpy as np
    from scipy import stats
    
    mean = np.mean(values)
    sem = stats.sem(values)  # Standard error of mean
    ci = stats.t.interval(confidence, len(values)-1, loc=mean, scale=sem)
    
    return f"{mean:.2f} ± {sem * 1.96:.2f} (95% CI)"

Handling Negative Results

Don't hide experiments where your approach didn't win. Negative results are valuable information:

"Surprisingly, adding retrieval augmentation decreased performance on tasks requiring precise factual recall (Table 4, rows 3-4). Analysis reveals that retrieved passages occasionally contained contradictory information that confused the model. We address this in Section 5.2."

EXERCISE

Create a results presentation for your research system covering: (1) one table comparing your system to at least two baselines, (2) one figure showing a trend or distribution, (3) one ablation experiment showing component contributions. For each, write the claim it supports and ensure the visualization makes that claim obvious.

← Chapter 12
Paper Structure
Chapter 14 →
Visualization