09. Your First Conversation

Interacting with Your Local Model

Now that you have Ollama running with a model, let's have a real conversation. Don't just test it—actually use it for something.

A coding task:

>>> Write a Python function that takes a list of numbers and returns the median

def median(numbers):
    sorted_nums = sorted(numbers)
    n = len(sorted_nums)
    if n % 2 == 0:
        return (sorted_nums[n//2 - 1] + sorted_nums[n//2]) / 2
    else:
        return sorted_nums[n//2]

Not bad. It works. But notice it doesn't include docstring, type hints, or example usage. We can ask:

>>> Add type hints and a docstring

def median(numbers: list[float | int]) -> float:
    """
    Calculate the median of a list of numbers.
    
    Args:
        numbers: A list of numbers (int or float)
        
    Returns:
        The median value as a float
        
    Raises:
        ValueError: If the list is empty
    """
    if not numbers:
        raise ValueError("List cannot be empty")
    
    sorted_nums = sorted(numbers)
    n = len(sorted_nums)
    if n % 2 == 0:
        return (sorted_nums[n//2 - 1] + sorted_nums[n//2]) / 2
    else:
        return sorted_nums[n//2]

Analyzing a document:

>>> Here's a paragraph from a contract: "Party A shall deliver all materials 
within thirty (30) business days of the Effective Date." Summarize the key 
obligation and identify any potential ambiguity.

Key obligation: Party A must deliver materials within 30 business days 
of contract start.

Potential ambiguities:
1. "Business days" - excludes weekends and holidays, making the actual 
   deadline longer than 30 calendar days
2. "All materials" - doesn't specify what materials or how to determine 
   completeness
3. No penalty clause - no specified consequence if delivery is late
4. "Effective Date" - assumes the contract's effective date is unambiguous 
   and clearly established elsewhere

Writing assistance:

>>> Write three different email openings for a client follow-up: one formal, 
one friendly, one urgent

Formal:
"I hope this message finds you well. I am writing to follow up on our 
previous conversation regarding the project timeline."

Friendly:
"Hey! Just wanted to check in and see how things are going with the 
deliverables we discussed."

Urgent:
"I'm reaching out as the deadline for the phase 2 deliverable is 
approaching, and I wanted to ensure we're aligned on next steps."

What "Good" Looks Like

Your first conversations might feel disappointing—especially compared to ChatGPT. A few reasons:

1. Model capability: Llama 3.2 7B is not GPT-4. It's a good local model, but it's smaller. Adjust expectations.

2. Prompting matters: Vague prompts get vague answers. Specific prompts get specific answers. We'll cover this in detail in Chapter 12.

3. Context accumulation: The model performs better when you iterate on responses. Don't expect perfect first drafts—expect good starting points.

Practical Tips

Ask for format:

>>> Summarize this as bullet points
>>> Put this in a table
>>> Write this as a Python class

Ask for alternatives:

>>> Give me 3 different approaches to this problem

Iterate:

>>> Good, but make it more concise
>>> Add error handling
>>> Explain this part I don't understand