Ollama Playground — ComplianceGPT Lab REU 2026

No notebook, no Python needed for this one. Everything below runs directly in your terminal. If you haven't installed Ollama yet, see Setup & Tools first (brew install ollama on Mac, or ollama.com/download otherwise) — then come back here.

0. Check It's Running 1 min

Start the background server (skip if it's already running — most installs auto-start it):

ollama serve # leave this running in its own terminal tab, or skip if already running

In a new terminal tab, confirm it's alive:

ollama list # shows models you already have downloaded — probably empty right now

1. Your First Model 3 min

Pull and run a small, fast model. This downloads once (a few GB) then drops you into an interactive chat right in the terminal:

ollama run gemma2:2b

Try asking it things, one line at a time:

>>> What is the capital of France? >>> Write a haiku about debugging code. >>> Explain what a REU is to a 10 year old. >>> What's 17 * 24? >>> Who won the most recent Super Bowl?

That last one is deliberate — watch what happens. A local 2B model has a knowledge cutoff baked in at training time. It doesn't know today's date and doesn't know anything that happened after it was trained — it will either say so, or (more interestingly) confidently guess wrong. Note which one it did.

Type /bye to exit back to your normal terminal.

2. Where a 2B Model Struggles 5 min

Still in gemma2:2b (or restart with ollama run gemma2:2b), try these — this is where "small and local" starts showing its limits:

>>> A farmer has 17 sheep. All but 9 die. How many are left? >>> If it takes 5 machines 5 minutes to make 5 widgets, how long does it take 100 machines to make 100 widgets? >>> Write a Python function to check if a number is prime, then explain the time complexity. >>> List the last 5 winners of the Nobel Prize in Physics with the year and their discovery.

Watch for: wrong arithmetic on the "trick" riddles (the sheep one is a classic — answer is 9, many small models say 8), confidently wrong Nobel Prize facts (this is hallucination — fluent, specific, and false), and whether the code answer actually runs.

3. Swap Models — Same Questions, Bigger Brain 8 min

Pull one or two bigger (but still laptop-sized) models and re-run the exact same prompts from Step 2:

ollama run llama3.2 # ~3B params, Meta ollama run phi3:mini # ~3.8B params, Microsoft, tuned for reasoning ollama run qwen2.5:7b # ~7B params — if your laptop has 16GB+ RAM

Ask each one the sheep riddle and the widgets riddle again. Does a bigger model get them right where the 2B one didn't? Do all of them still hallucinate on the Nobel Prize question?

What to notice: reasoning (the riddles) tends to improve clearly with size. Factual recall of obscure/recent things stays unreliable at every size you can run on a laptop — bigger local models are not automatically "more truthful," they're often just more fluently wrong.

4. Compare Against SOTA 5 min

Open claude.ai or chatgpt.com in a browser tab next to your terminal. Ask the exact same sheep riddle, widgets riddle, and Nobel Prize question there.

Question	gemma2:2b (local)	Claude / ChatGPT (SOTA)
Sheep riddle	often wrong	usually right
Widgets riddle	often wrong	usually right
Nobel Prize (recent)	hallucinates or refuses	usually correct, or honestly says it's unsure of the very latest

This is the real headline: SOTA models are trained on vastly more data with vastly more compute, so reasoning and factual recall both improve — but they run on someone else's server, cost money per token, and you can't inspect or modify them. That trade-off (capability vs. control/cost) is exactly why this lab's research pipeline uses both: small local models via Ollama for fast iteration, and a rented GPU running Qwen2.5:72B (still open-weight, still self-hostable — just too big for a laptop) for the strongest results that need to be reproducible.

5. Useful In-Chat Commands 2 min

While inside ollama run <model>, these work at the >>> prompt:

/bye # exit /clear # wipe conversation history, start fresh (still same model loaded) /set system "You are a pirate. Answer everything in pirate speak." /set parameter temperature 0 # 0 = deterministic, same question -> same answer every time /set parameter temperature 1.2 # high = more random/creative, less consistent /show info # model details: size, quantization, context length

Try /set system with a persona, then re-ask one of your earlier questions — this is role/persona prompting from the Prompting 101 deck, live.

6. One-Shot Terminal Calls (No Interactive Chat) 2 min

You don't have to enter the chat mode — ollama run also accepts a prompt directly as an argument, useful for scripting later:

ollama run gemma2:2b "Summarize the plot of Romeo and Juliet in exactly 2 sentences."

Pipe a file in as context:

cat some_notes.txt | ollama run gemma2:2b "Summarize the following notes in 3 bullet points:"

Model Size Cheat Sheet

Model	Params	Good for laptop?	Character
gemma2:2b	2B	Yes — fastest	Fast, weakest reasoning, most hallucination-prone
llama3.2	~3B	Yes	Balanced, good general default
phi3:mini	~3.8B	Yes	Tuned to punch above its size on reasoning/math
qwen2.5:7b	7B	Usually (16GB+ RAM)	Noticeably stronger reasoning, slower
gemma3:4b	4B	Yes	The model used in this lab's early experiments
qwen2.5:72b	72B	No — needs an 80GB-class GPU	What this lab rents a GPU for; see Cluster Access

Wrap-Up Question to Discuss

If a 2B model hallucinates confidently and a 72B/SOTA model mostly doesn't, but the 72B model needs a rented GPU and costs real money per experiment — when is "good enough and fast" the right research choice, and when do you actually need the expensive model? This is the exact trade-off Week 3's LLM Foundations slides cover with real accuracy numbers from this lab's own experiments.