Instructor Notebooks — ComplianceGPT Lab REU 2026

Before Class Checklist

Run notebook 05 once to download all-MiniLM-L6-v2 (~80MB, needs internet)
Verify FIFA_PATH in notebooks 01–02 points to your Downloads folder
Verify HIPAA_PATH in notebooks 01–06 points to final_vast_gemma3_4b.csv
Open all 6 in Jupyter before class — kernel warm-up takes ~10s

Data paths

FIFA: /Users/priscilladanso/Downloads/FIFA - 2014.csv

HIPAA: .../finalserverrun/final_vast_gemma3_4b.csv

Jump to: 01 ML · 02 DL · 03 NLP · 04 Transformers · 05 Encoders · 06 LLMs · 07 Prompting 101

Machine Learning

25 cells · FIFA 2014 + HIPAA · sklearn only · no GPU

⬇ teach_01_ml.ipynb

What you cover

Data replaces rules — the core ML idea
Features, labels, the training loop
Logistic Regression (weights)
Decision Tree (readable rules)
Random Forest (ensemble voting)
Pivot: HIPAA text → feature extraction wall

Teaching moment

"The algorithm predicted which teams reached the quarterfinals from stats alone. Now look at a HIPAA scenario — 300 words of legal text. You can't feed that to sklearn. You need to convert text to numbers first. That's everything we do for the rest of this week."

Key outputs

FIFA: 3-model accuracy table
Decision Tree rules printed as text
LR feature weights bar chart
RF feature importances
TF-IDF baseline on HIPAA (teaser)

Requires: pandas, numpy, sklearn | Data: FIFA 2014 CSV · HIPAA results CSV | When: Monday (Day 1)

Deep Learning

17 cells · synthetic + HIPAA · sklearn MLP · no PyTorch required

⬇ teach_02_dl.ipynb

What you cover

XOR — what logistic regression cannot solve
Neural network = layers of learned transforms
Manual neuron: weighted sum → ReLU
Backprop training loop (live loss curve)
Depth vs accuracy on checkerboard problem
Scale table: Perceptron → GPT-4

Teaching moment

"Logistic Regression gets 50% on XOR — a coin flip. The neural network gets 100% with one hidden layer. The hidden layer learned a new coordinate system where the problem is linearly separable. That's the magic of depth."

Key outputs

XOR: LR fails (50%), MLP succeeds (100%)
Live loss curve — 200 epochs printed
Depth comparison: 4 architectures vs checkerboard
Scale table (8 models, params, compute)
HIPAA: MLP vs LR vs DT accuracy

Requires: pandas, numpy, sklearn | Data: HIPAA results CSV | When: Monday (Day 1) or Tuesday

NLP — Text as Numbers

19 cells · real HIPAA scenarios · sklearn · no GPU

⬇ teach_03_nlp.ipynb

What you cover

Text representation problem: numbers only
Bag of Words — manual, then sklearn
IDF intuition: "subpoena" vs "the"
TF-IDF + bigrams — top features per class
Cosine similarity matrix on legal phrases
The semantic gap demonstrated on HIPAA

Teaching moment

"TF-IDF cosine similarity: 'court order' vs 'judicial mandate' = 0.08. Zero word overlap = near-zero similarity. Legally they trigger the exact same HIPAA exception. TF-IDF doesn't know that. This is the semantic gap."

Key outputs

BoW matrix printed on toy sentences
IDF scores for 7 legal words
TF-IDF top features → PERMITTED vs DENIED
6×6 cosine similarity matrix
False negative example from real HIPAA
4-method accuracy comparison

Requires: pandas, numpy, sklearn | Data: HIPAA results CSV | When: Tuesday (Day 2)

Transformers & Attention

19 cells · tiktoken (optional) · numpy only · no GPU

⬇ teach_04_transformers.ipynb

What you cover

Tokenization: text → integer IDs (subword)
Token count distribution across 137 scenarios
RNN information decay simulation
Self-attention: manual attention weights
"order" = court order vs medication order
Attention math (Q, K, V matrices)
Positional encoding (sinusoidal)
Gemma3-4B architecture specs

Teaching moment

"Same word 'order' in two sentences. In the legal sentence, it attends strongly to 'judge' and 'signed'. In the medical sentence, it attends to 'physician' and 'medication'. Same word, different representation. This is what TF-IDF fundamentally cannot do."

Key outputs

Token pieces for 4 HIPAA sentences
RNN decay table: 95% signal lost by token 50
Attention weight bar chart (legal context)
Attention weight bar chart (medical context)
Q/K/V attention matrix (toy, 5 tokens)
Sinusoidal PE table (10 pos × 8 dims)

Requires: numpy, pandas | Optional: tiktoken (pip install tiktoken) | When: Tuesday or Wednesday

Encoders & Sentence Embeddings

13 cells · sentence-transformers · CPU · pre-download model before class

⬇ teach_05_encoders.ipynb

What you cover

Encoder vs Decoder: two paths from one arch
Sentence embeddings: meaning as geometry
Closing the semantic gap (0.08 → 0.85+)
Adversarial failure: "implied court order" still close
HIPAA classification with embeddings
Semantic search on 137 real scenarios
Failure case analysis: nearest neighbor

Teaching moment

"Embedding similarity: 'court order' ↔ 'implied there might be a court order' = 0.79. That phrase is ABOUT court orders, it just isn't one. Embeddings capture topic, not legal validity. This is exactly the error pattern in 3 of our 8 failures."

Key outputs

TF-IDF vs embedding similarity table (12 phrases)
Adversarial analysis: group C near group A
Accuracy: TF-IDF vs embeddings vs LLM+engine
Semantic search: "judicial mandate" → top 5 cases
8 failures: nearest neighbor analysis

Requires: sentence-transformers (auto-installs) | ⚠ Pre-download: run Cell 1 before class (downloads all-MiniLM-L6-v2, ~80MB) | When: Wednesday (Day 3)

LLMs — Real System Results

19 cells · pandas only · Gemma3-4B actual output · no inference required

⬇ teach_06_llm.ipynb

What you cover

System overview: full pipeline walkthrough
94.2% accuracy — what 129/137 means
One case start-to-finish: text → JSON → verdict
The 8 failures: FP vs FN classification
Oracle hallucination anatomy (FP)
Under-extraction anatomy (FN)
Latency analysis (avg 15.8s/case)
Research directions: your 5.8% target

Teaching moment

"These 8 cases are your research targets this summer. Every error is an oracle extraction failure — either the model hallucinated a predicate that wasn't there, or it missed one that was. You've now seen every layer of this system. Which one do you want to fix?"

Key outputs

Stats box: 129/137, latency, GPU info
Full JSON extraction for one correct case
Deep dive: one FP + one FN case
Oracle risk table: which predicates fail most
Latency stats + FP/FN latency comparison
6 research directions printed
show_case(N) for live class exploration

Requires: pandas, numpy, sklearn | Data: HIPAA results CSV | When: Wednesday (Day 3) or Thursday

Prompting 101

35 cells · fully generic, zero HIPAA · local Ollama · has student TODO cells, not fully pre-worked

⬇ prompting_101.ipynb

What you cover

Vague vs. specific prompts, side by side
Zero-shot (worked) → where it breaks on ambiguous cases
Few-shot (worked) → student writes their own
Chain-of-thought on a trick riddle → student writes their own
Reflection: a real 2-pass self-critique, by hand
4 failure modes, each with a broken/fixed prompt pair: format drift, sycophancy, instruction drop, ambiguity

Teaching moment

"Everything here is one small model, generic examples, one or two calls. Our real pipeline runs these same four ideas — zero-shot, few-shot, CoT, reflection — on real HIPAA scenarios at much bigger scale, and the results aren't always intuitive: adding retrieval alone made Qwen2.5:72B worse (92.7%→84.7%), and reflection's real job turned out to be undoing that damage, not adding new capability. You can't guess that from theory — you have to run the ablation."

Key outputs

Side-by-side vague/specific, zero/few-shot, direct/CoT text outputs
Pass-1 vs. pass-2 reflection comparison (by hand, no scoring code)
JSON-validity check on the format-drift pair (a real json.loads() pass/fail)
Closing table: real Qwen2.5:72B ablation numbers as a bridge to Agentic Systems

Requires: requests, a running local Ollama (any small model — llama3.2, gemma2:2b) | Data: none — fully generic, no repo CSVs | When: Monday (Week 3, Day 1) — pairs with slides-prompting-101.html

Quick Reference

#	Topic	Data	Key demo	Day	Download
01	Machine Learning	FIFA 2014 + HIPAA	3 classifiers on FIFA → HIPAA feature wall	Mon	⬇ Download
02	Deep Learning	Synthetic + HIPAA	XOR: LR fails, MLP succeeds; backprop loop	Mon/Tue	⬇ Download
03	NLP	HIPAA CSV	Semantic gap: "court order" ↔ "judicial mandate" = 0.08	Tue	⬇ Download
04	Transformers	HIPAA CSV	"order" = different token depending on context	Tue/Wed	⬇ Download
05	Encoders	HIPAA CSV	Gap closed to 0.85 — adversarial still 0.79	Wed	⬇ Download
06	LLMs	Gemma3 results CSV	8 failures live — `show_case(N)` for any case	Wed/Thu	⬇ Download

The progression these notebooks teach

Each notebook ends at the wall the next one solves. Students can see exactly why each technique was necessary — not just that it happened, but what problem it fixed.

Rules → ~60% → ML + hand features → ~90% → TF-IDF → ~80% → Embeddings → ~87% → LLM + engine → 94.2%