Six weeks. Real research. Real AI.
"Can we build AI systems that make reliable, explainable decisions about whether an action complies with a legal regulation?"
Legal compliance is expensive, error-prone, and requires expert knowledge. Hospitals spend billions on compliance staff.
LLMs can read legal text and extract structured information. But they hallucinate. We need a way to make them reliable.
Combine LLM extraction with formal logic. The LLM reads the scenario. A rule engine makes the verdict. Separate the concerns.
The same patient question can trigger obligations under multiple regulations simultaneously.
| Patient Question | HIPAA | GDPR | CCPA | COPPA | SOX | GLBA |
|---|---|---|---|---|---|---|
| "Can I see my medical records?" | ✓ | ✓ | ✓ | – | – | – |
| "Can I delete my records?" | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ |
| "Who can share my health data?" | ✓ | ✓ | ✓ | ✓ | – | – |
| "Can my bank share my health info?" | ✓ | ✓ | ✓ | – | – | ✓ |
| "Do I have the right to be forgotten?" | – | ✓ | ✓ | ✓ | ✓ | – |
Each ✓ means that regulation has something to say about this question. Answering correctly requires knowing which laws apply, and what each one requires.
Protects Protected Health Information (PHI) — health data that can identify a patient. Governs hospitals, clinics, pharmacies, insurers. Permits disclosures for treatment, payment, operations, and specific legal exceptions.
Covers all personal data of EU residents — not just health. Key rights: right of access, right to erasure, right to portability. Requires a lawful basis for processing. Fines up to 4% of global revenue.
California residents' rights: know what data is collected, delete it, opt out of sale. Similar in spirit to GDPR but narrower scope and US-specific. Amended by CPRA (2023).
Protects children under 13. Requires parental consent before collecting personal data from minors. Overlaps with GDPR and HIPAA when the patient is a child.
Corporate financial integrity law. Public companies must keep accurate financial records and can't destroy them (Section 802). Relevant when health data overlaps with billing and financial audit trails.
Financial institutions must protect customers' non-public personal information (NPI). Analogous to HIPAA but for banks and insurers. Overlap with HIPAA when a health insurer is also a financial institution.
A neuro-symbolic pipeline: neural language model extracts structure, a formal logic engine checks compliance.
The LLM must extract a structured JSON "scenario" from the narrative. The most critical fields are called oracle predicates.
"A hospital responded to a police officer's request for a patient's billing records. The officer presented a subpoena signed by a judge requiring disclosure."
{
"sender": "hospital",
"receiver": "police_officer",
"phi_type": "billing_records",
"purpose": "law_enforcement",
"has_court_order": true ← oracle predicate
}
Oracle predicates are boolean flags for HIPAA enabling conditions. The formal engine cannot grant PERMITTED without them.
has_court_order — is there a real court order?obtained_authorization — did the patient consent?is_required_by_law — is disclosure legally mandated?If an LLM halluccinates has_court_order: true when no court order exists, the system issues a false PERMITTED verdict — a potential HIPAA violation.
A 4B parameter local model (Gemma3) ties a ~200B frontier model (Claude Sonnet) at 94.2% accuracy. Model size doesn't win — extraction quality does. This is what your research will explore.
5.8 percentage points separate our best model from the oracle ceiling. All remaining errors are extraction failures — the LLM reading the scenario wrong — not formalization errors. This is fixable.
You will propose your project in Week 3. Until then, explore all of them.
Gemma vs. Llama vs. Qwen — which model makes which errors? Build a systematic error taxonomy.
Can you craft scenarios that fool the system into a false PERMITTED verdict? What makes an attack succeed?
Zero-shot vs. few-shot vs. chain-of-thought vs. schema-constrained. Which prompting strategy wins on compliance extraction?
Are the natural-language explanations ComplianceGPT generates actually correct and useful? Rate 50 explanations on a rubric.
Does a prompt that works for HIPAA also work for GDPR? What happens when the same question triggers both laws?
Build the first formal taxonomy of LLM extraction errors in legal compliance. Which error types can targeted prompts fix?
Name, school, major, and: one regulation you've heard of before today.
It's okay to say "HIPAA because of a doctor's office" or "GDPR because of cookie popups." That's exactly the right starting point.
"Can you figure out which regulation (or regulations) applies to a given question — and then determine what the answer actually is?"
Everything you learn this summer — prompting, RAG, error analysis, benchmarking — feeds into answering this.