Foundations · Cluster Access

SLURM Quick Connect

Connect to the AI Cluster, submit an extraction job, download results, run Soufflé locally

Critical architecture: The cluster runs LLM extraction only — no Soufflé on the cluster. You extract on the GPU, download the CSV, then run Soufflé on your laptop.
Cluster (GPU)
llm1_extractor.py
→ extractions CSV
→ download →
Your Laptop
hipaa_connector.py → Soufflé
→ verdicts + analysis

Step 0 — Connect

You need cluster access first — email rt@cs.stonybrook.edu with your NetID before Tuesday. Then choose your connection method:

  1. Open Terminal (Cmd+Space → "Terminal")
  2. Connect: ssh yournetid@submit.ai.stonybrook.edu
  3. First time: type yes to accept the host key
  4. Enter your NetID password when prompted (nothing shows as you type — that's normal)

Optional shortcut — add to ~/.ssh/config so you just type ssh ai:

Host ai HostName submit.ai.stonybrook.edu User yournetid
  1. Install VS Code if you don't have it yet
  2. Install the Remote - SSH extension (search in Extensions panel: ms-vscode-remote.remote-ssh)
  3. Verify OpenSSH is installed — open PowerShell and run: ssh -V Should print something like OpenSSH_8.x. If not, install via Windows Settings → Optional Features → OpenSSH Client.
  4. In VS Code: Ctrl+Shift+P → type Remote-SSH: Add New SSH Host
  5. Enter: ssh yournetid@submit.ai.stonybrook.edu
  6. Click Connect to Host → choose Linux when asked about OS
  7. Enter your NetID password
Tip: Once connected, VS Code shows the cluster filesystem in the file Explorer and gives you a built-in terminal at the bottom. You can drag-drop files to download.
  1. Download PuTTY from putty.org
  2. Host Name: submit.ai.stonybrook.edu · Port: 22 · Connection type: SSH
  3. Click Open, type your NetID and password
  4. For file transfer, also download WinSCP — use the same host/port/credentials
VS Code is strongly recommended over PuTTY. VS Code lets you browse files, drag-drop downloads, and edit scripts visually — much easier for research.

Step 1 — First-Time Setup (do this once)

After your first login, run these once to set up your environment. Paste them one at a time.

1a

Load modules and add to your shell profile

module load slurm python3 echo "module load slurm python3" >> ~/.bashrc
1b

Install Miniconda (lightweight Python manager)

wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh # accept all defaults source ~/.bashrc
1c

Create the ComplianceGPT conda environment

conda create -n compliancegpt python=3.10 -y conda activate compliancegpt pip install torch transformers accelerate pandas tiktoken
1d

Create working directories

mkdir -p ~/compliancegpt/results ~/compliancegpt/logs
1e

Test GPU access — submit this job and check the output

# Create test job cat > ~/test_gpu.sh << 'EOF' #!/bin/bash #SBATCH --job-name=test_gpu #SBATCH --output=/home/YOURNETID/logs/test_%j.txt #SBATCH --time=0-00:05 #SBATCH --mem=4000 #SBATCH --gres=gpu:1 source ~/miniconda3/etc/profile.d/conda.sh conda activate compliancegpt python -c "import torch; print('CUDA:', torch.cuda.is_available()); print('GPU:', torch.cuda.get_device_name(0))" EOF # Replace YOURNETID above, then: sbatch ~/test_gpu.sh

Wait 1–2 min, then check output: cat ~/logs/test_*.txt. Should say CUDA: True and a GPU name.

Step 2 — Run LLM Extraction (Exercise 9)

This is the actual research job. It runs llm1_extractor.py on the GoldCoin dataset and writes a CSV of JSON extractions. No Soufflé here.

2a

Create the extraction job script

nano ~/compliancegpt/run_extraction.sh

Paste this content (replace YOURNETID):

#!/bin/bash #SBATCH --job-name=cg_extract #SBATCH --output=/home/YOURNETID/compliancegpt/logs/extract_%j_out.txt #SBATCH --error=/home/YOURNETID/compliancegpt/logs/extract_%j_err.txt #SBATCH --time=0-04:00 # 4 hours max #SBATCH --mem=32000 # 32 GB RAM #SBATCH --gres=gpu:1 # 1 GPU (do not request more) source ~/miniconda3/etc/profile.d/conda.sh conda activate compliancegpt # Extraction only — Soufflé runs locally after you download this CSV python ~/compliancegpt/connector/llm1_extractor.py \ --model google/gemma-3-4b-it \ --data ~/compliancegpt/data/goldcoin_hhs_merged.csv \ --output ~/compliancegpt/results/extractions_$SLURM_JOB_ID.csv
2b

Submit and monitor

sbatch ~/compliancegpt/run_extraction.sh # submit — note the job ID printed squeue -u yournetid # check status (PD=pending, R=running) tail -f ~/compliancegpt/logs/extract_*_out.txt # watch live output

The job typically takes 3–8 minutes per scenario on a V100. Full 137-scenario run takes ~15–30 minutes. Ctrl+C to stop tail — it won't cancel the job.

2c

Verify the output when done

ls -lh ~/compliancegpt/results/ # confirm CSV exists head -2 ~/compliancegpt/results/extractions_*.csv # peek at first 2 rows sacct -u yournetid --format=JobID,Elapsed,State # job history

Step 3 — Download the Extraction CSV

In a local terminal on your laptop (not on the cluster):

scp yournetid@submit.ai.stonybrook.edu:\ ~/compliancegpt/results/extractions_JOBID.csv \ ~/Downloads/

Replace JOBID with the number printed when you ran sbatch. Or use * to grab all CSVs: .../results/extractions_*.csv

  1. In VS Code with the cluster connected, open the Explorer panel
  2. Navigate to /home/yournetid/compliancegpt/results/
  3. Right-click the CSV → Download
  4. Choose your local Downloads folder
  1. Open WinSCP · Host: submit.ai.stonybrook.edu · Port: 22 · Protocol: SFTP
  2. Login with your NetID and password
  3. Navigate to /home/yournetid/compliancegpt/results/
  4. Drag the CSV to your local Downloads folder on the left panel

Step 4 — Run Soufflé Locally (on your laptop)

Open week2_nlp.ipynb on your laptop. Go to Exercise 9, Step 3. Set EXTRACTION_CSV to your downloaded file path and run the cell — it feeds each scenario_json to the local Soufflé engine and writes a new CSV with verdict_norm added.

Why this split? LLM inference (thousands of forward passes through a 4B-parameter model) takes hours on CPU but minutes on GPU. Soufflé (pure logic evaluation) takes milliseconds on any machine. Run the expensive part on the cluster; everything else stays local.

SLURM Commands Reference

CommandWhat it does
sbatch myjob.shSubmit a job — returns a job ID
squeue -u yournetidShow your queued/running jobs (PD=pending, R=running)
squeueShow all jobs on the cluster
scancel JOBIDCancel a job
sacct -u yournetidJob history including elapsed time and exit status
sinfoShow all nodes and their availability
tail -f logfile.txtWatch a log file update in real time (Ctrl+C to exit)

Job Script Flags — ComplianceGPT Defaults

FlagValueWhy
--gres=gpu:11 GPUOne GPU is enough for 4B inference. Do not request more — other users need them.
--mem=3200032 GB RAMGemma3:4B needs ~24 GB VRAM + system overhead
--time=0-04:004 hoursFull 137-scenario run takes ~30 min; 4 hrs gives buffer
--output=..._%j.txt%j = job IDUnique log file per job — don't clobber previous runs

Troubleshooting

ProblemFix
Connection refused / timeoutYou're not on campus network or VPN — connect to SBU VPN first (vpn.stonybrook.edu)
Permission deniedWrong NetID or password. Try again. Password is case-sensitive.
Job stuck in PD foreverAll GPUs busy. Run sinfo to see which nodes are free. Try adding --nodelist=gpu1 to target a specific node.
conda: command not foundRun source ~/.bashrc or log out and back in
CUDA: False in test jobForgot --gres=gpu:1 in your #SBATCH flags — job ran on a CPU node
Empty output CSVCheck logs/extract_*_err.txt — usually a Python import error or wrong file path
Can't download CSVMake sure you're running scp from your laptop terminal, not from inside the cluster

Questions?

Slack #cluster-help or email priscillakyeidanso@gmail.com. Include your job ID and paste the contents of your error log.

Full hardware specs and advanced usage: cluster.html