How We Fixed an AI Eval That Said 50% When the Model Was 97% Accurate
Our eval accuracy drop to 50% wasn't the model. Half the golden set pointed at retired IDs. How we remapped 1,009 items by content hash and fixed the labels.
Utkarsh Gupta
machine-learning
Utkarsh GuptaHow We Built an LLM Document Classifier That Labels Every Document in One Call
How we built an LLM document classifier: one model call returns five fields, label rules written as decision tests, our own schema check and a weekly eval.
Utkarsh Gupta