Context over prompts and LLM judges for subjective output
Why prompt wording and LLM judges failed to improve a model's taste in outfits, what measuring its context revealed, and why a judge must pass the contrast that matters before it gates output.

On this page
Give a capable model twelve different style frameworks and it can still return essentially the same outfit every time, down to the same stripe in ten of them. When an application produces subjective output, most engineers reach for two levers first: prompt engineering with personas, aesthetic directives, rules for each failure mode, and worked examples, followed by an LLM-as-a-judge to score candidates. Both are sensible defaults that pay off quickly on structured tasks with explicit rubrics.
Neither moved taste here, and both failed measurably. A model’s output follows the data in its context window far more than prose telling it how to think, and a judge’s accuracy on an easy contrast says nothing about its performance on the narrower decision boundary a user cares about. In practice, that meant engineering the context first, keeping factual constraints in deterministic checks, and refusing to put a judge in the loop until it passes a held-out bar on the contrast that matters.
I hit this building Wardrobe, a personal daily outfit composer where a host model such as Claude, Codex, or ChatGPT assembles four outfits a day from roughly 135 to 200 eligible garments in my closet via tool calls to a thin server that stores the catalog and runs deterministic checks for warmth and other stated rules.
Prompt wording and rule amplification
Changing the wording of the brief barely shifted the output distribution, and naming concrete fixes made results worse through literal amplification. Whatever a brief names, the model puts everywhere: mentioning bright socks put them on every outfit, and a single mention of a cardigan in a sentence about layering removed knitwear from four test weeks. Worked examples produced copying rather than generalization, with one popover out of fifty garments reappearing in nearly every run.
The composing brief now describes qualities without naming a garment, carries no worked examples and no negative constraints, and the daily catalog payload holds only what can be worn. Factual constraints stay in the deterministic checker, which reports broken facts such as insufficient warmth for the day’s weather; rules remove bad outfits reliably, but they cannot generate taste.
Measuring the context before rewriting the prompt
Inspecting the payload reaching the model exposed an engineering bug posing as a taste problem. One day’s context ran to roughly 110,000 characters against the host’s 100,000-character tool-result limit, so tool output was silently truncated and the model never saw the garments at the tail of the list.
A second truncation cut individual garment descriptions at 280 characters. Because those descriptions opened with construction details, the model received an inventory of pockets and zips while losing color undertone, cut, and how a pattern reads at a distance.
Replacing raw prose with a compact composing card per garment fixed what enters the context window: color, cloth, shape, heritage, wearing notes, and warmth. No card names a partner garment, since a named partner collapses into a fixed pairing across runs. Order and proportion inside the payload act as selection levers as well: sorting each category least-recently offered first places neglected pieces at the front of the context window, and counting four plain blue oxfords as one removes frequency skew from duplicate staples.
Testing an LLM judge on the contrast that matters
Because I was not going to act as a daily human rater, I tested whether an LLM judge could replace my eye. The first calibration run scored below chance, which looked like a broken judge until a label audit pointed to the dataset. The same outfit sat in both the positive and negative piles, and verdicts recorded against a four-outfit daily set had been applied uniformly to individual outfits within it.
With cleaned labels, I ran a preregistered evaluation: all judge variants frozen before inference, separate tuning and held-out sets, three runs per configuration, a usefulness bar fixed in advance at 0.85 AUC, a budget ledger, and an archive that replays every recorded result offline at zero cost.
Every judge separated styled lookbook outfits from Drake’s, the menswear house whose standard I am aiming for, from deliberately built failures, scoring up to 0.93 AUC. When the task narrowed to separating outfits I loved from ones that were merely fine, every variant missed the 0.85 threshold, topping out at 0.65 AUC on the held-out set. Passing the easy control showed that these judges detect obvious failure, and that signal did not transfer to the narrower decision boundary.
What runs in production
No LLM judge gates production. Optimizing against a judge that fails the loved-versus-fine boundary would only tune the composer away from obvious failures it already avoids. Instead, the host model composes twelve candidates a day and a deterministic variety selector picks four, cutting adjacent-day repeats of main pieces from 24 to about two a week. Because the selector optimizes directly for garment diversity across days, that drop confirms an operational improvement in rotation rather than independent evidence of better taste.
Chris Chabot, November 2026