Compare two AI answers with a rubric before seeing the model name
Use the same task and evidence, score factual support and usefulness separately and inspect the failures behind your preference.
In brief
To compare answers fairly, hide the model names at first and judge the same task against defined requirements. Attractive wording should not outweigh a wrong instruction or invented source.
A simple rubric keeps a personal preference from being presented as a general performance benchmark.
Define the criteria
For a troubleshooting answer, check platform accuracy, a sensible diagnostic order and the limits of each fix. For a summary, check omitted conditions and source support. A longer answer gets no automatic bonus; extra unsupported details can reduce its usefulness.
Record what failed
Use a simple qualitative record such as meets requirement, partly meets or fails, with a reason. Do not turn illustrative rubric values into published model scores. Verify factual claims yourself rather than letting one candidate judge the other without evidence.
Report only the tested scope
If you repeat the exercise, preserve task, date, models and conditions. Describe results as applying to those examples. Keep a few held-back cases if tuning prompts, and do not advertise a universal ranking from a private preference or one favourable output.
Before you act
One comparison does not establish a winner for all tasks. Output variation, different tools and unequal evidence can make a comparison misleading.
What to do next
- Give both candidates the same permitted input, evidence and output requirements.
- Review factual support, task completion and clarity using a rubric defined beforehand.
- Reveal names afterward and repeat representative cases before drawing a scoped conclusion.
