Run a test

You need an access code. Each code runs one complete test. If a run fails on our side, the same code works again.

Access code

Your questions (CSV)Columns: id, question, and optionally category, context, reference_answer. Download an example · Blank template

How to format your file

A plain CSV file, comma separated, UTF-8, one question per row, first row is the header.

Column	Required	What to put in it
`id`	Yes	A short unique label for the question, for example q1, q2.
`question`	Yes	The question exactly as your customer would ask it.
`category`	Optional	A topic such as refunds or shipping. The report groups results by it.
`context`	Optional	Text given to the models together with the question, such as the policy paragraph that answers it. Answers are also checked against it.
`reference_answer`	Optional	The answer you consider correct. Only the judge sees it, the models do not.

Limits: up to 200 KB, up to 8,000 characters per cell, up to the number of questions your code includes. Accuracy can only be scored where a question has a context or a reference_answer. To get the cost decisions, at least 90% of your questions need one of them; models only know what you put in context.

Model A

Model B

Judge modelOnly models that can return structured answers are listed. A cheaper judge lowers the cost, a stronger judge scores more reliably.

What to measure

Report languageThe report and the judge’s comments are written in this language.

Instructions for all models

Name this test

Answers per month in production

I understand my questions are sent to OpenRouter and to the providers of the models I chose, and I will not upload personal or confidential data I am not allowed to share.