Claude Code 学习站

CCA-F 练习题库· Prompt Engineering & Structured Output

Which method is most scalable for evaluating prompt quality across hundreds of test cases?

  1. AManual human review of every output
  2. BLLM-as-judge: use Claude to score outputs against a rubric, combined with automated unit tests for structured outputs
  3. CA/B testing with real users
  4. DAsk the model to score its own output

正确答案:B

解析

  • Bis correct: Anthropic's own evaluation guidance recommends combining scalable, automated grading methods: code-based/rule-based checks (unit tests) for objective and structured outputs, plus LLM-as-judge (using a model like Claude to score against a rubric via Likert/binary/ordinal scales) for subjective criteria — explicitly prioritizing volume over hand-graded quality.This is exactly what B describes, making it the most scalable choice for hundreds of test cases.
  • Amanual review of every output is ground truth but does not scale and is reserved only for high-stakes final validation.
  • C(A/B testing with real users) measures production outcomes rather than offline eval quality and depends on live user traffic, so it is slow and not suited to scoring a fixed test set.
  • D(self-scoring) suffers from self-preference/consistency bias and is unreliable at scale — for LLM grading you want an independent judge and a rubric, not the model rating its own output.Only B is right, and no other option is defensibly correct, so the single-answer key is unambiguous.

延伸阅读

本题为本站自有原创练习题,非任何官方考试内容;解析对照 Anthropic 公开文档撰写, 如与最新文档不符请以官方为准。想在限时环境下检验水平,可参加 模拟考(题目与本页题库不重叠)。