Promptfoo Eval Suite

复制 Prompt

/goal
GOAL:
Complete Promptfoo Eval Suite for an AI evaluation project: Add a runnable Promptfoo eval suite for an AI application before changing production prompts or model behavior.

CONTEXT:
- Before editing, read the nearest AGENTS.md/CLAUDE.md, current issue or PLAN.md, and any failing logs already in the repo.
- Inspect eval datasets, rubrics, model outputs, judge code, and regression reports.
- Establish a baseline by running or locating evidence for: `target adapter, seed cases, assertions, files, env requirements, local eval command, passing and failing cases`.

CONSTRAINTS:
- Keep the scope limited to this goal; do not expand into unrelated cleanup.
- Do not weaken tests, delete assertions, or mask errors to make verification pass.
- Respect the repository's AGENTS.md/CLAUDE.md instructions and existing patterns.
- Do not tune prompts against hidden labels or delete failing eval cases to improve the score.
- Keep before/after eval evidence and representative failures.

DONE WHEN:
- The implementation or documentation directly satisfies: Add a runnable Promptfoo eval suite for an AI application before changing production prompts or model behavior.
- The verification command or evidence path succeeds: `target adapter, seed cases, assertions, files, env requirements, local eval command, passing and failing cases`.
- The final diff is scoped to the relevant files and has no unrelated formatting churn.

VERIFY:
- Run `target adapter, seed cases, assertions, files, env requirements, local eval command, passing and failing cases` or the closest repo-local equivalent if the exact command is not available.
- Capture before/after evidence for the behavior, metric, report, or artifact involved.
- If verification cannot run locally, stop and report the missing dependency instead of guessing success.

OUTPUT:
- Summarize changed files, key decisions, verification output, and remaining risks.
- Include any follow-up that is required for production rollout or human review.

STOP RULES:
- Pause if secrets, production access, stakeholder decisions, or destructive data operations are required.
- Pause after three failed fix attempts on the same symptom and challenge the root-cause hypothesis.
- Do not mark the goal complete until the current repository state has been audited against DONE WHEN.