GitHub Copilot Evals Enters Public Preview for Copilot Users
GitHub Copilot Evals allows users to create repeatable tests for AI behavior in Copilot, now available as an extension or plugin for VS Code, Copilot CLI, and the Copilot app across all paid Copilot plans.
What changed?
GitHub Copilot Evals is now in public preview across Copilot Individual, Business, and Enterprise plans. This tool lets users create, run, and inspect repeatable tests (“evals”) for Copilot’s AI behavior. It integrates as an extension or plugin with Visual Studio Code, Copilot CLI, and the Copilot app. Evals uses Vally as its local evaluation engine.

Why does it matter to an everyday developer?
With Copilot Evals, developers can systematically test how Copilot responds to realistic tasks as their prompts, skills, agents, or tools change. This helps identify regressions or improvements after customizations, and ensures Copilot remains effective for specific workflows. The tool offers practical benefits when tuning Copilot for tasks, integrating it with custom agents, or validating AI-generated results during development.
What can the developer do now?
- Install the GitHub Copilot Evals extension or plugin for Visual Studio Code, Copilot CLI, or the Copilot app as appropriate for your workflow.
- Create evals that define tasks and specify what “good” results look like for Copilot in your project.
- Run evals to check which behaviors pass or fail after changes to your Copilot configuration, skills, or agents.
- Compare results from before and after customizations to assess improvement or detect regressions.
