Decoding Copilot impact dashboard now shows feature engagement from official information

This article is a technical explanation and implementation example generated using AI. The code and procedures presented are based on primary sources, but the author has not verified their operation on actual hardware. Behavior may vary depending on the environment and version.

The official GitHub Changelog announced the addition of a feature engagement display per feature in the Copilot impact dashboard. This article organizes the changes, API specifications, permission requirements, and aggregation definitions shown in the primary sources.

By reading this article, Enterprise and Organization administrators can safely and practically understand how to track the adoption effects of Copilot and the structure of the data obtainable from reporting APIs and CLIs.

1. Background and Purpose of the Update

As the adoption of GitHub Copilot progresses, administrators need to understand which features are actually established in developers' daily workflows. According to primary sources, while conventional dashboards and reports could capture overall usage trends, it was difficult to track engagement for individual features in detail, or the breakdown of AI adoption phases was limited to daily active user counts.

With this update, Enterprise administrators can quickly determine which experiences are widely accepted and where additional support or training is needed.

2. Main New Features of the Copilot Impact Dashboard

The Copilot impact dashboard now displays the number of active users who have used key Copilot features for at least two days during a 28-day aggregation period.

This visualizes not just "users who touched it once," but the segment of users who utilize it continuously. In addition to the figures on the dashboard, similar data can now be retrieved via the 28-day aggregation reporting APIs for Enterprises and Organizations.

3. Extension of Data Structures in Reporting APIs

Primary sources provide a detailed explanation of the specific objects and fields added to the APIs and summary reports.

• copilot_feature_engagement: Adds dashboard total active users and engagement counts by feature to 28-day aggregation reports at the Enterprise and Organization levels.

• totals_by_feature: Categorizes features into the following categories to show the breakdown of engagement.

  • code completion

  • agent edit

  • passive Copilot code review

  • active Copilot code review

  • Copilot cloud agent

  • Copilot CLI

  • Copilot app

• users_in_phase_28d: Reports the population for each AI adoption phase across the rolling 28-day period as of each report date. The existing total_engaged_users field will continue to report only the users within the phase who were active on that specific day.

4. Notes on Feature and Code Review Definitions

The primary sources outline several important notes regarding the definitions and limitations of the features being aggregated.

• Report Level: Feature engagement is available in the 28-day aggregated reports for Enterprises and Organizations, and is not added to user-level reports.

• Duplicate Counting: If a single user utilizes multiple features, they will be counted under each respective feature, meaning the sum of multiple features may contain duplicates.

• Distinction in Code Reviews:

  • Active Copilot code review: Occurs when a user manually requests a Copilot review or applies a Copilot review suggestion.

  • Passive Copilot code review: Occurs when Copilot is automatically assigned to review a pull request without active user involvement.

• Anonymity and Missing Values: users_in_phase_28d and total_engaged_users are aggregated values and do not identify individual users. If not measured, users_in_phase_28d is omitted, and if measurements were taken but no users were present, 0 is returned. Furthermore, if the calculation is unavailable, the copilot_feature_engagement object may be absent or null.

5. Required Permissions and Policies

To access this data, you must meet the appropriate permissions and organizational policy settings.

• Target Roles: Enterprise Owners and billing managers, Organization Owners, and users with custom Organization/Enterprise roles granted the View Copilot Metrics permission.

• Prerequisite Policy: The Copilot usage metrics policy must be enabled.

6. Conclusion

This article outlines the addition of feature engagement to the Copilot impact dashboard and API based on the official Changelog.

Points to Verify and Constraints Before Execution

• This article is an explanation of primary information, and API calls or UI rendering results in an actual environment are in a [pre-verification] state.

• Retrieving data requires appropriate roles (Enterprise Owner, billing manager, Organization Owner, or View Copilot Metrics permission).

• Ensure in advance that the Copilot usage metrics policy is enabled.

• Note that user-level reports do not include feature engagement.

References

• GitHub Changelog: Copilot impact dashboard now shows feature engagement