Pattern 2C: Report system performance information

Problem

The user needs to form accurate expectations about how well the system can do what it can do.

Solution

Provide grounded information about how well the system can do what it can do.

Use when

The user needs system performance information to make or justify decisions about adopting the system.
The user needs system performance information to decide how much to rely on the system.
There are established benchmarks for the system type.

How

Collaborate with an AI/ML practitioner to collect information about:

What information about system performance is relevant to the user.
How to retrieve system performance indicators such as false positive/negative rates, accuracy, etc. Performance indicators vary across types of applications.

Performance information may cover overall system performance as well as system performance under certain conditions or for certain user groups where the system is suspected to perform less well.

Enable the user to understand practical implications for system performance by clearly relating system performance to system capabilities, and reporting information in units the user understands.

Performance information may be presented in documentation or as an interactive tool for inspecting model performance.

User benefits

Enables the user to gain insights into overall system performance.
Enables the user to assess system performance independent of system use.
Enables the user to compare across similar systems.

Common pitfalls

System performance information does not translate into practical implications for system use.
System performance information is not discoverable.
System performance information is limited to overall performance and does not cover specific conditions.
Using this pattern as a substitute for G2-D: Provide low performance alerts.