AI Feedback Governance
Question: To be added.
Overview
This metric is part of the AI Alignment - Community Aligned Training (Feedback) metrics model, which measures whether a community can give feedback to a model builder and see its impact, as the means by which alignment with that community increases.
Whether the community has policy for when and how feedback to AI builders happens: a decided, written expectation that feedback is something this project does, through which path, triggered by what, and decided by whom.
Establish that feedback is governed by policy rather than left to whoever happens to speak up. Effective feedback governance does three things: it provides a consensus path for reaching a shared community position; it provides a non-consensus path (through the Code of Conduct or another confidential route) for feedback that cannot or should not wait for consensus, such as harm or safety; and it routes through the community's own leadership first, because the issue is often how AI is being used within the project rather than the model itself. Governance decides whether a matter is resolved internally or escalated to the builder.
| Indicator | Implementation | Measurable now? |
|---|---|---|
| Feedback policy exists | Check whether the project has written policy for when and how feedback to AI builders happens (who decides, through which path, triggered by what) | Partly |
| Consensus path defined | Check for a defined way to reach a shared community position to give feedback on behalf of the project | Partly |
| Non-consensus / CoC path defined | Check for a defined route (Code of Conduct or other confidential means) for feedback that cannot wait for consensus, such as harm or safety | Partly |
| Routes through leadership first | Check whether policy sends feedback through the community's decision-makers before it leaves the project | Partly |
| Distinguishes internal AI-use from model issues | Check whether governance separates 'how AI is used here' (resolved internally) from 'the model misbehaved' (escalated to the builder) | Partly |
Want to Know More?
Data Collection Strategies
Known methods (as of this writing) for collecting data related to this metric.
| Attribute | How we'd collect it | Detectable today? |
|---|---|---|
| Governance behind signal | Read project decision process / consensus records where published | Partial. Visible where projects document governance |
| Community governance changed | Track the community's own policy, bounty, or moderation changes | Detectable; it's the community's own record |
Filters
- Path: consensus, non-consensus / Code of Conduct
- Resolution route: resolved internally ('how AI is used here'), escalated to the builder ('the model misbehaved')
- Feedback type: ◆ quality of contribution, ◆ accuracy / factuality, safety, bias / discriminatory, privacy, ◆ tooling / agent behavior, environmental impact, accessibility, attribution / provenance, licensing / compliance, financial cost
- Lean: both care, leans community
- Tier: detectable-today core, standards-we're-asking-for
- Measurable now: partly
Visualizations
(model-level visualizations are listed in the metrics model)
References
(see the References section of the metrics model)
Contributors
- Emma Irwin
- Coraline Ada Ehmke
- Justin Wheeler
- Adrian Edwards
Add your name if you contributed
To edit this metric please submit a Change Request here: wg-ai-alignment/metrics/ai-alignment-community-feedback/ai-feedback-governance.md at main · chaoss/wg-ai-alignment
To reference this metric in software or publications, please use this stable URL: https://www.chaoss.community/kb/ai-feedback-governance/