Testing · worksheet · CSVtoCSA editorial

AI human-review evaluation

Version 2.0 · Updated 2026-09-21 · Testing

Test whether the complete review workflow delivers the risk reduction being claimed.

Adapt this template to your intended use, applicable requirements, and approved procedures.

When to use it

Use when you need to test whether the complete review workflow delivers the risk reduction being claimed.

Information needed before starting

  • Defined AI use, authority limits and assessed configuration
  • Controlled sources and independent reference results relevant to this worksheet
  • Applicable risk assessment, acceptance basis and responsible reviewers

Field instructions

  • Task and proposed reliance: Define what reviewers must decide and what AI output may influence.
  • Reviewer prerequisites: Record process knowledge, source access, AI limitations, time, authority and procedural training.
  • Cases and expected results: Identify representative cases, independently adjudicated reference results and scoring rubric.
  • Design and independence: Record users, conditions, assignments, order and measures against answer leakage.
  • Acceptance criteria: Set outcome and critical-failure rules before testing; justify them for the intended use.
  • Observed outcome: Score final correct/false/missed findings, source verification and escalation.
  • Limits and deviations: Retain dependency/coverage limits, failures and open corrective actions.
  • Decision and retest: State reliance restrictions and the fresh evidence needed after changes.

Completed fictional example

Synthetic HR-01 uses six reviewers, each completing two assisted and two unassisted sessions. Twelve documents appear once per condition with different reviewers: 24 sessions total. Each condition covers 40 reference issues including ten critical.

MeasureAssisted final reviewUnassisted final review
Correct findings3637
Missed issues43
Critical issues detected / reference9 / 1010 / 10
Critical misses10
False findings remaining21
DecisionDo not approve expanded reliance; DEV-02/03 remain openSmall study does not prove zero risk or causal superiority

Blank template

System / record ID: ____________________

Prepared by / date: ____________________

Required review / approval under local procedure: ____________________

FieldYour entry
Task and proposed reliance____________________
Reviewer prerequisites____________________
Cases and expected results____________________
Design and independence____________________
Acceptance criteria____________________
Observed outcome____________________
Limits and deviations____________________
Decision and retest____________________

Blank results or assessment rows

MeasureAssisted final reviewUnassisted final review
____________________________________________________________
____________________________________________________________
____________________________________________________________
Relevant regulations and guidance

These sources apply within their stated scope. The suggested worksheet fields, test designs and fictional decisions are practitioner recommendations, unless a source requirement is expressly identified. Completing a template does not establish compliance.

Computer Software Assurance for Production and Quality Management System Software ↗

Regulatory guidance / GMP guideline · United States · February 2026

Scope: Medical-device manufacturers assuring software used in production or their quality management system.

Relevance: For medical-device production or QMS software, supports selecting assurance activities from intended use, risk and available evidence. It is nonbinding guidance and does not determine other sectors’ obligations.

Status: Final / published · Checked 2026-09-21

Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.

Read source details and applicability →

EudraLex Volume 4, Annex 11 — Computerised Systems ↗

Regulatory guidance / GMP guideline · European Union · January 2011 revision

Scope: Computerised systems supporting activities in the applicable EU pharmaceutical GMP context.

Relevance: For computerised systems used in EU GMP activities, informs lifecycle, supplier and record controls. Confirm applicability to the actual activity; the example does not establish it.

Status: Final / published · Checked 2026-09-21

The Commission index confirms the January 2011 operative revision. The linked PDF could not be retrieved in this check; retained detailed section mappings need review against an authorized copy.

Read source details and applicability →

Artificial Intelligence Risk Management Framework (AI RMF 1.0) and NIST AI 600-1 ↗

Industry guidance / framework · United States · AI RMF 1.0 (2023); GenAI Profile (2024)

Scope: Cross-sector AI governance and risk management, including generative AI.

Relevance: A voluntary framework for describing AI risks, evaluation and ongoing management. Its suggested practices are not regulatory requirements.

Status: Final / published · Checked 2026-09-21

Publication identity, scope and status checked against the linked official source. This is not an independent audit of every retained guide statement.

Read source details and applicability →
Was this useful? Leave feedback

Was this resource useful?

Was this template clear and useful? Tell us which instruction or field could be improved.

Loading contribution options…

Questions and moderated discussion

Questions and discussion

Ask a question or share a perspective. Contributions are reviewed before publication.

Loading contribution options…

Suggest a correction

Sent privately to the editorial moderation queue. Your account email and correction are not published as a comment. Do not include confidential or personal information.

Resource: AI human-review evaluation

Loading sign-in status…

Correction details