Using Generative AI to Create Effective Assessment Rubrics

What AI can contribute

An analytic rubric connects criteria with descriptions of observable performance at different levels. Rubrics can clarify expectations and support consistent grading and formative feedback when they are aligned, understood, and tested; their appearance alone does not establish validity or reliability.^{1 2}

AI can propose wording and formats from supplied outcomes, assignment instructions, and standards. It cannot establish that the criteria are fair or that two graders will interpret a descriptor in the same way. Use it to draft alternatives, then test the rubric with people.

Draft, calibrate, and publish

1. Define the evidence. Identify the capability each criterion measures and what the submitted work can actually show. Do not add language or presentation criteria unless they belong to the outcomes.
2. Draft the levels. Supply required criteria, number of levels, and any agreed weights. Ask for observable differences; replace labels such as “excellent insight” with evidence another grader can locate.
3. Check overlap and fairness. Remove double-counting and requirements absent from the brief. Inspect cultural and linguistic assumptions that could disadvantage students.³
4. Calibrate with colleagues. Independently apply the draft to several educator-created or appropriately authorised anchor responses. Compare the evidence and level selected, resolve disagreements, and revise descriptors. Synthetic anchors are useful for drafting but do not establish reliability across real student work.
5. Share before use. Publish the reviewed rubric and discuss examples with students. Keep the announced criteria stable during assessment; handle any necessary correction through institutional procedures.

Copyable rubric-drafting prompt

Draft an analytic rubric using only the supplied assignment, learning outcomes,
and required standards.

For each criterion:
- map it to one or more learning outcomes;
- describe observable evidence rather than personality, effort, or intent;
- write four performance levels with differences that another qualified grader
  could distinguish; and
- flag overlap, missing evidence, or requirements unsupported by the assignment.

Return:
1. an outcome-to-criterion mapping;
2. the draft rubric table;
3. questions the educator must resolve; and
4. a checklist for testing the rubric on synthetic anchor responses.

Do not add presentation, language, citation, or formatting criteria unless the
supplied outcomes or assignment require them. Do not assign weights when their
relative importance has not been provided.

[assignment]
[learning outcomes]
[required standards]

Example: distinguish adjacent levels

For a criterion about evaluating evidence, “good use of sources” and “excellent use of sources” leave the difference unclear. A more checkable distinction might be “uses relevant evidence but does not address a material conflicting finding” and “weighs relevant and conflicting evidence and explains how it affects the conclusion”. Adapt this to the taught outcome and test it on anchor responses; these are illustrative descriptors, not a validated universal scale.

Keep the educator responsible for applying the rubric and deciding the grade. For feedback drafting, use Providing feedback; for a pilot record covering quality and review time, use Evaluating an AI teaching workflow.

Worked calibration: resolve a disagreement

Use the synthetic submission and evidence pack with two colleagues. The draft evidence criterion says only “uses sources effectively”. Ask each colleague to select a level and locate the evidence before comparing judgments.

These are illustrative judgments, not measured reliability results. The team should return to the learning outcome: evaluating what evidence supports. On that outcome, citing a number is insufficient when the conclusion exceeds it.

Replace the vague descriptor with observable distinctions: 0, makes a material inference the evidence does not support; 1, interprets the central evidence accurately but leaves a material limitation unexplained; 2, interprets it accurately and explains how the limitation affects the recommendation. The original excerpt falls at 0 for this criterion because of its unsupported population and causal claims; a correct opening sentence does not resolve them.

Agree how to handle mixed evidence, then independently apply the revised descriptor to additional anchor responses, including a well-supported conclusion different from the educator’s preference. Record remaining disagreements and revise before publication. Do not change an announced rubric informally during marking. This small exercise improves the specificity of the draft; it does not establish reliability for a whole cohort.

The worked assessment designs show complete briefs and illustrative rubrics for essay, quantitative, and professional tasks.

References & Footnotes