Creating LLM Skills for Assessment and Personalised Feedback

Students prepare and perform a phone call in the target language to their housing company to report a leaking water pipe - practising real-world communication for everyday life needs.

Τομέας(-είς) ικανοτήτων DigCompEdu

Digital resources, Assessment, Empowering learners, Facilitating learners’ digital competence

Χρόνος

Preparation time 30 minutes

Implementation time 90 minutes

Επίπεδο Εξέλιξης Εκπαιδευτικού

Ηγέτης

Ομάδες-στόχοι

Advanced educators · Any subject · Any  language level · Any group size

Ελάχιστο επίπεδο ψηφιακών δεξιοτήτων των μαθητών

Intermediate

Ελάχιστο επίπεδο γλωσσομάθειας του μαθητή

A2

🎓 Μαθησιακοί Στόχοι

Teachers will be able to…

  • Understand the architecture of an effective LLM Skill: role definition, context, constraints, rubric encoding, and output format.
  • Write a complete system prompt that configures an LLM as a personalised feedback assistant for a specific assessment task.
  • Encode a marking rubric into a system prompt so the LLM produces criterion-referenced, level- appropriate feedback.
  • Test, iterate, and refine an LLM Skill using structured prompt evaluation techniques.
  • Share and deploy LLM Skills across a team or institution using ChatGPT Custom Instructions, Claude Projects, or a shared prompt library.
  • Critically evaluate LLM-generated feedback for accuracy, bias, and pedagogical appropriateness before sharing with learner

Ingredients

Item / Resource Quantity Purpose
ChatGPT (Plus/Team), Claude.ai (Pro), or Gemini Advanced 1 account per educator LLM platform for building and testing Skills
Existing assessment rubric or marking criteria 1 per task Core content to encode into the LLM Skill
Sample learner submissions (anonymised) 3-5 examples Test material for evaluating Skill output quality
LLM Skill Template (see Preparation section) 1 copy Structured framework for writing the system prompt
Shared document or prompt library (e.g. Google Doc, Notion) 1 per team Store and share Skills across the department or institution
Feedback quality checklist (see Assessment section) 1 per educator Evaluate LLM-generated feedback before sharing with learners
Optional: ChatGPT GPT Builder or Claude Projects 1 account Package the Skill as a reusable, named AI assistant

Utensils

  • Computer with internet access
  • LLM account with system prompt / custom instruction access (ChatGPT Plus, Claude Pro, or Gemini Advanced)
  • Text editor or shared document for drafting and storing prompts
  • Anonymised learner work samples for testing
  • Projector or screen sharing for group workshop delivery

🥣 Προετοιμασία – Πριν από την υλοποίηση

  • Before the session, gather your existing assessment rubric and 3-5 anonymised learner submissions. Familiarise yourself with the LLM Skill Template below – this is the structural framework you will use to write your system prompt.
  • LLM Skill Template Structure:
  • ROLE: You are [role description - e.g. 'an experienced language teacher specialising in adult migrant education at B1 CEFR level']. 

TASK: Your task is to [specific assessment task - e.g. 'assess a learner's written email and provide structured, personalised feedback']. 

RUBRIC: Use the following criteria to assess the work: 
- [Criterion 1]: [descriptor for Excellent / Good / Needs Work] 
- [Criterion 2]: [descriptor for Excellent / Good / Needs Work] 
- [Criterion 3]: [descriptor for Excellent / Good / Needs Work] 

LEARNER PROFILE: The learner is [profile - e.g. 'an adult migrant at B1 level, first language Arabic, studying Swedish for work purposes']. 

TONE & LANGUAGE: Feedback must be [tone - e.g. 'encouraging, specific, and written in plain English at B1 reading level. Avoid jargon.']. 

OUTPUT FORMAT: Structure your response as follows: 1. Overall impression (2-3 sentences) 2. Criterion-by-criterion feedback (one paragraph per criterion) 
3. Two specific strengths 4. Two specific, actionable areas for improvement 5. One suggested next learning step 

CONSTRAINTS: Do not assign a numerical grade. Do not rewrite the learner's work. Do not use phrases like 'Great job!' without specific justification.
  • Select your assessment task – choose a written, oral, or project-based task you regularly assess. Gather your existing rubric or create one with 3-4 criteria.
  • Anonymise 3-5 learner submissions at different quality levels – these will be your test cases for evaluating the Skill.
  • Draft your LLM Skill using the template above – fill in each section with your specific role, task, rubric, learner profile, tone, output format, and constraints.
  • Enter your system prompt into your chosen LLM platform (ChatGPT Custom Instructions, Claude Project Instructions, or Gemini system prompt field).
  • Run your 3-5 test submissions through the Skill and evaluate the output using the Feedback Quality Checklist in the Assessment section.
  • Iterate – refine the prompt based on what the LLM gets wrong. Common issues: too vague a rubric, missing learner profile context, or no output format constraints.

🔥 Βήματα υλοποίησης

This recipe is designed for a CPD workshop or self-directed professional learning session. The following phases guide educators through building, testing, and deploying their first LLM Skill for assessment and feedback.

  • Phase 1 – Deconstruct (15 min)
    • Begin by examining 2-3 examples of LLM-generated feedback – one good, one mediocre, one poor. Discuss as a group: what makes feedback pedagogically useful? What does the LLM get wrong? Identify the key variables: specificity, criterion-alignment, tone, actionability, and learner-appropriateness.
  • Phase 2 – Design (25 min)
    • Using the LLM Skill Template, each educator drafts a system prompt for one of their own assessment tasks. Focus on encoding the rubric precisely – vague criteria produce vague feedback. Pair up and peer-review each other’s prompts before testing.
  • Phase 3 – Test & Iterate (30 min)
    • Enter your Skill into the LLM platform and run your anonymised test submissions through it. Use the Feedback Quality Checklist to evaluate each output. Identify the weakest element of your prompt and revise it. Run the test again. Aim for at least two iteration cycles.
  • Phase 4 – Deploy & Share (20 min)
    • Save your finalised Skill to a shared team prompt library. If using ChatGPT, consider packaging it as a named GPT using GPT Builder. If using Claude, save it as a Project. Walk the group through how to activate and use the Skill in a live lesson or marking session.
  • Phase 5 – Reflect (10 min)
    • Whole-group discussion: Where does this Skill add most value? Where must the educator always retain final judgement? How will you communicate to learners that feedback is AI-assisted? What are your institutional policy obligations?

🧂 Διαφοροποίηση και Συμπερίληψη

  • Explorer-level educators: Provide a pre-filled Skill template with a simple rubric already encoded — ask them to adapt the learner profile and tone only.
  • Integrator-level educators: Challenge them to encode a multi-level rubric (A2/B1/B2) into a single Skill that adapts its feedback based on a learner profile parameter.
  • Expert/Pioneer educators: Explore chaining Skills — e.g. a first Skill that identifies errors, a second that generates a targeted grammar exercise based on those errors.
  • Multilingual contexts: Build Skills that generate feedback in the learner’s first language alongside the target language, using a language parameter in the prompt.
  • Large classes: Design Skills that generate feedback for batches of submissions using a structured input format, reducing marking time significantly.
  • Institutional deployment: Work with IT/admin to store approved Skills in a shared system prompt library accessible to all staff, ensuring consistency across the institution.

🍴 Assessment & Feedback

Criterion Excellent Good Needs Work
Criterion Alignment Every piece of feedback maps directly to a named rubric criterion with specific evidence from the learner’s work. Most feedback is criterion-aligned; one or two comments are generic. Feedback is largely generic; rubric criteria are not clearly referenced.
Specificity & Evidence Feedback cites specific words, sentences, or examples from the submission to justify every judgement. Most feedback is specific; some comments lack direct evidence from the work. Feedback is vague; no specific examples from the submission are cited.
Tone & Learner-Appropriateness Tone is consistently encouraging and professional; language is pitched precisely at the learner’s level. Tone is mostly appropriate; occasional mismatch in register or complexity. Tone is inappropriate (too harsh, too effusive, or too complex for the learner).
Actionability Every area for improvement includes a concrete, achievable next step the learner can act on immediately. Most improvements are actionable; one or two are too vague to act on. Areas for improvement are identified but no actionable guidance is provided.
Accuracy & Reliability All factual claims and language assessments in the feedback are correct; no hallucinations detected. Mostly accurate; one minor error or overstatement that requires correction. Contains factual errors, incorrect language assessments, or hallucinated content.

⚠️ Ethics, Privacy & Safeguarding

 

  • Data Privacy & GDPR: Never submit identifiable learner work (names, ID numbers, personal details) to any LLM platform. Always anonymise submissions before testing or using a Skill. Check your institution’s data processing agreements — many LLM providers process data on servers outside the EU, which may require a Data Processing Agreement (DPA).
  • Transparency with Learners: Learners have a right to know when AI has been involved in generating their feedback. Develop a clear institutional disclosure statement and include it in your feedback communications.
  • Educator Accountability: LLM-generated feedback must always be reviewed and approved by the educator before sharing. The educator remains professionally and legally responsible for all feedback given to learners. AI is a tool, not a substitute for professional judgement.
  • Bias & Fairness: LLMs can reproduce cultural, linguistic, and demographic biases present in their training data. Actively test your Skill with diverse learner profiles and check for differential quality in feedback across language backgrounds, writing styles, and topics.
  • Institutional Policy: Verify that your school, programme, or institution has a policy on AI-assisted assessment before deploying LLM Skills with real learners. Where no policy exists, advocate for one.

🍱 Extensions / Follow‑Ups

  • Chained Skills: Build a two-step workflow – Skill 1 identifies errors and generates a diagnosis; Skill 2 uses that diagnosis to generate a targeted practice exercise for the learner.
  • Oral feedback: Combine your LLM Skill with a text-to-speech tool (e.g. ElevenLabs, Microsoft Azure TTS) to deliver personalised spoken feedback – particularly valuable for low-literacy learners.
  • Learner self-assessment: Share a simplified version of the Skill directly with learners so they can self-assess their drafts before submission – building metacognitive awareness.
  • Departmental prompt library: Facilitate a team session to build a shared library of approved LLM Skills for your institution’s most common assessment tasks – reducing duplication and ensuring consistency.
  • Research & reflection: Document your Skill iterations and learner outcomes over a term. Use this as the basis for a practitioner research project or CPD portfolio entry on AI-assisted assessment.

🔗 Cross‑Subject Link

  • Civics / samhällskunskap – tenant rights in Sweden, how bostadsföretag work
  • Digital literacy – how to use online felanmälan portals (e.g. Mitt Boende, Riksbyggen app)

References & Resources

OpenAI ChatGPT Custom Instructions: https://help.openai.com/en/articles/8096356-custom-instructions-for-chatgpt

Anthropic Claude Projects: https://www.anthropic.com/claude

Google Gemini System Instructions: https://ai.google.dev/gemini-api/docs/system-instructions

DigCompEdu — European Framework for the Digital Competence of Educators: https://joint-research-centre.ec.europa.eu/digcompedu_en

UNESCO Guidance for Generative AI in Education and Research (2023): https://www.unesco.org/en/digital-education/artificial-intelligence

Mollick, E. & Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. SSRN: https://ssrn.com/abstract=4475995

European Commission — Ethics Guidelines for Trustworthy AI: https://digital-strategy.ec.europa.eu/en/library/ethics-guidelines-trustworthy-ai