Creating LLM Skills for Assessment and Personalised Feedback
Students prepare and perform a phone call in the target language to their housing company to report a leaking water pipe - practising real-world communication for everyday life needs.
DigCompEdu Competence Area(s)
Time
Preparation time minutes
Implementation time minutes
Educator Progression Level
Target groups
Minimum student digital skill level
B2
Minimum student language level
A2
🎓 Learning Objectives
- Teachers will be able to...
- Understand the architecture of an effective LLM Skill: role definition, context, constraints, rubric encoding, and output format.
- Write a complete system prompt that configures an LLM as a personalised feedback assistant for a specific assessment task.
- Encode a marking rubric into a system prompt so the LLM produces criterion-referenced, level- appropriate feedback.
- Test, iterate, and refine an LLM Skill using structured prompt evaluation techniques.
- Share and deploy LLM Skills across a team or institution using ChatGPT Custom Instructions, Claude Projects, or a shared prompt library.
- Critically evaluate LLM-generated feedback for accuracy, bias, and pedagogical appropriateness before sharing with learners
- Answer AI‑generated quizzes to check understanding of a topic.
- Reflect on question types (multiple‑choice, recall, short answer) and identify areas for improvement.
- Provide feedback on the accuracy and relevance of AI‑generated questions.
Ingredients
| Item / Resource | Quantity | Purpose |
|---|---|---|
| ChatGPT (Plus/Team), Claude.ai (Pro), or Gemini Advanced | 1 account per educator | LLM platform for building and testing Skills |
| Existing assessment rubric or marking criteria | 1 per task | Core content to encode into the LLM Skill |
| Sample learner submissions (anonymised) | 3-5 examples | Test material for evaluating Skill output quality |
| LLM Skill Template (see Preparation section) | 1 copy | Structured framework for writing the system prompt |
| Shared document or prompt library (e.g. Google Doc, Notion) | 1 per team | Store and share Skills across the department or institution |
| Feedback quality checklist (see Assessment section) | 1 per educator | Evaluate LLM-generated feedback before sharing with learners |
| Optional: ChatGPT GPT Builder or Claude Projects | 1 account | Package the Skill as a reusable, named AI assistant |
Utensils
Computer with internet access
LLM account with system prompt / custom instruction access (ChatGPT Plus, Claude Pro, or Gemini Advanced)
Text editor or shared document for drafting and storing prompts
Anonymised learner work samples for testing
Projector or screen sharing for group workshop delivery
🥣 Preparation – Before Implementation
- Before the session, gather your existing assessment rubric and 3-5 anonymised learner submissions. Familiarise yourself with the LLM Skill Template below - this is the structural framework you will use to write your system prompt.
- LLM Skill Template Structure:
- ROLE: You are [role description - e.g. 'an experienced language teacher specialising in adult migrant education at B1 CEFR level']. TASK: Your task is to [specific assessment task - e.g. 'assess a learner's written email and provide structured, personalised feedback']. RUBRIC: Use the following criteria to assess the work: - [Criterion 1]: [descriptor for Excellent / Good / Needs Work] - [Criterion 2]: [descriptor for Excellent / Good / Needs Work] - [Criterion 3]: [descriptor for Excellent / Good / Needs Work] LEARNER PROFILE: The learner is [profile - e.g. 'an adult migrant at B1 level, first language Arabic, studying Swedish for work purposes']. TONE & LANGUAGE: Feedback must be [tone - e.g. 'encouraging, specific, and written in plain English at B1 reading level. Avoid jargon.']. OUTPUT FORMAT: Structure your response as follows: 1. Overall impression (2-3 sentences) 2. Criterion-by-criterion feedback (one paragraph per criterion) 3. Two specific strengths 4. Two specific, actionable areas for improvement 5. One suggested next learning step CONSTRAINTS: Do not assign a numerical grade. Do not rewrite the learner's work. Do not use phrases like 'Great job!' without specific justification.
- Select your assessment task - choose a written, oral, or project-based task you regularly assess. Gather your existing rubric or create one with 3-4 criteria.
- Anonymise 3-5 learner submissions at different quality levels - these will be your test cases for evaluating the Skill.
- Draft your LLM Skill using the template above - fill in each section with your specific role, task, rubric, learner profile, tone, output format, and constraints.
- Enter your system prompt into your chosen LLM platform (ChatGPT Custom Instructions, Claude Project Instructions, or Gemini system prompt field).
- Run your 3-5 test submissions through the Skill and evaluate the output using the Feedback Quality Checklist in the Assessment section.
- Iterate - refine the prompt based on what the LLM gets wrong. Common issues: too vague a rubric, missing learner profile context, or no output format constraints.
🔥 Implementation Steps
- This recipe is designed for a CPD workshop or self-directed professional learning session. The following phases guide educators through building, testing, and deploying their first LLM Skill for assessment and feedback.
- Phase 1 - Deconstruct (15 min)
- Begin by examining 2-3 examples of LLM-generated feedback - one good, one mediocre, one poor. Discuss as a group: what makes feedback pedagogically useful? What does the LLM get wrong? Identify the key variables: specificity, criterion-alignment, tone, actionability, and learner-appropriateness.
- Phase 2 - Design (25 min)
- Using the LLM Skill Template, each educator drafts a system prompt for one of their own assessment tasks. Focus on encoding the rubric precisely - vague criteria produce vague feedback. Pair up and peer-review each other's prompts before testing.
- Phase 3 - Test & Iterate (30 min)
- Enter your Skill into the LLM platform and run your anonymised test submissions through it. Use the Feedback Quality Checklist to evaluate each output. Identify the weakest element of your prompt and revise it. Run the test again. Aim for at least two iteration cycles.
- Phase 4 - Deploy & Share (20 min)
- Save your finalised Skill to a shared team prompt library. If using ChatGPT, consider packaging it as a named GPT using GPT Builder. If using Claude, save it as a Project. Walk the group through how to activate and use the Skill in a live lesson or marking session.
- Phase 5 - Reflect (10 min)
- Whole-group discussion: Where does this Skill add most value? Where must the educator always retain final judgement? How will you communicate to learners that feedback is AI-assisted? What are your institutional policy obligations?
🧂 Differentiation & Inclusion
🍴 Assessment & Feedback
⚠️ Ethics, Privacy & Safeguarding
🍱 Extensions / Follow‑Ups
- Chained Skills: Build a two-step workflow - Skill 1 identifies errors and generates a diagnosis; Skill 2 uses that diagnosis to generate a targeted practice exercise for the learner.
- Oral feedback: Combine your LLM Skill with a text-to-speech tool (e.g. ElevenLabs, Microsoft Azure TTS) to deliver personalised spoken feedback - particularly valuable for low-literacy learners.
- Learner self-assessment: Share a simplified version of the Skill directly with learners so they can self-assess their drafts before submission - building metacognitive awareness.
- Departmental prompt library: Facilitate a team session to build a shared library of approved LLM Skills for your institution's most common assessment tasks - reducing duplication and ensuring consistency.
- Research & reflection: Document your Skill iterations and learner outcomes over a term. Use this as the basis for a practitioner research project or CPD portfolio entry on AI-assisted assessment.
🔗 Cross‑Subject Link
- Civics / samhällskunskap - tenant rights in Sweden, how bostadsföretag work
- Digital literacy - how to use online felanmälan portals (e.g. Mitt Boende, Riksbyggen app)