{"id":899,"date":"2026-06-23T16:04:21","date_gmt":"2026-06-23T16:04:21","guid":{"rendered":"https:\/\/www.aicookbook.eu\/recipe\/creating-llm-skills-for-assessment-and-personalised-feedback\/"},"modified":"2026-06-23T16:04:21","modified_gmt":"2026-06-23T16:04:21","slug":"creating-llm-skills-for-assessment-and-personalised-feedback","status":"publish","type":"recipe","link":"https:\/\/www.aicookbook.eu\/sv\/recipe\/creating-llm-skills-for-assessment-and-personalised-feedback\/","title":{"rendered":"Creating LLM Skills for Assessment and Personalised Feedback"},"content":{"rendered":"<p>Students prepare and perform a phone call in the target language to their housing company to report a leaking water pipe &#8211; practising real-world communication for everyday life needs.<\/p>\n<h2>Learning Objectives<\/h2>\n<ul>\n<li>Teachers will be able to&#8230;<\/li>\n<li>Understand the architecture of an effective LLM Skill: role definition, context, constraints, rubric encoding, and output format.<\/li>\n<li>Write a complete system prompt that configures an LLM as a personalised feedback assistant for a specific assessment task.<\/li>\n<li>Encode a marking rubric into a system prompt so the LLM produces criterion-referenced, level- appropriate feedback.<\/li>\n<li>Test, iterate, and refine an LLM Skill using structured prompt evaluation techniques.<\/li>\n<li>Share and deploy LLM Skills across a team or institution using ChatGPT Custom Instructions, Claude Projects, or a shared prompt library.<\/li>\n<li>Critically evaluate LLM-generated feedback for accuracy, bias, and pedagogical appropriateness before sharing with learners<\/li>\n<\/ul>\n<h2>Ingredients<\/h2>\n<table>\n<tr>\n<th>Item \/ Resource<\/th>\n<th>Quantity<\/th>\n<th>Purpose<\/th>\n<\/tr>\n<tr>\n<td>ChatGPT (Plus\/Team), Claude.ai (Pro), or Gemini Advanced<\/td>\n<td>1 account per educator<\/td>\n<td>LLM platform for building and testing Skills<\/td>\n<\/tr>\n<tr>\n<td>Existing assessment rubric or marking criteria<\/td>\n<td>1 per task<\/td>\n<td>Core content to encode into the LLM Skill<\/td>\n<\/tr>\n<tr>\n<td>Sample learner submissions (anonymised)<\/td>\n<td>3-5 examples<\/td>\n<td>Test material for evaluating Skill output quality<\/td>\n<\/tr>\n<tr>\n<td>LLM Skill Template (see Preparation section)<\/td>\n<td>1 copy<\/td>\n<td>Structured framework for writing the system prompt<\/td>\n<\/tr>\n<tr>\n<td>Shared document or prompt library (e.g. Google Doc, Notion)<\/td>\n<td>1 per team<\/td>\n<td>Store and share Skills across the department or institution<\/td>\n<\/tr>\n<tr>\n<td>Feedback quality checklist (see Assessment section)<\/td>\n<td>1 per educator<\/td>\n<td>Evaluate LLM-generated feedback before sharing with learners<\/td>\n<\/tr>\n<tr>\n<td>Optional: ChatGPT GPT Builder or Claude Projects<\/td>\n<td>1 account<\/td>\n<td>Package the Skill as a reusable, named AI assistant<\/td>\n<\/tr>\n<\/table>\n<h2>Utensils<\/h2>\n<p>Computer with internet access<\/p>\n<p>LLM account with system prompt \/ custom instruction access (ChatGPT Plus, Claude Pro, or Gemini Advanced)<\/p>\n<p>Text editor or shared document for drafting and storing prompts<\/p>\n<p>Anonymised learner work samples for testing<\/p>\n<p>Projector or screen sharing for group workshop delivery<\/p>\n<h2>Preparation<\/h2>\n<ul>\n<li>Before the session, gather your existing assessment rubric and 3-5 anonymised learner submissions. Familiarise yourself with the LLM Skill Template below &#8211; this is the structural framework you will use to write your system prompt.<\/li>\n<li>LLM Skill Template Structure:<\/li>\n<li>ROLE: You are [role description &#8211; e.g. &#x27;an experienced language teacher specialising in adult migrant education at B1 CEFR level&#x27;]. TASK: Your task is to [specific assessment task &#8211; e.g. &#x27;assess a learner&#x27;s written email and provide structured, personalised feedback&#x27;]. RUBRIC: Use the following criteria to assess the work: &#8211; [Criterion 1]: [descriptor for Excellent \/ Good \/ Needs Work] &#8211; [Criterion 2]: [descriptor for Excellent \/ Good \/ Needs Work] &#8211; [Criterion 3]: [descriptor for Excellent \/ Good \/ Needs Work] LEARNER PROFILE: The learner is [profile &#8211; e.g. &#x27;an adult migrant at B1 level, first language Arabic, studying Swedish for work purposes&#x27;]. TONE &amp; LANGUAGE: Feedback must be [tone &#8211; e.g. &#x27;encouraging, specific, and written in plain English at B1 reading level. Avoid jargon.&#x27;]. OUTPUT FORMAT: Structure your response as follows: 1. Overall impression (2-3 sentences) 2. Criterion-by-criterion feedback (one paragraph per criterion) 3. Two specific strengths 4. Two specific, actionable areas for improvement 5. One suggested next learning step CONSTRAINTS: Do not assign a numerical grade. Do not rewrite the learner&#x27;s work. Do not use phrases like &#x27;Great job!&#x27; without specific justification.<\/li>\n<li>Select your assessment task &#8211; choose a written, oral, or project-based task you regularly assess. Gather your existing rubric or create one with 3-4 criteria.<\/li>\n<li>Anonymise 3-5 learner submissions at different quality levels &#8211; these will be your test cases for evaluating the Skill.<\/li>\n<li>Draft your LLM Skill using the template above &#8211; fill in each section with your specific role, task, rubric, learner profile, tone, output format, and constraints.<\/li>\n<li>Enter your system prompt into your chosen LLM platform (ChatGPT Custom Instructions, Claude Project Instructions, or Gemini system prompt field).<\/li>\n<li>Run your 3-5 test submissions through the Skill and evaluate the output using the Feedback Quality Checklist in the Assessment section.<\/li>\n<li>Iterate &#8211; refine the prompt based on what the LLM gets wrong. Common issues: too vague a rubric, missing learner profile context, or no output format constraints.<\/li>\n<\/ul>\n<h2>Implementation Steps<\/h2>\n<ul>\n<li>This recipe is designed for a CPD workshop or self-directed professional learning session. The following phases guide educators through building, testing, and deploying their first LLM Skill for assessment and feedback.<\/li>\n<li>Phase 1 &#8211; Deconstruct (15 min)<\/li>\n<li>Begin by examining 2-3 examples of LLM-generated feedback &#8211; one good, one mediocre, one poor. Discuss as a group: what makes feedback pedagogically useful? What does the LLM get wrong? Identify the key variables: specificity, criterion-alignment, tone, actionability, and learner-appropriateness.<\/li>\n<li>Phase 2 &#8211; Design (25 min)<\/li>\n<li>Using the LLM Skill Template, each educator drafts a system prompt for one of their own assessment tasks. Focus on encoding the rubric precisely &#8211; vague criteria produce vague feedback. Pair up and peer-review each other&#x27;s prompts before testing.<\/li>\n<li>Phase 3 &#8211; Test &amp; Iterate (30 min)<\/li>\n<li>Enter your Skill into the LLM platform and run your anonymised test submissions through it. Use the Feedback Quality Checklist to evaluate each output. Identify the weakest element of your prompt and revise it. Run the test again. Aim for at least two iteration cycles.<\/li>\n<li>Phase 4 &#8211; Deploy &amp; Share (20 min)<\/li>\n<li>Save your finalised Skill to a shared team prompt library. If using ChatGPT, consider packaging it as a named GPT using GPT Builder. If using Claude, save it as a Project. Walk the group through how to activate and use the Skill in a live lesson or marking session.<\/li>\n<li>Phase 5 &#8211; Reflect (10 min)<\/li>\n<li>Whole-group discussion: Where does this Skill add most value? Where must the educator always retain final judgement? How will you communicate to learners that feedback is AI-assisted? What are your institutional policy obligations?<\/li>\n<\/ul>\n<h2>Differentiation &amp; Inclusion<\/h2>\n<ul>\n<li>Explorer-level educators: Provide a pre-filled Skill template with a simple rubric already encoded &#8211; ask them to adapt the learner profile and tone only.<\/li>\n<li>Integrator-level educators: Challenge them to encode a multi-level rubric (A2\/B1\/B2) into a single Skill that adapts its feedback based on a learner profile parameter.<\/li>\n<li>Expert\/Pioneer educators: Explore chaining Skills &#8211; e.g. a first Skill that identifies errors, a second that generates a targeted grammar exercise based on those errors.<\/li>\n<li>Multilingual contexts: Build Skills that generate feedback in the learner&#x27;s first language alongside the target language, using a language parameter in the prompt.<\/li>\n<li>Large classes: Design Skills that generate feedback for batches of submissions using a structured input format, reducing marking time significantly.<\/li>\n<li>Institutional deployment: Work with IT\/admin to store approved Skills in a shared system prompt library accessible to all staff, ensuring consistency across the institution.<\/li>\n<\/ul>\n<h2>Assessment &amp; Feedback<\/h2>\n<table>\n<tr>\n<th>Criterion<\/th>\n<th>Excellent<\/th>\n<th>Good<\/th>\n<th>Needs Work<\/th>\n<\/tr>\n<tr>\n<td>Criterion Alignment<\/td>\n<td>Every piece of feedback maps directly to a named rubric criterion with specific evidence from the learner&#x27;s work.<\/td>\n<td>Most feedback is criterion-aligned; one or two comments are generic.<\/td>\n<td>Feedback is largely generic; rubric criteria are not clearly referenced.<\/td>\n<\/tr>\n<tr>\n<td>Specificity &amp; Evidence<\/td>\n<td>Feedback cites specific words, sentences, or examples from the submission to justify every judgement.<\/td>\n<td>Most feedback is specific; some comments lack direct evidence from the work.<\/td>\n<td>Feedback is vague; no specific examples from the submission are cited.<\/td>\n<\/tr>\n<tr>\n<td>Tone &amp; Learner-Appropriateness<\/td>\n<td>Tone is consistently encouraging and professional; language is pitched precisely at the learner&#x27;s level.<\/td>\n<td>Tone is mostly appropriate; occasional mismatch in register or complexity.<\/td>\n<td>Tone is inappropriate (too harsh, too effusive, or too complex for the learner).<\/td>\n<\/tr>\n<tr>\n<td>Actionability<\/td>\n<td>Every area for improvement includes a concrete, achievable next step the learner can act on immediately.<\/td>\n<td>Most improvements are actionable; one or two are too vague to act on.<\/td>\n<td>Areas for improvement are identified but no actionable guidance is provided.<\/td>\n<\/tr>\n<tr>\n<td>Accuracy &amp; Reliability<\/td>\n<td>All factual claims and language assessments in the feedback are correct; no hallucinations detected.<\/td>\n<td>Mostly accurate; one minor error or overstatement that requires correction.<\/td>\n<td>Contains factual errors, incorrect language assessments, or hallucinated content.<\/td>\n<\/tr>\n<\/table>\n<h2>Extensions \/ Follow-Ups<\/h2>\n<ul>\n<li>Chained Skills: Build a two-step workflow &#8211; Skill 1 identifies errors and generates a diagnosis; Skill 2 uses that diagnosis to generate a targeted practice exercise for the learner.<\/li>\n<li>Oral feedback: Combine your LLM Skill with a text-to-speech tool (e.g. ElevenLabs, Microsoft Azure TTS) to deliver personalised spoken feedback &#8211; particularly valuable for low-literacy learners.<\/li>\n<li>Learner self-assessment: Share a simplified version of the Skill directly with learners so they can self-assess their drafts before submission &#8211; building metacognitive awareness.<\/li>\n<li>Departmental prompt library: Facilitate a team session to build a shared library of approved LLM Skills for your institution&#x27;s most common assessment tasks &#8211; reducing duplication and ensuring consistency.<\/li>\n<li>Research &amp; reflection: Document your Skill iterations and learner outcomes over a term. Use this as the basis for a practitioner research project or CPD portfolio entry on AI-assisted assessment.<\/li>\n<\/ul>\n<h2>Cross-Subject Links<\/h2>\n<ul>\n<li>Civics \/ samh\u00e4llskunskap &#8211; tenant rights in Sweden, how bostadsf\u00f6retag work<\/li>\n<li>Digital literacy &#8211; how to use online felanm\u00e4lan portals (e.g. Mitt Boende, Riksbyggen app)<\/li>\n<\/ul>\n<h2>References &amp; Resources<\/h2>\n<p>OpenAI ChatGPT Custom Instructions: https:\/\/help.openai.com\/en\/articles\/8096356-custom-instructions-for-chatgpt<\/p>\n<p>Anthropic Claude Projects: https:\/\/www.anthropic.com\/claude<\/p>\n<p>Google Gemini System Instructions: https:\/\/ai.google.dev\/gemini-api\/docs\/system-instructions<\/p>\n<p>DigCompEdu &#8211; European Framework for the Digital Competence of Educators: https:\/\/joint-research-centre.ec.europa.eu\/digcompedu_en<\/p>\n<p>UNESCO Guidance for Generative AI in Education and Research (2023): https:\/\/www.unesco.org\/en\/digital-education\/artificial-intelligence<\/p>\n<p>Mollick, E. &amp; Mollick, L. (2023). Assigning AI: Seven Approaches for Students, with Prompts. SSRN: https:\/\/ssrn.com\/abstract=4475995<\/p>\n<p>European Commission &#8211; Ethics Guidelines for Trustworthy AI: https:\/\/digital-strategy.ec.europa.eu\/en\/library\/ethics-guidelines-trustworthy-ai<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Students prepare and perform a phone call in the target language to their housing company to report a leaking water pipe &#8211; practising real-world communication for everyday life needs.<\/p>","protected":false},"featured_media":0,"template":"","meta":{"_acf_changed":false},"class_list":["post-899","recipe","type-recipe","status-publish","hentry"],"acf":[],"_links":{"self":[{"href":"https:\/\/www.aicookbook.eu\/sv\/wp-json\/wp\/v2\/recipe\/899","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.aicookbook.eu\/sv\/wp-json\/wp\/v2\/recipe"}],"about":[{"href":"https:\/\/www.aicookbook.eu\/sv\/wp-json\/wp\/v2\/types\/recipe"}],"wp:attachment":[{"href":"https:\/\/www.aicookbook.eu\/sv\/wp-json\/wp\/v2\/media?parent=899"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}