Is My AI Lying? A Beginner’s Guide to Spotting Bias and Mistakes
A critical thinking session for learners where they analyse AI-generated responses to identify cultural biases, stereotypes, and factual inaccuracies, learning to verify information.
DigCompEdu Competence Area(s)
Professional engagement, Digital resources, Teaching & learning, Assessment, Empowering learners, Facilitating learners’ digital competence
Time
Preparation time 20 minutes
Implementation time 75 minutes
Educator Progression Level
Newcomer
Target groups
Any
Minimum student digital skill level
Basic
Minimum student language level
A1
🎓 Learning Objectives
Students will be able to:
- Identify a potential stereotype or cultural bias in an AI-generated text.
- Use a simple fact-checking strategy to verify a piece of AI-generated information.
- Explain why critically evaluating AI outputs is important.
Ingredients
| Item / Resource | Quantity | Purpose |
|---|---|---|
| AI tool(s) (e.g., ChatGPT) | 1 (teacher demo) | To generate examples for analysis |
| Content materials (Fact-checking guides) | 1 per pair | Support resource |
| Physical materials (Worksheet, highlighters) | 1 set per pair | For analysis tasks |
| Online platforms / LMS | Optional | For sharing links to reliable sites |
Utensils
- Display: Projector and screen.
- Devices: Learners’ smartphones or laptops for fact-checking step (optional, can be done as a group).
🥣 Preparation – Before Implementation
- Example Generation: Pre-generate examples of biased/inaccurate AI responses. Good topics: descriptions of national cuisines, traditional clothing, historical figures from non-Western countries, or current events.
- Fact-Checking Checklist: Prepare a simple “Fact-Checking Checklist” handout (e.g., 1. Check the date. 2. Cross-reference with a trusted website (e.g., BBC, Wikipedia). 3. Ask yourself: Does this seem fair and balanced?).
- Reliable Sources: Identify 2-3 reliable websites for fact-checking (e.g., BBC News, Wikipedia).
🔥 Implementation Steps
The Hook (10 mins):
- Show a blatantly biased or incorrect AI-generated text or image. Ask learners: “Do you believe this? Why or why not?” Introduce the idea that AI can be wrong and biased.
Bias Detective (25 mins):
- In pairs, learners receive a worksheet with 2-3 short AI-generated texts. Their task is to underline words or sentences that seem like a stereotype or feel inaccurate about their culture or another culture.
- Sample Prompt for Demo Text: “Describe a traditional family from [Country].” (Often produces overly simplistic or stereotypical views).
Fact-Checking Rally (25 mins):
- Show an AI-generated text with a factual error (e.g., an incorrect date for a historical event, a wrong capital city). Using the “Fact-Checking Checklist,” guide the class through verifying the information using trusted sources online.
Discussion & Empowerment (15 mins):
- Discuss why AI has these problems (trained on biased internet data). Recap the strategies learned. Empower them: “You are the expert on your own experience. You can question the AI.”
Tip: Choose examples that are relevant to the learners’ cultural backgrounds to make the activity more engaging and meaningful.
🧂 Differentiation & Inclusion
- For B1 learners: Focus on identifying obvious stereotypes in images or single sentences.
- For B2/C1 learners: Use longer texts and focus on nuanced bias and complex fact-checking.
🍴 Assessment & Feedback
| Criterion | Excellent | Good | Needs Work |
| Bias Identification | Identifies subtle and obvious biases and can explain why they are problematic | Identifies obvious stereotypes | Struggles to distinguish biased statements from neutral ones |
| Fact-Checking | Successfully uses multiple sources to verify and correct information | Verifies information with guidance | Relies solely on the AI-generated information |
⚠️ Ethics, Privacy & Safeguarding
- Cultural Sensitivity: Handle discussions about cultural bias with sensitivity. Frame it as a flaw in the AI’s training, not an attack on any culture.
- Critical Thinking vs. Distrust: Use this lesson to build critical thinking, not to foster distrust in all technology.
🍱 Extensions / Follow‑Ups
- AI Feedback: Learners write a “letter to the AI” explaining what it got wrong about their country or culture.
- Research & Verification: Follow-up lesson on using AI for research, focusing on verification and citation.
🔗 Cross‑Subject Link
References & Resources
BBC News: bbc.com/news – A reliable source for fact-checking current events.
Wikipedia: wikipedia.org – A good starting point for verifying general facts.