📄 DeMINT: Automated Language Debriefing for English Learners via AI Chatbot Analysis of Meeting Transcripts

Summary

DeMINT is a prototype conversational tutor for non-native English speakers. It reviews recordings of video meetings the learners took part in and then talks with each learner about the language mistakes they made. The system is a modular pipeline:

• pyannote.audio splits the audio by speaker.
• A Whisper model, adapted on spoken sentences that contain grammatical errors, transcribes the speech.
• A T5 model trained on JF-LEG proposes corrections.
• ERRANT labels the edits between the original and corrected sentences.
• An LLM turns those labels into detailed explanations, using a few examples in the prompt.
• Retrieval from English textbooks adds supporting material.
• A fine-tuned Llama-3.1-8B suggests an empathetic teaching stance.
• A Python orchestrator assembles all of this into a prompt, and GPT-4 writes the tutor's next turn.

The evaluation was small: seven Spanish-speaking B2/C1 learners used the system after 15 recorded calls. They enjoyed it and wanted similar tools in the future. Their main complaints were the tutor's fluency and how reliably it detected errors. The work shows how standard MT-adjacent components can be combined into a personalised, error-driven tutor.

Key points

• Built by the Transducens group at Universitat d'Alacant (January–September 2024), funded through the UTTER project under Horizon Europe (GA 101,070,631).
• Components:
  • pyannote.audio for diarization
  • fine-tuned Whisper for speech recognition
  • T5 fine-tuned on JF-LEG for grammatical error correction
  • ERRANT for edit annotation
  • RAG over textbooks for supporting material
  • Llama-3.1-8B trained on teacher–student dialogues for empathetic guidance
  • GPT-4 for generating the tutor's next turn
• The evaluation involved 7 L1-Spanish learners at B2/C1, 15 video calls and roughly 75 minutes of chatbot interaction, with answers collected on a 1–5 Likert survey.
• Every participant rated enjoyment 4 or 5, and five of seven gave a 4 for helping their English. Six of seven rated interest in future use 4 or 5.
• Fluency and error-detection accuracy were the weakest aspects, both averaging 3.

Interesting ideas

• I1. The speech recognizer is fine-tuned on utterances that contain grammatical errors, so it records what the learner actually said instead of silently fixing it.
• I2. Errors are found by correcting a sentence first and then diffing the original against the correction with ERRANT, which yields typed edits rather than an opaque rewrite.
• I3. The LLM only verbalises errors that a symbolic tool has already labelled, so its explanations stay anchored to structured annotations.
• I4. A deterministic orchestrator walks through the errors and builds the prompt, while a separate small model advises only on tone and teaching approach.
• I5. Learners named error-detection accuracy as the factor that most limited trust in the tutor.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (tokens): The tokenizer and tagger could be trained or tested on deliberately ungrammatical text so they keep learner errors visible instead of normalising them away.
• I2 (learn): When a self-induced repair rule changes a parse, Mova could diff the tree before and after the repair to log a typed, explainable edit category.
• I3 (back): Generation could follow the same split, with the meaning graph fixing what to say and a language model only choosing the wording, which keeps the output traceable.
• I4 (answer): Mova's tutoring "students" could keep content selection in explicit code and treat style or empathy as a separate, swappable advisory layer.
• I5 (learn): Evaluation should measure detection precision of the upstream analysis separately from end-user satisfaction, since a weak detector undermines everything built on top of it.