📄 CueBuddy: helping non-native English speakers navigate English-centric STEM education

Summary

Students in English-taught STEM courses, especially in the Global South, often understand everyday English but struggle with specialised vocabulary. Many learned their prerequisites in a lower-resource language. Full live speech translation can address this, but it is costly to scale, weak on technical terminology, and may pull attention away from the lecture. This short system-description paper proposes CueBuddy, which gives targeted "lexical cues" instead of translating everything. The planned pipeline has three stages:

1. Streaming speech recognition transcribes the lecture.
2. A keyword extractor picks out technical terms from the transcript.
3. Each detected term is looked up in a pre-built multilingual glossary, and a brief explanation appears in the student's language.

The paper reports no experiments or numbers. It describes the design, likely weak points and the evaluation work still needed. The main contribution is a framing: a lightweight design, potentially usable offline, that helps students who sit between "no English" and "full fluency."

Key points

• The proposed components are Whisper-tiny for ASR, optionally fine-tuned on scientific speech data such as BhasaAnuvaad, plus biased decoding ("jargon injection") to improve recognition of domain terms.
• keyBERT handles keyword extraction. The author expects it to perform worse on ASR output than on clean text and to have trouble with scientific words unlike everyday English.
• LLMs generate the glossary in advance, with short explanations per target language. The example entry is "backpropagation" in Hindi (transliterated) and Swahili. Quality may suffer for languages with little STEM data.
• The design assumes the lecturer speaks only English, without code-switching.
• The paper gives no quantitative evaluation. Future work includes:
  • subject-specific test sets (first-year math expected to do better than niche engineering or medical topics),
  • classroom user and impact studies,
  • decoding speech directly into keywords to cut latency.

Interesting ideas

• I1. Supporting only the specific gap, the rare technical terms, instead of re-rendering the whole message keeps help sparse and non-intrusive for users who already handle most of the language.
• I2. An expensive LLM is used once, offline, to compile a resource that runtime only consults by cheap lookup, separating generation cost from inference.
• I3. Biasing a recogniser's decoding toward a domain term list improves accuracy on jargon without retraining the whole model.
• I4. The author warns that an extractor tuned on clean text degrades on noisy upstream output, so each stage should be tested on what the previous stage actually produces.
• I5. The paper calls for evaluation split by subject, expecting accuracy to vary sharply between common and niche domains.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

• I1 (answer): A student reader could flag only out-of-course domain terms in a test question and attach a short definition, explaining where comprehension may fail without paraphrasing everything.
• I2 (learn): LLM teacher tiers can pre-compile term glossaries or definitions that are then frozen and audited, keeping the live pipeline deterministic and explainable.
• I3 (tokens): The tagger and lemmatizer could take a domain lexicon as a prior for unknown scientific words, such as compound technical nouns, instead of guessing from suffixes alone.
• I4 (tree): Evaluate the parser on noisy input (lowercased, unpunctuated, transcript-like text) as a separate slice, not only on gold-tokenised treebank sentences.
• I5 (answer): Report school-course test results per subject and domain so that weak areas of compiled knowledge become visible instead of being averaged away.