Elamur Research⌕
☰
home / papers / 2023 / Prototype of a robotic system to assist the learning process of English language with text-generation through DNN

Prototype of a robotic system to assist the learning process of English language with text-generation through DNN

Paper · 2023 · Carlos Morales-Torres, Mario Campos-Soberanis and Diego Campos-Sobrino

Summary

The authors build a humanoid robot prototype that supports self-study of English by generating text suited to the learner's proficiency. With help from an IELTS examiner, they wrote a small corpus graded into three levels (4,785 sentences, about 150,000 words). They trained several next-word predictors in Keras on it: a simple LSTM, a fine-tuned BERT, an encoder-decoder LSTM and a bidirectional LSTM. The text is produced word by word: each prediction is added to the input and the window slides forward. The body was designed with Denavit–Hartenberg kinematics, modelled in Fusion360 and 3D-printed. A Flask service connects it to cloud speech services and to a Gradio tablet interface. The bidirectional LSTM scored best on held-out data, and the authors explain BERT's weaker score by the scarcity of fine-tuning data. A five-day pilot with three learners, scored on the IELTS rubric, showed modest gains in grammar and fluency but none in pronunciation. The study is preliminary. Its value is as a demonstration that a small, level-controlled corpus plus a light recurrent model can drive an interactive tutor in real time.

Key points

  • Corpus: 4,785 sentences, roughly 150,000 words, written with an IELTS evaluator and split by level (elemental, pre-intermediate, and a higher tier). Punctuation is used to segment sentences and is then removed.
  • Test accuracy on next-word prediction (80/20 split, plus 10% validation during training):
    • Simple LSTM: 80%
    • fine-tuned BERT: 80%
    • Encoder-Decoder LSTM: 89%
    • Bidirectional LSTM: 95%, trained for 20 epochs
  • Pipeline:
    • speech capture and cloud transcription;
    • a Flask endpoint takes the transcription as a seed and returns a 5-word continuation;
    • the reply is spoken aloud through a speaker;
    • a Gradio GUI serves users with speech or hearing impairments.
  • Pilot study: three learners (elementary, pre-intermediate, upper-intermediate), 50 minutes a day for five days. The paper states this as 250 minutes in one place and as "hours" in the conclusion.
  • Reported average gains:
    • grammatical range: about 4%
    • grammatical accuracy: about 4%
    • fluency: about 3.33%
    • pronunciation: no change
  • The validation curves were unstable, which the authors attribute to uneven level mixing within the corpus.

Interesting ideas

  • I1. An expert authors the corpus in explicit proficiency tiers, so the generator's output complexity is controlled by which data it learned from rather than by post-hoc filtering; this suits any system that must adapt text to a reader.
  • I2. With very little in-domain data, a compact recurrent model beat a fine-tuned large transformer, a reminder to always run a cheap baseline before assuming the big model wins.
  • I3. The authors blame noisy validation curves on mixed difficulty levels inside one training set, which suggests that heterogeneous data strata can destabilise learning.
  • I4. The before/after scores are broken down by rubric dimension, and one dimension (pronunciation) did not move; that dimension acts as an informal check that the gains are specific rather than general noise.
  • I5. A single inference service is exposed through two front ends, voice and a tablet GUI, keeping the model independent of the interaction channel and accessible to more users.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): Generation from the meaning graph could take a target-level parameter and pick syntactic constructions per tier, checked against small level-graded reference sets.
  • I2 (learn): Every evaluation of a gparse or LLM-teacher component should report a lightweight baseline (e.g. the perceptron parser) trained on the same small slice, as a sanity control.
  • I3 (learn): Silver treebank data could be tagged by source and register, with training curves tracked per stratum to locate which slice causes instability.
  • I4 (answer): School-test evaluations could report per-skill scores and include a skill the change should not affect, as a negative control on claimed improvements.
  • I5 (answer): The readers that answer could sit behind one explainable API, with demos (chat, quiz UI) as thin clients that all show the same reasoning trace.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

Also on the byline: Universidad Politécnica de Yucatán.

 Fields, topics and concepts

📚 Shelves 6The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 worksPsychology · 1,552 worksSocial Psychology · 306 works

🧭 Topics 2Research subjects and languages the work is about.English · 2,263 worksText generation · 202 works

Cite

Cite as Carlos Morales-Torres, Mario Campos-Soberanis and Diego Campos-Sobrino. 2023. Prototype of a robotic system to assist the learning process of English language with text-generation through DNN. arXiv:2309.11142.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2309.11142 · arXiv 2309.11142

@misc{moralestorres2023prototype,
  title = {{Prototype of a robotic system to assist the learning process of English language with text-generation through DNN}},
  author = {Carlos Morales-Torres, Mario Campos-Soberanis and Diego Campos-Sobrino},
  year = {2023},
  doi = {10.48550/arxiv.2309.11142},
  eprint = {2309.11142},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.11142}
}

References

3 references, 3 of them with a card of ours. ↗ leads to the work.

  1. 1. Abid, A., Abdalla, A., Abid, A., Khan, D., Alfozan, A., Zou, J.: Gradio: Hassle-free sharing and testing of ml models in the wild. arXiv preprint arXiv:1906.02569 (2019) 2. Akkila, A., Almasri, A., Ahmed, A., Al-Masri, N., Abu, Y., Mahmoud, A., Zaqout, I., Abu-Naser, S.: Survey of intelligent tutoring systems up to the end of 2017. International Journal of Academic Information Systems Research vol. 3, 36–49 3. Anwar, S., Bascou, N., Menekse, M.: A systematic review of studies on educational robotics. Journal of Pre-College Engineering Education Research (J-PEER) 9(2) (2019) 4. Breuel, T.M.: Benchmarking of LSTM networks. CoRR abs/1508.02774 (2015), http://arxiv.org/abs/1508.02774 5. Cho, K., Merrienboer, B.v., Gulcehre, C., Bougares, F., Schwenk, H., Bengio, Y.: Learning Phrase Representations using RNN Encoder-Decoder for Statistical Machine Translation. In: EMNLP (2014), https://hal.archives-ouvertes.fr/hal-01433235 6. Dale, R.: Gpt-3: What’s it good for? Natural Language Engineering 27(1), 113–118 (2021). https://doi.org/10.1017/S1351324920000601 7. Devlin, J., Chang, M., Lee, K., Toutanova, K.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Burstein, J., Doran, C., Solorio, T. (eds.) Proceedings of the NAACL-HLT Volume 1. pp. 4171–4186. Association for Computational Linguistics (2019), https://doi.org/10.18653/v1/n19-1423 8. Ezen-Can, A.: A comparison of lstm and bert for small corpus. ArXiv abs/2009.05451 (2020), http://arxiv.org/abs/2009.05451 ↗
  2. Carlos Morales-Torres, Mario Campos-Soberanis, Diego Campos-Sobrino 9. Grubbs, M.: Robotics intrigue middle school students and build stem skills. Technology and Engineering Teacher 72(6), 12–16 (2013) 10. Han, J., Kim, D.: r-learning services for elementary school students with a teaching assistant robot. In: 2009 4th ACM/IEEE International Conference on Human-Robot Interaction (HRI). pp. 255–256 (2009) 11. Hayat, A.A., Chittawadigi, R.G., Udai, A.D., Saha, S.K.: Identification of denavit-hartenberg parameters of an industrial robot. In: Proceedings of Conference on Advances In Robotics. p. 1–6. AIR ’13, Association for Computing Machinery, New York, NY, USA (2013). https://doi.org/10.1145/2506095.2506121 12. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF Models for Sequence Tagging (2015), Retrievedfromhttp://arxiv.org/abs/1508.01991 13. Joselson, N., Hallén, R.: Emotion classification with natural language processing (comparing bert and bi-directional lstm models for use with twitter conversations) (2019), student Paper 14. Khurana, D., Koli, A., Khatter, K., Singh, S.: Natural language processing: State of the art, current trends and challenges. CoRR abs/1708.05148 (2017), http://arxiv.org/abs/1708.05148 15. Lai Poh, E.T., Albert, C., Pei-Wen, T., I-Ming, C., Song Huat, Y.: A review on the use of robots in education and young children. Journal of Educational Technology & Society 19(2), 148–163 (2016), http://www.jstor.org/stable/jeductechsoci.19.2.148 16. Melchior, A., Cohen, F., Cutter, T., Leavitt, T.: More than robots: An evaluation of the first robotics competition participant and institutional impacts. In: Brandeis University Center for Youth and Communities Heller School for Social Policy and Management. (2005) 17. Menekse, M., Higashi, R., Schunn, C.D., Baehr, E.: The role of robotics teams collaboration quality on team performance in a robotics tournament. Journal of Engineering Education 106(4), 564–584 (2017) 18. Merity, S., Keskar, N.S., Socher, R.: Regularizing and optimizing LSTM language models. 6th International Conference on Learning Representations, ICLR 2018 -Conference Track Proceedings (2018) 19. Muzaffar, S., Afshari, A.: Short-Term Load Forecasts Using LSTM Networks. Energy Procedia 158, 2922–2927 (2019), https://www.sciencedirect.com/science/article/pii/S1876610219310008 20. Newton, D.P., Newton, L.D.: Humanoid robots as teachers and a proposed code of practice. Frontiers in Education 4, 125 (2019), https://www.frontiersin.org/article/10.3389/feduc.2019.00125 21. Peters, F.: Master thesis: Design and implementation of a chatbot in the context of customer support (2018), http://hdl.handle.net/2268.2/4625 22. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J.: Exploring the limits of transfer learning with a unified text-to-text transformer. CoRR abs/1910.10683 (2019), http://arxiv.org/abs/1910.10683 23. Rosvall, E.: Comparison of sequence classification techniques with bert for named entity recognition. Electrical Engineering and computer science faculty (2019) 24. Santhanam, S.: Context based text-generation using LSTM networks. CoRR abs/2005.00048 (2020), https://arxiv.org/abs/2005.00048 25. Topal, M.O., Bas, A., van Heerden, I.: Exploring transformers in natural language generation: Gpt, bert, and xlnet. CoRR abs/2102.08036 (2021), https://arxiv.org/abs/2102.08036 ↗
  3. Robotic system to assist the learning of English through DNN 26. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, L., Polosukhin, I.: Attention is all you need. CoRR abs/1706.03762 (2017), http://arxiv.org/abs/1706.03762 ↗

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/morales-torres-2023-prototype-robotic-system-assist-learning/