Elamur Research⌕
☰
home / papers / 2002 / An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor

An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor

Paper · 2002 · V Aleven

Summary

Students often finish instruction with shallow knowledge that does not transfer. In geometry, for example, they may decide two angles are equal simply because they look alike in the diagram. Laboratory studies show that self-explanation helps, but prompting each learner individually is impractical in real classrooms. Few studies had also compared self-explanation against an already proven method.

Aleven and Koedinger extended the Geometry Cognitive Tutor, intelligent software for guided problem-solving practice. In the new version, students had to justify every step by naming the relevant theorem or definition, and the tutor gave feedback and hints on these justifications. Two classroom experiments compared this version with the original tutor, which itself outperforms traditional teaching.

Students who explained became better at justifying their solutions and performed better on transfer problems. The authors attribute this to declarative knowledge that links visual patterns with verbal rules, and to less reliance on shallow procedural heuristics. The work shows that self-explanation's benefits can come from a fairly simple, scalable computer tool rather than a human tutor.

Key points

  • The study was set in Cognitive Tutor Geometry™, a high-school curriculum that splits time roughly equally between classroom lessons and tutor practice. The control condition used the earlier tutor version, which had no explanation support.
  • Explanations were references to problem-solving principles entered through a structured interface, not free natural language, and they received feedback and hints. Explainers also consulted the tutor's on-line Glossary more often.
  • The motivation came from an earlier formative evaluation with 71 students in two schools. Those students were better at producing answers than at stating the geometry reasons behind them.
  • The authors cite published effect sizes relative to classroom instruction:
    • about 1 SD for Cognitive Tutors;
    • about 0.4 SD for ordinary human tutors;
    • an estimated 2.0 SD for expert tutors.
  • At the time of writing, the Algebra I tutor was used in about 700 US schools and the Geometry tutor in about 100. The self-explanation version is sold commercially through Carnegie Learning, Inc.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States633 works · 195,218 citations

 Fields, topics and concepts

📚 Shelves 5The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 worksPsychology · 1,673 worksExperimental and Cognitive Psychology · 950 worksDevelopmental and Educational Psychology · 533 works

Cite

Cite as V Aleven. 2002. An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor. Cognitive Science 26(2).

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.1016/s0364-0213(02)00061-7

@inproceedings{aleven2002effective,
  title = {{An effective metacognitive strategy: learning by doing and explaining with a computer-based Cognitive Tutor}},
  author = {V Aleven},
  booktitle = {Cognitive Science 26(2)},
  year = {2002},
  doi = {10.1016/s0364-0213(02)00061-7},
  url = {https://doi.org/10.1016/s0364-0213(02)00061-7}
}

References

69 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Ahlum-Heath, M. E., & DiVesta, F. J. (1986). The effect of conscious controlled verbalization of a cognitive strategy on transfer in problem solving. Memory and Cognition, 14, 281–285. find
  2. Aleven, V., & Koedinger, K. R. (2000a). Limitations of student control: do students know when they need help? In G. Gauthier, C. Frasson, & K. VanLehn (Eds.), Proceedings of the 5th International Conference on Intelligent Tutoring Systems, ITS 2000 (pp. 292–303). Berlin: Springer Verlag. find
  3. Aleven, V., & Koedinger, K. R. (2000b). The need for tutorial dialog to support self-explanation. In C. P. Rose & R. Freedman (Eds.), Building Dialogue Systems for Tutorial Applications. Papers from the 2000 AAAI Fall Symposium (pp. 65–73). Menlo Park, CA: AAAI Press. find
  4. Aleven, V., Koedinger, K. R., Sinclair, H. C., & Snyder, J. (1998). Combatting shallow learning in a tutor for geometry problem solving. In B. P. Goettl, H. M. Halff, C. L. Redfield, & V. J. Shute (Eds.), Intelligent Tutoring Systems, Fourth International Conference, ITS ’98 (pp. 364 –373). Lecture… find
  5. Aleven, V., Popescu, O., & Koedinger, K. R. (2001). Towards tutorial dialog to support self-explanation: adding natural language understanding to a cognitive tutor. In J. D. Moore, C. L. Redfield, & W. L. Johnson (Eds.), Artificial Intelligence in Education: AI-ED in the Wired and Wireless Future (p… find
  6. Amsterdam, IOS Press Anderson, J. R. (1999). Cognitive Psychology and Its Implications (5th ed.). New York: Worth Publishing. find
  7. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive Tutors: Lessons Learned. The Journal of the Learning Sciences, 4, 167–207 find
  8. Anderson, J. R., & Lebière, C. (1998). The Atomic Components of Thought. Mahwah, NJ: Erlbaum. find
  9. Berardi-Coletta, B., Buyer, L. S., Dominowsky, R. L., & Rellinger, E. R. (1995). Metacognition and problem-solving: a process-oriented approach. Journal of Experimental Psychology: Learning Memory, and Cognition, 21 (1), 205–223. find
  10. Bereiter, C., & Scardamalia, M. (1985). Cognitive coping strategies and the problem of “inert knowledge.” In S. Chipman, J. Segal, & R. Glaser (Eds.), Thinking and Learning Skills: Research and Open Question (vol. 2, pp. 65– 80). Hillsdale, NJ: Erlbaum. find
  11. Berry, D. C. (1983). Metacognitive experience and transfer of logical reasoning. Quarterly Journal of Experimental Psychology, 35A, 39 – 49. find
  12. Bielaczyc, K., Pirolli, P. L., & Brown, A. L. (1995). Training in self-explanation and self-regulation strategies: investigating the effects of knowledge acquisition activities on problem solving. Cognition and Instruction, 13, 221–252. find
  13. Bloom, B. S. (1984). The 2 sigma problem: the search for methods of group instruction as effective as one-to-one tutoring. Educational Researcher, 13, 3–16. find
  14. Brown, A. L., & Campione, J. C. (1996). Guided discovery in a community of learners. In K. McGilly (Ed.), Classroom Lessons: Integrating Cognitive Theory and Classroom Practice (pp. 229 –270). Cambridge, MA: The MIT Press. find
  15. Brown, A. L., & Kane, M. J. (1988). Preschool children can learn to transfer: learning to learn and learning from example. Cognitive Psychology, 20, 493–523. find
  16. Burger, W. F., & Shaughnessy, J. M. (1986). Characterizing the Van Hiele levels of development in geometry. Journal for Research in Mathematics Education, 17, 31– 48. find
  17. Chase, W. G., & Simon, H. A. (1973). The mind’s eye in chess. In W. G. Chase (Ed.), Visual Information Processing. find
  18. New York: Academic Press Chi, M. T. H. (2000). Self-explaining expository texts: the dual processes of generating inferences and repairing mental models. In R. Glaser (Ed.), Advances in Instructional Psychology, (pp. 161–237). Mahwah, NJ: Erlbaum. find
  19. Chi, M. T. H., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: how students study and use examples in learning to solve problems. Cognitive Science, 13, 145–182. find
  20. Chi, M. T. H., de Leeuw, N., Chiu, M., & Lavancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439 – 477. find
  21. Chi, M. T. H., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.
  22. Cognition and Technology Group at Vanderbilt. (1990). Anchored instruction and its relationship to situated cognition. Educational Researcher, 19 (6), 2–10. find
  23. Cognition and Technology Group at Vanderbilt. (1996). From visual word problems to learning communities: changing conceptions of cognitive research. In K. McGilly (Ed.), Classroom Lessons: Integrating Cognitive Theory and Classroom Practice (pp. 157–200). Cambridge, MA: The MIT Press. find
  24. Cohen, P. A., Kulik, J. A., & Kulik, C. C. (1982). Education outcomes of tutoring: a meta-analysis of findings. American Educational Research Journal, 19 (2), 237–248. find
  25. Collins, A., & Brown, J. S. (1988). The computer as a tool for learning through reflection. In H. Mandl & A. Lesgold (Eds.), Learning Issues for Intelligent Tutoring Systems (pp. 1–18). find
  26. New York: Springer Verlag Conati, C., & VanLehn, K. (2000). Toward computer-based support of meta-cognitive skills: a computational framework to coach self-explanation. International Journal of Artificial Intelligence in Education, 11, 398 – 415. find
  27. Corbett, A. T. (2001). Cognitive computer tutors: solving the two-sigma problem. In M. Bauer, P. Gmytrasiewicz, & J. Vassileva (Eds.) User Modeling 2001: Proceedings of the Eighth International Conference, UM 2001 (pp. 137–147). New York: Springer. find
  28. Corbett, A. T., Koedinger, K. R., & Hadley, W. S. (2001). Cognitive tutors: from the research classroom to all classrooms. In P. Goodman (Ed.), Technology Enhanced Learning: Opportunities For Change (pp. 235–263). Mahway, NJ: Erlbaum. find
  29. Corbett, A. T., & Trask, H. (2000). Instructional interventions in computer-based tutoring: differential impact on learning time and accuracy. In Proceedings of the ACM CHI ’2000 Conerence on Human Factors in Computing Systems (pp. 97–104). find
  30. Corbett, A. T., & Anderson, J. R. (1995). Knowledge tracing: modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4, 253–278. find
  31. De Groot, A. (1966). Perception and memory versus thought: some old ideas and recent findings. In B. Kleinmuntz (Ed.), Problem Solving. NY: Wiley. find
  32. Dienes, Z., & Perner, J. (1999). A theory of implicit and explicit knowledge. Behavioral and Brain Sciences, 22 (5). find
  33. Du Boulay, B., Luckin, R., & del Soldato, T. (1999). The plausibility problem: human teaching tactics in the ‘hands’ of a machine. In S. P. Lajoie & M. Vivet (Eds.), Artificial Intelligence in Education, Open Learning Environments: New Computational Technologies to Support Learning, Exploration, and… find
  34. Ferguson-Hessler, M. G. M., & de Jong, T. (1990). Studying physics texts: differences in study processes between good and poor performers. Cognition and Instruction, 7, 41–54. find
  35. Flavell, J. (1979). Metacognition and cognitive monitoring. A new area of cognitive development inquiry. American Psychologist, 34, 906 –911. find
  36. Gagné, R. M., & Smith, E. C. (1962). A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63 (1), 12–18. find
  37. Graesser, A. C., Person, N. K., & Magliano, J. P. (1995). Collaborative dialogue patterns in naturalistic one-to-one tutoring. Applied Cognitive Psychology, 9 (6), 495–522. find
  38. Griffin, S., Case, R., & Siegler, R. S. (1994). Rightstart: providing the central conceptual prerequisities for first formal learning of arithmetic to students at risk for school failure. In K. McGilly (Ed.), Classroom Lessons: Integrating Cognitive Theory and Classroom Practice (pp. 25– 49). Cambri… find
  39. Holland, J. H., Holyoak, K. J., Nisbett, R. E., & Thagard, P. R. (1986). Induction: Processes of Inference, Learning, and Discovery. Cambridge, MA: The MIT Press. find
  40. Judd, C. H. (1908). The relation of special training to general intelligence. Educational Review, 36, 28 – 42. find
  41. Kalchman, M., Moss, J., & Case, R. (2001). Psychological models for development of mathematical understanding: rational numbers and functions. In S. Carver & D. Klahr (Eds.), Cognition and instruction: Twenty-five years of progress (pp. 1–38). Mahwah, NJ: Erlbaum. find
  42. Katona, G. (1940). Organizing and Memorizing. New York: Columbia University Press. find
  43. Koedinger, K. R., & Anderson, J. R. (1990). Abstract planning and perceptual chunks: elements of expertise in geometry. Cognitive Science, 14, 511–550. find
  44. Koedinger, K. R., Anderson, J. R., Hadley, W. H., & Mark, M. A. (1997). Intelligent tutoring goes to school in the big city. International Journal of Artificial Intelligence in Education, 8, 30 – 43. find
  45. Koedinger, K. R., Corbett, A. T., Ritter, S., & Shapiro, L. (2000). Carnegie Learning’s Cognitive Tutor™: Summary Research Results. White paper. Available from Carnegie Learning Inc., 1200 Penn Avenue, Suite 150, Pittsburgh, PA 15222, E-mail: info@carnegielearning.com, Web: http://www.carnegielearni… ↗
  46. Kulik, C. C., & Kulik, J. A. (1991). Effectiveness of computer-based instruction: an updated analysis. Computers in Human Behavior, 7, 75–95. find
  47. Lepper, M. R., Woolverton, M., Humme, D. L., & Gurtner, J. L. (1993). Motivational techniques of expert human tutors: lessons for the design of computer-based tutors. In S. Lajoie & S. Derry (Eds.), Computers as Cognitive Tools (pp. 75–105). Hillsdale, NJ: Erlbaum. find
  48. Lovett, M. C. (1992). Learning by problem solving versus by examples: the benefits of generating and receiving information. In Proceedings of the Fourteenth Annual Meeting of the Cognitive Science Society (pp. 956 – 961). Hillsdale, NJ: Erlbaum. find
  49. Mwangi, W., & Sweller, J. (1998). Learning to solve compare word problems: the effect of example format and generating self-explanations. Cognition and Instruction, 16, 173–199. find
  50. NCTM. (1989). Curriculum and Evaluation Standards for School Mathematics. National Council of Teachers of Mathematics. Reston, VA: The Council. find
  51. Needham, D. R., & Begg, I. M. (1991). Problem-oriented training promotes spontaneous analogical transfer, memory-oriented training promotes memory for training. Memory and Cognition, 19, 543–557. find
  52. Paivio, A. (1971). Imagery and Verbal Processes. New York: Holt, Rinehart, and Winston. find
  53. Palincsar, A. S., & Brown, A. L. (1984). Reciprocal teaching of comprehension-fostering and comprehension monitoring activities. Cognition and Instruction, 1, 117–175. find
  54. Renkl, A. Learning from worked-out examples: instructional explanations supplement self-explanations. Learning and Instruction (in press). find
  55. Renkl, A. (1997). Learning from worked-out examples: a study on individual differences. Cognitive Science, 21, 1–29. find
  56. Renkl, A., Stark, R., Gruber, H., & Mandl, H. (1998). Learning from worked-out examples: the effects of example variability and elicited self-explanations. Contemporary Educational Psychology, 23, 90 –108. find
  57. Rose, C. P., & R. Freedman (Eds.). (2000). Building Dialogue Systems for Tutorial Applications. Papers from the 2000 AAAI Fall Symposium. Menlo Park, CA: AAAI Press. find
  58. Scardamalia, M., Bereiter, C., & Lamon, M. (1996). The CSILE Project: trying to bring the classroom into world 3. In K. McGilly (Ed.), Classroom Lessons: Integrating Cognitive Theory and Classroom Practice (pp. 201–228). Cambridge, MA: The MIT Press. find
  59. Schoenfeld, A. H. (1987). What’s all the fuss about metacognition? In A. H. Schoenfeld (Ed.), Cognitive Science and Mathematics Education (pp. 189 –215). Hillsdale, NJ: Erlbaum. find
  60. Schofield, J. (1995). Computers and Classroom Culture. Cambridge University Press. find
  61. Simon, H. A. (1987). The information-processing explanation of Gestalt phenomena. In Models of Thought, Volume II (pp. 481– 493). New Haven, CT: Yale University Press. (Reprinted from Computers in Human Behavior, 2, 1–15, 1987.) find
  62. Schwartz, J. L., Yerushalmy, M., & Wilson, B. (Eds.). (1993). The Geometric Supposer: What is it a case of? Hillsdale, NJ: Erlbaum. find
  63. Singley, K., & Anderson, J. R. (1989). The transfer of cognitive skill. Cambridge, MA: Harvard University Press. find
  64. VanLehn, K., Jones, R. M., & Chi, M. T. H. (1992). A model of the self-explanation effect. The Journal of The Learning Sciences, 2 (10), 1–59. find
  65. Wiske, M. S., & Houde, R. (1993). From recitation to construction: Teachers change with new technologies. In J. L. Schwartz, M. Yerushalmy, & B. Wilson (Eds.). The Geometric Supposer: What is it a case of? Hillsdale, NJ: Erlbaum. find
  66. Webb, N. M. (1989). Peer interaction and learning in small groups. International Journal of Education Research, 13, 21–39. find
  67. Wenger, E. (1987). Artificial Intelligence and Tutoring Systems: Computational and Cognitive Approaches to the Communication of Knowledge. Los Altos, CA: Morgan Kaufmann. find
  68. Wertheimer, M. (1959). Productive Thinking. New York: Harper & Row. find
  69. White, B. Y., Shimoda, T. A., & Fredericksen, J. R. (1999). Enabling students to construct theories of collaborative inquiry and reflective learning: computer support for metacognitive development. International Journal of Artificial Intelligence in Education, 10 (2), 151–182. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/aleven-2002-effective-metacognitive-strategy-learning-doing/