Elamur Research⌕
☰
home / papers / 2023 / Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc

Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc

Paper · 2023 · Doug Lenat and Gary Marcus

Summary

Lenat and Marcus argue that today's generative systems such as ChatGPT and Bard optimize for fluent, plausible output rather than truth. This makes them unreliable, inconsistent over time and prone to errors no human would make. They trace this to weak knowledge, reasoning and world models. The paper is a position piece and introduces no new dataset or benchmark. It first proposes 16 capabilities that a trustworthy general AI should have, including explanation, deduction, induction, analogy, abduction, theory of mind, handling of quantifiers and modal statements, defeasible beliefs, and weighing pro and con arguments. It then measures Cyc against these capabilities. Cyc is a long-running symbolic system built from hand-curated knowledge and an inference engine. The authors claim Cyc avoids the usual trade-off between expressive logic and speed, reasoning in higher-order logic in real time with fully traceable justifications. Their main conclusion is that LLMs and Cyc-style symbolic reasoning should be combined. They outline several concrete ways to do this, aiming at AI whose answers can be audited and trusted.

Key points

  • The paper lists 16 desiderata for trustworthy AI. Examples include step-by-step explanation with provenance, quantifier fluency (e.g., the difference between everyone having "a king" and everyone having "a mother"), modal reasoning, and revising beliefs when new information arrives.
  • The motivating LLM failures include:
    • an invented journal citation;
    • lawyers sanctioned for citing six nonexistent cases;
    • month-to-month instability on primality questions (Chen et al., 2023);
    • adversarial attacks (Zou et al., 2023).
  • The authors claim Cyc performs higher-order logic inference in real time. They contrast this with fast but less expressive representations such as knowledge graphs.
  • The proposed hybrid methods are:
    • LLMs draft rough CycL translations of English, which Cyc then repairs and contextualizes;
    • Cyc's CycL-to-English NLG generates training corrections for the LLM;
    • Cyc acts as an inference "amplifier" that derives new consequences from dialogue content;
    • Cyc supplies explanations and audit trails.
  • The authors cite a Cycorp and Cleveland Clinic project for NIH's National Library of Medicine as an example. In it, Cyc built multi-step causal chains explaining noisy mutation–disease correlations. These chains produced intermediate predictions that could be checked against patient data.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 19,503 worksArtificial Intelligence · 18,805 works

🧭 Topics 2Research subjects and languages the work is about.Language models & pretraining · 4,145 worksReasoning · 1,948 works

Cite

Cite as Doug Lenat and Gary Marcus. 2023. Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc. arXiv:2308.04445.

The paper itself, not this page. BibTeX from the library's record of the work. DOI 10.48550/arxiv.2308.04445 · arXiv 2308.04445

@misc{lenat2023getting,
  title = {{Getting from Generative AI to Trustworthy AI: What LLMs might learn from Cyc}},
  author = {Doug Lenat and Gary Marcus},
  year = {2023},
  doi = {10.48550/arxiv.2308.04445},
  eprint = {2308.04445},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.04445}
}

References

14 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Barnes, K. T., “Hintikka's Argument for the Need for Quantifying into Opaque Contexts” Philosophical Studies: An International Journal for Philosophy in the Analytic Tradition 23:6, December, 1972, 385-392.’ Chen, Lingjiao,i Zaharia, Matei, and Zou,James Zou. How Is ChatGPT’s Behavior Changing over… find
  2. Davis, Ernest, and Gary Marcus, “Commonsense reasoning and commonsense knowledge in artificial intelligence”, Communications of the ACM, 58:924, August 2015, pp 92–103.
  3. Faust, Jeremy, https://www.medpagetoday.com/opinion/faustfiles/102723, January 20, 2023 ↗
  4. Goertzel, Ben, “Three Viable Paths to True AGI”, https://bengoertzel.substack.com/p/three-viable-paths-to-true-agi? , August 24, 2022 ↗
  5. Kahneman, Daniel. Thinking, Fast and Slow. Farrar, Straus and Giroux, 2011. find
  6. Lenat, D., Prakash, M., & Shepherd, M. (1986). CYC: Using common sense knowledge to overcome brittleness and knowledge acquisition bottlenecks. AI Magazine, 6(4), 65–85. find
  7. Lenat, D., and R. V. Guha, Building Large Knowledge-Based Systems, McGraw-Hill 1990. find
  8. Lenat, Douglas B., “What AI Can Learn From Romeo & Juliet”, Forbes, July 3, 2019. https://www.forbes.com/sites/cognitiveworld/2019/07/03/what-ai-can-learn-from-romeo--juliet/?sh=4ab750cf1bd0 ↗
  9. Lewis, C. I., and C. H. Langford, Logic, Symbolic and mathematical, Century Company, 1932. find
  10. Marcus, Gary, The Algebraic Mind: Integrating Connectionism and Cognitive Science, MIT Press, 2023. Marcus, Gary, The Next Decade in AI, arXiv, 2020. find
  11. McCarthy, J. and P. J. Hayes, Some Philosophical Problems from the Standpoint of Artificial Intelligence. In: (Meltzer, B. and Michie, D., Eds.) Machine Intelligence, Vol. 4, Edinburgh University Press, Edinburgh, 1969, 463-502. find
  12. Minsky, Marvin (1986). The Society of Mind. New York: Simon & Schuster. find
  13. Olson, Taylor, and Kenneth Forbus. Mitigating Adversarial Norm Training with Moral Axioms. https://www.qrg.northwestern.edu/papers/Files/QRG_Dist_Files/QRG_2023/MitigatingAdversari alNormTrainingWithMoralAxioms.pdf Ribeiro, D. N. and Forbus, K., “Combining Analogy with Language Models for Knowledge… ↗
  14. Zou, Andy, Wang, Zifang,, Kolter, J. Zico and Fredrikson, Matt.,Universal and Transferable Adversarial Attacks on Aligned Language Models, arXiv, 2023.

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/lenat-2023-getting-generative-ai-trustworthy/