Elamur Research⌕
☰
home / papers / 2022 / Modeling the Ordering of English Adjectives using Collaborative Filtering

Modeling the Ordering of English Adjectives using Collaborative Filtering

Paper · 2022 · Sagar Indurkhya

Summary

The paper asks how a learner could pick up productive knowledge of English adjective order (why "big red ball" sounds right and "red big ball" does not) using only counts of adjective pairs, without hand-built semantic classes. The author treats ordering as a recommender problem. The first adjective of a bigram plays the role of a "user", the second plays an "item", and an adjective-ordering matrix built from Google Books NGram bigram counts serves as the rating matrix. A binomial test decides which order is preferred for each observed pair. Two model-based Collaborative Filtering methods, SVD and NMF, are trained on subsets of this matrix and then asked to predict the order of held-out pairs. Both do well. On all eight matrix variants they beat a baseline that ranks adjectives along a single linear scale. This suggests that real ordering data contains structure, including cycles, that a strict class hierarchy cannot capture. The method generalizes to unseen pairs and needs no multilingual embeddings or context words, which makes it a minimal-assumption baseline for later work.

Key points

  • Data and labels: the data are adjective bigrams from the Google Books NGram database. For each pair, a binomial test against a 1:1 null decides whether one order is significantly preferred, following Shaw and Hatzivassiloglou (1999).
  • Dataset variants: eight adjective-ordering matrices are built. They cross "full" versus "acyclic" versions with subsets named all, noun, verb and pure. The paper documents ordering cycles that violate transitivity.
  • Models and baseline: SVD and NMF are compared with a γ baseline. The baseline turns the difference of two per-adjective rank scores h into a value in [1, 2]. It does better on acyclic data, but both CF models beat it on every matrix.
  • FAAP analysis: FAAP is the fraction of "adjacent adjective pairs" present in the training data. Correctly classified test pairs have lower mean FAAP than misclassified ones. Welch's t-test gives p ≤ 3.5×10⁻⁹ in every case; for NMF on the all set the means are 0.237 vs 0.253, with t = 28.97.
  • Link to performance: across (model, dataset) pairs, the t-statistic tracks the best F1-score, with R² = 0.61 (p = 3.7×10⁻⁴).

Interesting ideas

  • I1. Recasting a word-order preference as a user–item rating matrix lets low-rank factorization fill in pairs never seen together, by borrowing evidence from how each word behaves with others.
  • I2. Turning raw A-B vs B-A counts into labels only when a binomial test rejects the 1:1 hypothesis is a simple way to leave genuinely free pairs unlabeled rather than forcing a preference.
  • I3. The paper explicitly hunts for cycles in pairwise preferences and keeps a separate acyclic variant. This shows how much a linear-hierarchy assumption costs on real data.
  • I4. A rank-difference baseline that matches the hierarchy theory acts as a negative control: if the learned model cannot beat it, the extra machinery is not justified.
  • I5. The FAAP diagnostic measures how much neighbouring training evidence a test pair has, then checks whether errors concentrate where that evidence pattern differs.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (back): when generating noun phrases with several adjectives, a small factorized ordering model could choose the order for unseen adjective pairs instead of falling back on a fixed class list.
  • I2 (learn): the same significance gate could filter silver data from witness parsers so that only statistically clear preferences become training targets.
  • I3 (world): before compiling ordering or scale knowledge into the ontology, checking for preference cycles would show where a total order is a false simplification.
  • I4 (learn): evaluations of learned components could routinely include such theory-driven cheap baselines alongside the main metric.
  • I5 (tree): a similar neighbourhood-coverage statistic over treebank head-dependent pairs could explain which parser errors come from sparse evidence rather than from model flaws.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Fields, topics and concepts

📚 Shelves 2The field of science this work belongs to (OpenAlex fields).Computer Science · 18,614 worksArtificial Intelligence · 17,965 works

🧭 Topics 1Research subjects and languages the work is about.English · 2,379 works

Cite

Cite as Sagar Indurkhya. 2022. Modeling the Ordering of English Adjectives using Collaborative Filtering. Proceedings of the 5th International Conference on Natural Language and Speech Processing (ICNLSP 2022).

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{indurkhya2022modeling,
  title = {{Modeling the Ordering of English Adjectives using Collaborative Filtering}},
  author = {Sagar Indurkhya},
  booktitle = {Proceedings of the 5th International Conference on Natural Language and Speech Processing (ICNLSP 2022)},
  year = {2022},
  url = {https://aclanthology.org/2022.icnlsp-1.18/}
}

References

57 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Gediminas Adomavicius and Alexander Tuzhilin. 2005. Toward the next generation of recommender systems: A survey of the state-of-the-art and possible extensions. IEEE transactions on knowledge and data engineering, 17(6):734–749. find
  2. Artemis Alexiadou. 2001. Adjective syntax and noun raising: Word order asymmetries in the dp as the result of adjective distribution. Studia linguistica, 55(3):217–248. find
  3. Mark Baker and William Croft. 2017. Lexical categories: Legacy, lacuna, and opportunity for functionalists and formalists. Annual Review of Linguistics, 3:179–197. find
  4. Mark C Baker and Mark Cleland Baker. 2003. Lexical categories: Verbs, nouns and adjectives, volume 102. Cambridge University Press. find
  5. Galia Bar-Sever, Rachael Lee, Gregory Scontras, and Lisa Pearl. 2018. Little lexical learners: Quantitatively assessing the development of adjective ordering preferences. In Proceedings of the 42nd annual Boston university conference on language development, pages 58–71. Cascadilla Press Somerville, MA. find
  6. Ingrid Lossius Falkum. 2015. The how and why of polysemy: A pragmatic account. Lingua, 157:83–99. find
  7. Tom Fawcett. 2006. An introduction to roc analysis. Pattern recognition letters, 27(8):861–874. find
  8. Didier L Goyvaerts. 1968. An introductory study on the ordering of a string of adjectives in present-day english. Philologica Pragensia, 11(1):12–28. find
  9. Michael Hahn, Judith Degen, Noah D Goodman, Dan Jurafsky, and Richard Futrell. 2018. An information-theoretic explanation of adjective ordering preferences. In CogSci. find
  10. Jonathan L Herlocker, Joseph A Konstan, Loren G Terveen, and John T Riedl. 2004. Evaluating collaborative filtering recommender systems. ACM Transactions on Information Systems (TOIS), 22(1):5–53. find
  11. Felix Hill. 2012. Beauty before age? applying subjectivity to automatic english adjective ordering. In Proceedings of the NAACL HLT 2012 student research workshop, pages 11–16.
  12. Thomas Hofmann. 2004. Latent semantic models for collaborative filtering. ACM Transactions on Information Systems (TOIS), 22(1):89–115. find
  13. Judy B Bernstein. 1993. Topics in the syntax of nominal structure across Romance. City University of New York. find
  14. Nicolas Hug. 2020. Surprise: A python library for recommender systems. Journal of Open Source Software, 5(52):2174. find
  15. Jesús Bobadilla, Fernando Ortega, Antonio Hernando, and Abraham Gutiérrez. 2013. Recommender systems survey. Knowledge-based systems, 46:109–132. find
  16. Sagar Indurkhya. 2020. Inferring minimalist grammars with an SMT-solver. Proceedings of the Society for Computation in Linguistics, 3(1):476–479. find
  17. Robin Burke. 2002. Hybrid recommender systems: Survey and experiments. User modeling and user-adapted interaction, 12(4):331–370. find
  18. Sagar Indurkhya. 2021. Using collaborative filtering to model argument selection. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 629– 639. find
  19. Robin Burke. 2007. Hybrid web recommender systems. The adaptive web, pages 377–408. find
  20. Guglielmo Cinque. 1994. On the evidence for partial n-movement in the romance dp. In Paths towards universal grammar. Studies in honor of Richard S. Kayne. Georgetown University Press. find
  21. Guglielmo Cinque. 2010. The syntax of adjectives: A comparative study, volume 57. MIT press. find
  22. Guglielmo Cinque. 2014. The semantic classification of adjectives. a view from syntax. Studies in Chinese Linguistics, 35.(1) 3-32. find
  23. Guglielmo Cinque and Luigi Rizzi. 2012. The cartography of syntactic structures. In The Oxford handbook of linguistic analysis, pages 51–65. Oxford University Press. find
  24. Robert M. W. Dixon. 1982. Where have all the adjectives gone? Berlin: Mouton de Gruyter. find
  25. Peter Eades, Xuemin Lin, and William F Smyth. 1993. A fast and effective heuristic for the feedback arc set problem. Information Processing Letters, 47(6):319– 323. find
  26. Sagar Indurkhya. 2022. Incremental acquisition of a Minimalist Grammar using an SMT-solver. In Proceedings of the Society for Computation in Linguistics 2022, pages 212–216, online. Association for Computational Linguistics. find
  27. Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Matrix factorization techniques for recommender systems. Computer, 42(8):30–37. find
  28. Christopher Laenzlinger. 2005. French adjective ordering: Perspectives on dp-internal movement types. Lingua, 115(5):645–689. find
  29. Randy J LaPolla and Chenglong Huang. 2004. Adjectives in qiang. Adjective classes: A cross-linguistic typology, pages 306–322. find
  30. Jun Yen Leung, Guy Emerson, and Ryan Cotterell. 2020. Investigating Cross-Linguistic Adjective Ordering Tendencies with a Latent-Variable Model. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4016–4028, Online. Association for Computational Linguistics. find
  31. Yuri Lin, Jean-Baptiste Michel, Erez Aiden Lieberman, Jon Orwant, Will Brockman, and Slav Petrov. 2012. Syntactic annotations for the google books ngram corpus. In Proceedings of the ACL 2012 system demonstrations, pages 169–174. find
  32. Linyuan Lü, Matúš Medo, Chi Ho Yeung, Yi-Cheng Zhang, Zi-Ke Zhang, and Tao Zhou. 2012. Recommender systems. Physics Reports, 519(1):1–49. Recommender Systems. find
  33. Brian MacWhinney. 2000. The CHILDES project: The database, volume 2. Psychology Press. find
  34. Gary-John Scott. 2002. Stacked adjectival modification. Functional Structure in DP and IP: The Cartography of Syntactic Structures, vol, 1:91–122. find
  35. James Shaw and Vasileios Hatzivassiloglou. 1999. Ordering among premodifiers. In Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics, pages 135–143. find
  36. Ur Shlonsky. 2010. The cartographic enterprise in syntax. Language and linguistics compass, 4(6):417– 429. find
  37. Brian MacWhinney and Catherine Snow. 1985. The child language data exchange system. Journal of child language, 12(2):271–295. find
  38. Richard Sproat and Chilin Shih. 1991. The cross-linguistic distribution of adjective ordering restrictions. In Interdisciplinary approaches to language, pages 565–593. Springer. find
  39. Robert Malouf. 2000. The order of prenominal adjectives in natural language generation. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 85–92. find
  40. Xiaoyuan Su and Taghi M Khoshgoftaar. 2009. A survey of collaborative filtering techniques. Advances in artificial intelligence, 2009. find
  41. Jean-Baptiste Michel, Yuan Kui Shen, Aviva Presser Aiden, Adrian Veres, Matthew K Gray, Google Books Team, Joseph P Pickett, Dale Hoiberg, Dan Clancy, Peter Norvig, et al. 2011. Quantitative analysis of culture using millions of digitized books. science, 331(6014):176–182. find
  42. Peter Svenonius. 1994. C-selection as feature-checking. Studia linguistica, 48(2):133–155. find
  43. Peter Svenonius. 2008. The position of adjectives and other phrasal modifiers in the decomposition of dp. In Adjectives and Adverbs: Syntax, Semantics, and Discourse. find
  44. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39–41. ↗
  45. John R Taylor. 2003. Polysemy’s paradoxes. Language sciences, 25(6):637–655. find
  46. George A Miller, Richard Beckwith, Christiane Fellbaum, Derek Gross, and Katherine J Miller. 1990. Introduction to wordnet: An on-line lexical database. International journal of lexicography, 3(4):235–244. find
  47. Nitzan Trainin and Einat Shetreet. 2021. It’s a dotted blue big star: on adjective ordering in a post-nominal language. Language, Cognition and Neuroscience, 36(3):320–341. find
  48. Luigi Rizzi and Guglielmo Cinque. 2016. Functional categories and syntactic theory. Annual Review of Linguistics, 2:139–163. find
  49. Andreas Trotzke and Eva Wittenberg. 2019. Long-standing issues in adjective order and corpus evidence for a multifactorial approach. Linguistics, 57(2):273– 282. find
  50. Alessandro Sanchez, Stephan C Meylan, Mika Braginsky, Kyle E MacDonald, Daniel Yurovsky, and Michael C Frank. 2019. childes-db: A flexible and reproducible interface to the child language data exchange system. Behavior research methods, 51(4):1928–1941. find
  51. Badrul Sarwar, George Karypis, Joseph Konstan, and John Riedl. 2001. Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web, pages 285–295. find
  52. J Ben Schafer, Dan Frankowski, Jon Herlocker, and Shilad Sen. 2007. Collaborative filtering recommender systems. In The adaptive web, pages 291– 324. Springer. find
  53. Gregory Scontras, Judith Degen, and Noah D Goodman. 2017. Subjectivity predicts adjective ordering preferences. Open Mind, 1(1):53–66. find
  54. Gregory Scontras, Judith Degen, and Noah D Goodman. 2019. On the grammatical source of adjective ordering preferences. Semantics and Pragmatics, 12:7. find
  55. Zeno Vendler. 1968. Adjectives and nominalizations. find
  56. Bernard L Welch. 1947. The generalization of Student’s ’problem when several different population varlances are involved. Biometrika, 34(1-2):28–35. find
  57. Charles Yang. 2011. Computational models of language acquisition. In Handbook of generative approaches to language acquisition, pages 119–154. Springer. find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/indurkhya-2022-modeling-ordering-english-adjectives-collaborative/