Elamur Research⌕
☰
home / papers / 2011 / Perception of Personality and Naturalness through Dialogues by Native Speakers of American English and Arabic

Perception of Personality and Naturalness through Dialogues by Native Speakers of American English and Arabic

Paper · 2011 · Maxim Makatchev and Reid Simmons

Summary

Dialogue systems used across cultures risk pragmatic failure: listeners may read attitudes into English phrasing that the speaker never intended. Makatchev and Simmons ask whether English utterances varying in verbosity, hedging, lexical/syntactic alignment and formality signal the same personality traits and naturalness to native speakers of American English and of Arabic. Their setting is an information-giving robot receptionist deployed in Qatar and the US. They hand-built short text dialogues covering four typical receptionist dialogue acts. Each linguistic dimension was set to negative, neutral or positive valence. Workers on Amazon's Mechanical Turk then rated the receptionist's turns, either on Big Five traits or on naturalness. Linear mixed-effects modelling found that every dimension relates to every trait, and to naturalness, in both groups. It also found culture-dependent interactions. Most strikingly, Americans judged formal speech unnatural, while Arabic speakers did not. The study gives generation and interpretation modules a concrete, crowd-validated map of which surface choices shift perceived persona differently by audience.

Key points

  • Stimulus design: four dialogue acts (greeting, question–answer, disagreement, apology) were crossed with four dimensions at three valences. A single shared neutral dialogue per act served as the common baseline.
  • Measures:
    • Personality was rated with TIPI (Big Five).
    • Naturalness was rated on a 7-point Likert item.
    • The two scales ran as separate parallel studies to avoid scale interference.
  • Participants:
    • 155 Arabic speakers, mostly from Egypt (56), Morocco (45), Jordan (32) and the UAE (13).
    • 166 US English speakers.
    • Gender was skewed in opposite directions: 100 male / 55 female in the Arabic group, 63 male / 103 female in the US group.
    • Repeat participation was blocked through IP and account checks, and hidden questions tested native-language fluency.
  • Analysis: LME models with ANOVA-based model selection were followed by post-hoc paired t-tests between valences. Every trait, and naturalness, showed a significant three-way interaction of valence × native language × dialogue act for at least one dimension.
  • Cross-cultural findings:
    • Formality was unnatural to Americans in greetings, question–answer and disagreements.
    • For Arabic speakers only, formality signalled openness (in disagreements) and conscientiousness (in apologies).
    • Hedged apologies read as agreeable only to Americans.
    • Extraversion and emotional stability showed no qualitative cultural difference.

Interesting ideas

  • I1. To compare groups that may use rating scales differently, the authors avoid raw scores. They compare only the rank order of valences and whether a score sits significantly above or below the scale midpoint, which guards against culture-specific scoring habits.
  • I2. Each linguistic dimension is a controlled three-level knob around a shared neutral version of the same dialogue, so any perceived change is attributable to one manipulated feature.
  • I3. The effect of a surface feature is judged per dialogue act rather than globally; for example, hedging matters for apologies specifically.
  • I4. Crowd annotations are protected by planted hidden questions that verify claimed competence and by duplicate-participant checks.
  • I5. Alignment, meaning reuse of the interlocutor's words and syntax, is treated as a measurable signal of cooperativeness and perspective-taking.

For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

  • I1 (learn): When comparing parsers or teacher tiers whose confidence scores are calibrated differently, compare rankings and thresholded decisions rather than raw numbers.
  • I2 (back): Generation could expose hedging, formality, verbosity and alignment as explicit, separately tunable parameters applied to one neutral realization of a meaning graph.
  • I3 (frames): The speech-act layer in prag could condition its interpretation of hedges and politeness markers on the detected act type instead of using one fixed reading.
  • I4 (learn): Silver-data pipelines and LLM teachers can be audited with embedded known-answer probes, acting as negative controls, to detect unreliable annotators.
  • I5 (frames): Lexical and syntactic overlap between turns could be computed from UD trees as an explainable feature for detecting cooperative versus confrontational stance.

Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.

 Institutions

🎓Carnegie Mellon UniversityPittsburgh · United States539 works · 179,533 citations

 Fields, topics and concepts

📚 Shelves 4The field of science this work belongs to (OpenAlex fields).Computer Science · 16,667 worksArtificial Intelligence · 16,194 worksArts and Humanities · 16,039 worksLanguage and Linguistics · 15,891 works

🧭 Topics 3Research subjects and languages the work is about.English · 2,263 worksSemitic languages · 2,908 worksDialogue systems · 624 works

Cite

Cite as Maxim Makatchev and Reid Simmons. 2011. Perception of Personality and Naturalness through Dialogues by Native Speakers of American English and Arabic. Proceedings of the SIGDIAL 2011 Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

@inproceedings{makatchev2011perception,
  title = {{Perception of Personality and Naturalness through Dialogues by Native Speakers of American English and Arabic}},
  author = {Maxim Makatchev and Reid Simmons},
  booktitle = {Proceedings of the SIGDIAL 2011 Conference},
  year = {2011},
  url = {https://aclanthology.org/W11-2032/}
}

References

35 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

  1. Michael Agar. 1994. Language shock: Understanding the culture of conversation. William Morrow, New York. find
  2. Maria Jose Coperias Aguilar. 1998. Intercultural (mis)communication: The influence of L1 and C1 on L2 and C2. A tentative approach to textbooks. Cuadernos de Filologı́a Inglesa, 7(1):99–113. find
  3. Kathleen Bardovi-Harlig, Marda Rose, and Edelmira L. Nickels. 2007. The use of conventional expressions of thanking, apologizing, and refusing. In Proceedings of the 2007 Second Language Research Forum, pages 113–130. find
  4. Charles F. Bond, Adnan Omar, Adnan Mahmoud, and Richard Neal Bonser. 1990. Lie detection across cultures. Journal of Nonverbal Behavior, 14:189–204. ↗
  5. P. Brown and S. C. Levinson. 1987. Politeness: Some universals in language usage. Cambridge University Press, Cambridge.
  6. F. Burkhardt, N. Audibert, L. Malatesta, O. Trk, Arslan, L., and V Auberge. 2006. Emotional prosody—does culture make a difference? In Proc. Speech Prosody. find
  7. Eirlys E. Davies. 1987. A contrastive approach to the analysis of politeness formulas. Applied Linguistics, 8(1):75–88. ↗
  8. Said Hassan Farahat. 2009. Politeness phenomena in Palestinian Arabic and Australian English: A cross-cultural study of selected contemporary plays (PhD thesis). Australian Catholic University, Australia. ↗
  9. Ellen Feghali. 1997. Arab cultural communication patterns. International Journal of Intercultural Relations, 21(3):345–378. ↗
  10. A. Furnham. 1990. Language and personality. In H. Giles and W. Robinson, editors, Handbook of Language and Social Psychology, pages 73–95. Wiley. find
  11. Mohammed Ghawi. 1993. Pragmatic transfer in Arabic learners of English. El Two Talk, 1(1):39–52. find
  12. A. Gill and J. Oberlander. 2002. aking care of the linguistic features of extraversion. In Proceedings of the 24th Annual Conference of the Cognitive Science Society, pages 363–368. find
  13. A. Gill, A. Harrison, and J. Oberlander. 2004. Inter-personality: Individual differences and interpersonal priming. In Proceedings of the 26th Annual Conference of the Cognitive Science Society, pages 464–469. find
  14. E. S. Glenn, D. Witmeyer, and K. A. Stevenson. 1977. Cultural styles of persuasion. International Journal of Intercultural Relations, 1(3):52–66. ↗
  15. Samuel D. Gosling, Peter J. Rentfrow, and Jr. William B. Swann. 2003. A very brief measure of the Big-Five personality domains. Journal of Research in Personality, 37:504–528. ↗
  16. W. B. Gudykunst and S. Ting-Toomey. 1988. Culture and interpersonal communication. Sage, Newbury Park, CA. find
  17. S. Haywood, M. Pickering, and H. Branigan. 2003. Cooperation and co-ordination in the production of noun phrases. In Proceedings of the 25th Annual Conference of the Cognitive Science Society, pages 533–538. find
  18. Michael Huspek. 1989. Linguistic variability and power: An analysis of you know/I think variation in working-class speech. Journal of Pragmatics, 13(5):661 – 683. ↗
  19. Oliver P. John and Sanjay Srivastava. 1999. The Big Five trait taxonomy: History, measurement, and theoretical perspectives. In Lawrence A. Pervin and Oliver P. John, editors, Handbook of Personality: Theory and Research, pages 102–138. find
  20. Robin Lakoff. 1973. Language and woman’s place. Language in Society, 2(1):45–80. ↗
  21. Francois Mairesse and Marilyn Walker. 2008. Trainable generation of big-five personality styles through data-driven parameter estimation. In Proc. of 46th Annual Meeting of the Association for Computational Linguistics (ACL). find
  22. F. Mairesse, M. A. Walker, M. R. Mehl, and R. K. Moore. 2007. Using linguistic cues for the automatic recognition of personality in conversation and text. Journal of Artificial Intelligence Research, 30:457–500. ↗
  23. Maxim Makatchev, Min Kyung Lee, and Reid Simmons. 2009. Relating initial turns of human-robot dialogues to discourse. In Proc. of the Int. Conf. on Human-Robot Interaction (HRI), pages 321–322. ACM. find
  24. Maxim Makatchev, Imran Aslam Fanaswala, Ameer Ayman Abdulsalam, Brett Browning, Wael Mahmoud Gazzawi, Majd Sakr, and Reid Simmons. 2010. Dialogue patterns of an arabic robot receptionist. In Proc. of the Int. Conf. on Human-Robot Interaction (HRI), pages 167–168. ACM. ↗
  25. M. McPherson, L. Smith-Lovin, and J. M. Cook. 2000. What is a language community? American Journal of Political Science, 44(1):142–155. find
  26. Clifford Nass, Y. Moon, B. Fogg, and B. Reeves. 1995. Can computer personalities be human personalities? Journal of Human-Computer Studies, 43:223–239. ↗
  27. Gaylel Nelson, Mahmoud Al-Batal, and Erin Echols. 1996. Arabic and english compliment responses: Potential for pragmatic failure. Applied Linguistics, 17(4):411–432. ↗
  28. R. Patai. 1983. The Arab mind. Charles Scribner’s Sons, New York. find
  29. J. C. Pinheiro and D. M. Bates. 2000. Mixed-Effects Models in S and S-PLUS. Springer. ↗
  30. Joel Ross, Lilly Irani, M. Six Silberman, Andrew Zaldivar, and Bill Tomlinson. 2010. Who are the crowdworkers?: shifting demographics in mechanical turk. In Proceedings of the 28th of the international conference extended abstracts on Human factors in computing systems, CHI EA ’10, pages 2863–2872, New York, NY, USA. ACM. find
  31. Emanuel A. Schegloff and Harvey Sacks. 1973. Opening up closings. Semiotica, 8(4):289–327. ↗
  32. Klaus R. Scherer. 1972. Judging personality from voice: A cross-cultural approach to an old issue in interpersonal perception. Journal of Personality, 40:191–210. find
  33. John Searle. 1969. Speech acts: An essay in the philosophy of language. Cambridge University Press. ↗
  34. Jenny Thomas. 1983. Cross-cultural pragmatic failure. Applied Linguistics, 4(2):91–112. ↗
  35. Jenny Thomas. 1984. Cross-cultural discourse as ’unequal encounter’: Towards a pragmatic analysis. Applied Linguistics, 5(3):226–235. Greeting Question-Answer Disagreement A: Good morning. B: What’s up? Need anything? A: Could you tell me where the library is? B: Just go to the end of the hallway, you can’t miss it. A: Could you tell me where the library is? B: Go to the second floor. A: I thought it was on the first floor. B: No, honey, there is none on the first floor. A: Good morning. B: Good morning. How may I help you? A: Could you tell me where the library is? B: It’s at the end of the hallway on your left. A: Could you tell me where the library is? B: It’s on the second floor. A: I thought it was on the first floor. B: No, there is no library on the first floor. A: Good morning. B: Good morning, sir (madam). Would you allow me to help you with anything? A: Could you tell me where the library is? B: Kindly follow this hallway and you will encounter the entrance on your left. A: Could you tell me where the library is? B: Yes, you may find the library on the second floor. A: I thought it was on the first floor. B: I am afraid that is not correct, there is no library on the first floor. Table 2: Stimuli that correspond to negative (top row), neutral (middle row), and positive (bottom row) formality. American English, formality, naturalness ** ** ** ** ** ** ** ** greeting qa disagree apology American English, formality, conscienciousness * * ** ** * greeting qa disagree apology American English, hedging, agreeableness ** ** ** ** ** ** * ** greeting qa disagree apology find

Updated

↑ back to top

This page in Markdown → https://md.research.elamur.ai/free/en/papers/makatchev-2011-perception-personality-naturalness-through-dialogues/