🧑‍🔬 Yingyu Liang

Yingyu Liang's papers here study the theory behind word embeddings: why vectors learned from word co-occurrence statistics behave as they do. The work asks what geometric structure such vectors carry, why low-dimensional inner products approximate co-occurrence measures, and how a single vector can stand for a word with many meanings. The method is to build a probabilistic generative model of text and derive from it the properties of embeddings such as word2vec and GloVe. This places the papers within lexical semantics and word embeddings and word sense disambiguation.

The most cited paper, A Latent Variable Model Approach to PMI-based Word Embeddings (2016), proposes that text is produced by a slowly drifting latent discourse vector. The chance of any given word appearing rises exponentially with how closely its own vector aligns with the current discourse. Under an assumption that word vectors are spread evenly in space, the model accounts for two observations. The pointwise mutual information matrix is well fit by low-dimensional vectors, and analogies can be solved by simple vector arithmetic. The follow-up, Linear Algebraic Structure of Word Senses, with Applications to Polysemy (2018), argues that the vector of an ambiguous word is a weighted nonnegative combination of vectors for its separate senses. It supports this with a Gaussian variant of the random-walk model, in which a word's embedding is a linear transformation of averaged context embeddings.

The later of these papers moves from explaining embeddings in general to the problem of polysemy, asking how the several meanings of a word sit inside its single vector. Liang's current institutions are the University of Wisconsin–Madison and the University of Hong Kong, and earlier bylines include Princeton University. Co-authors on these papers include Sanjeev Arora, Yuanzhi Li, Tengyu Ma and Andrej Risteski.

Yingyu Liang has 2 papers in our library, in 2016–2018, most of them on Word sense disambiguation, Lexical semantics & word embeddings; often with Andrej Risteski, Sanjeev Arora, Tengyu Ma.