🧑‍🔬 Pascal Vincent

Pascal Vincent works in machine learning, with papers here centred on how data can be encoded as learned representations rather than hand-designed features. A recurring question is how models can generalise from limited observations when the space of possible inputs is vast. The answer explored in this work is distributed representations: continuous vectors that place similar items close together. The work spans language models & pretraining and the broader study of deep and unsupervised feature learning.

The most cited paper here is A Neural Probabilistic Language Model (2003). It tackles the problem that most word sequences seen at test time never appear in training. Instead of counting short n-gram fragments, it trains a neural network that jointly learns a feature vector for every word and a next-word probability function built on those vectors. Words with related meanings or grammatical roles thus end up with nearby vectors. The second most cited, Representation Learning: A Review and New Perspectives (2012), is a survey rather than a new model. It argues that good representations should disentangle the hidden factors behind observed data, and it organises the field around general priors such as smoothness, hierarchical concepts and semi-supervised structure.

The 2012 review is the latest of the papers here. It moves from a single modelling problem to a wider view of unsupervised feature learning and deep learning as a whole. Vincent's bylines are mostly from the Université de Montréal, with further affiliations to the Mila - Quebec Artificial Intelligence Institute, the Canadian Institute for Advanced Research and Meta (United States). Co-authors here include Yoshua Bengio and Aaron Courville.

Pascal Vincent has 2 papers in our library, in 2003–2012, most of them on Language models & pretraining; often with Yoshua Bengio, Aaron Courville.