🧑‍🔬 AmirAli Bagher Zadeh

AmirAli Bagher Zadeh works on multimodal language, meaning the way spoken words, tone of voice and facial expression carry meaning together, and on how machine learning models can combine these channels. The papers here, from 2018 to 2020, cover sentiment and emotion in video, the design of fusion methods that merge language, acoustic and visual representations, and the computational cost of such methods as more modalities are added.

The most cited of them, Multimodal Language Analysis in the Wild (2018), introduced CMU-MOSEI. It is a large corpus of single-speaker YouTube monologues labelled for sentiment and emotion, with hand-written transcripts aligned to the audio at the phoneme level. The same paper presented the Dynamic Fusion Graph, a fusion module that models interactions among one, two and three modalities. Efficient Low-rank Multimodal Fusion (2018) tackled the exponential growth of tensor-based fusion. It factors the fusion weights into low-rank, modality-specific parts, so the joint output can be computed directly from the unimodal vectors.

The newest paper, Integrating Multimodal Information in Large Pretrained Transformers (2020), turns to text-only pretrained models such as BERT and XLNet. It adds the Multimodal Adaptation Gate, a small component that uses aligned acoustic and visual cues to displace each word's representation inside the model. The base architecture stays intact, and the combined system is fine-tuned end to end.

Bagher Zadeh's papers list Carnegie Mellon University in Pittsburgh as the affiliation from 2018 to 2020. The most frequent co-author is Louis-Philippe Morency, on three of the papers, followed by Paul Pu Liang on two. The work sits mainly under Vision & language, with further papers in Sentiment & emotion and Efficient models & compression.

AmirAli Bagher Zadeh has 3 papers in our library, in 2018–2020, most of them on Vision & language, Sentiment & emotion, Efficient models & compression; often with Louis-Philippe Morency, Paul Pu Liang, Ehsan Hoque.