🧑‍🔬 Zhun Liu

Zhun Liu's papers here, from 2018 to 2022, deal with the cost of building neural models that either combine several kinds of signal or grow to very large sizes. One line of work belongs to vision & language and multimodal learning: how to merge separate language, audio and visual representations into a single joint vector without the expense exploding as more modalities are added. The other belongs to language models & pretraining, and specifically to the systems engineering needed to train a generative transformer too large for one GPU's memory. Both lines connect to efficient models & compression.

The most cited work, Efficient Low-rank Multimodal Fusion With Modality-Specific Factors (2018), begins from tensor-based fusion methods. Their outer-product tensors grow exponentially in size and computation with each extra modality, which also raises the risk of overfitting. The paper introduces Low-rank Multimodal Fusion, which decomposes the fusion weights into a few low-rank factors, one set per modality. Since the combined tensor itself breaks down into the unimodal vectors, the fused result can be obtained directly, without ever building the full high-dimensional object.

The later paper, Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B (2022), shifts from fusion to scale and to text generation. It describes a joint Microsoft–NVIDIA project that trained an autoregressive, dense transformer of 530 billion parameters reading text left to right. The team joined Megatron-LM with DeepSpeed in a 3D-parallel design: model replicas sharded with ZeRO, a pipeline split across stages, and tensor slicing, with the tensor-parallel part confined to a single DGX node where NVLink bandwidth is high.

Co-authors on these papers include Louis-Philippe Morency, Paul Pu Liang, Mostofa Patwary and Samyam Rajbhandari. Recent bylines list Zhejiang University in Hangzhou.

Zhun Liu has 2 papers in our library, in 2018–2022, most of them on Language models & pretraining, Text generation, Vision & language; often with AmirAli Bagher Zadeh, Jared Casper, Louis-Philippe Morency.