Monash University
Final_Thesis.pdf (2.85 MB)
Download file

From Autoregressive to Non-Autoregressive: Studies on Text Generation in Neural Sequence Models

Download (2.85 MB)
posted on 2022-04-26, 22:32 authored by SYED NAJAM ABBAS ZAIDI
There is a balancing act between quality and time as we move from autoregressive to non-autoregressive models. Autoregressive models are slower but produce better quality output compared to non-autoregressive models which are faster. Semi-autoregressive models provide the best of both worlds by decoding faster but at the expense of output quality. This thesis, therefore, explores the problem of text generation to improve generation across the three model families. It identifies and addresses various literature gaps such as ineffective decoding methods for autoregressive models, lack of length flexibility in autoregressive and non-autoregressive models, and inferior output quality by non-autoregressive models compared to autoregressive models.


Campus location


Principal supervisor

Gholamreza Haffari

Additional supervisor 1

Trevor Cohn

Additional supervisor 2

Hans De Sterck

Year of Award


Department, School or Centre

Data Science & Artificial Intelligence


Doctor of Philosophy

Degree Type



Faculty of Information Technology