Final_Thesis.pdf (2.85 MB)
Download file

From Autoregressive to Non-Autoregressive: Studies on Text Generation in Neural Sequence Models

Download (2.85 MB)
thesis
posted on 26.04.2022, 22:32 authored by SYED NAJAM ABBAS ZAIDI
There is a balancing act between quality and time as we move from autoregressive to non-autoregressive models. Autoregressive models are slower but produce better quality output compared to non-autoregressive models which are faster. Semi-autoregressive models provide the best of both worlds by decoding faster but at the expense of output quality. This thesis, therefore, explores the problem of text generation to improve generation across the three model families. It identifies and addresses various literature gaps such as ineffective decoding methods for autoregressive models, lack of length flexibility in autoregressive and non-autoregressive models, and inferior output quality by non-autoregressive models compared to autoregressive models.

History

Campus location

Australia

Principal supervisor

Gholamreza Haffari

Additional supervisor 1

Trevor Cohn

Additional supervisor 2

Hans De Sterck

Year of Award

2022

Department, School or Centre

Data Science & Artificial Intelligence

Course

Doctor of Philosophy

Degree Type

DOCTORATE

Faculty

Faculty of Information Technology